Our exposition follows the axiomatic view of probability, promoted by A. Kolmogorov. Alternative viewpoints are available, including the frequency viewpoint ($\P(A)$ is the frequency of $A$ occurring in a long sequence of repetitive experiments) and the subjective viewpoint ($\P(A)$ measures the belief that $A$ will occur).
More information on the basic concepts of probability is available in nearly any probability textbook. One example is Feller's first volume (Feller, 1968), which inspired a generation of probabilists as well several of the examples in this chapter. Examples of books with rigorous coverage of probability based on measure theory are (Breiman, 1992), (Billingsley, 1995), and (Resnick, 1999). Elementary exposition that avoids measure theory is available in most undergraduate probability textbooks, such as (Weiss, 2005) or (Dasgupta, 2010). More information on combinatorics is available in combinatorics textbooks, for example (Rosen, 2006).