.

# Probability

## The Analysis of Data, Volume 1

1. Basic Definitions
• Sample Space and Events
• The Probability Function
• The Classical Probability Model on Finite Spaces
• The Classical Probability Model on Continuous Spaces
• Conditional Probability and Independence
• Basic Combinatorics for Probability
• Probability and Measure Theory*
• Notes
• Exercises
2. Random Variables
• Basic Definitions
• Functions of a Random Variable
• Discrete g(X)
• Continuous g(X)
• Expectation and Variance
• Moment and Moment Generating Function
• Random Variables and Measurable Functions*
• Notes
• Exercises
3. Important Random Variables
• The Bernoulli Trial Distribution
• The Binomial Distribution
• The Geometric Distribution
• The Hypergeometric Distribution
• The Negative Binomial Distribution
• The Poisson Distribution
• The Uniform Distribution
• The Exponential Distribution
• The Gaussian Distribution
• The Gamma and Chi-Squared Distribution
• The t Distribution
• The Beta Distribution
• Mixture Distributions
• The Empirical Distribution
• The Smoothed Empirical Distribution
• Notes
• Exercises
4. Random Vectors
• Basic Definitions
• Joint PMF, PDF, and CDF
• Marginal Random Vectors
• Functions of a Random Vector
• Conditional Probabilities and Random Vectors
• Moments
• Conditional Expectations
• Moment Generating Function
• Random Vectors and Independent Sigma-Algebras*
• Notes
• Exercises
5. Important Random Vectors
• The Multinomial Random Vector
• The Multivariate Normal Random Vector
• The Dirichlet Random Vector
• Mixture Random Vectors
• The Exponential Family Random Vector
• Notes
• Exercises
6. Random Processes
• Basic Definitions
• Random Processes and Marginal Distributions
• Moments
• One Dimensional Random Walk
• Random Processes and Measure Theory*
• The Borel-Cantelli Lemmas and the Zero-One Law*
• Notes
• Exercises
7. Important Random Processes
• Markov Chains
• Basic Definitions
• Examples
• Transience, Persistence, and Irreducibility
• Periodicity in Markov Chains
• The Stationary Distribution
• Poisson Processes
• Postulates and Differential Equation
• Relationship to Poisson Distribution
• Relationship to Exponential Distribution
• Relationship to Binomial Distribution
• Relationship to Uniform Distribution
• Gaussian Processes
• The Wiener Process
• Notes
• Exercises
8. Limit Theorems
• Modes of Stochastic Convergence
• Relationships Between the Modes of Convergences
• Dominated Convergence Theorem for Random Vectors*
• Scheffe's Theorem
• The Portmanteau Theorem
• The Law of Large Numbers
• The Characteristic Function*
• Levy's Continuity Theorem and the Cramer-Wold Device*
• The Central Limit Theorem
• The Continuous Mapping Theorem
• Slutsky's Theorem
• Notes
• Exercises

### Mathematical Prerequisites

1. Set Theory
• Basic Definitions
• Functions
• Cardinality
• Limits of Sets
• Notes
• Exercises
2. Metric Spaces
• Basic Definitions
• Limits
• Continuity
• The Euclidean Space
• Growth of Functions
• Notes
• Exercises
3. Linear Algebra
• Basic Definitions
• Rank
• Eigenvalues, Determinant, and Trace
• Positive Semi-Definite Matrices
• Singular Value Decomposition
• Notes
• Exercises
4. Differentiation
• Unvivariate Differentiation
• Taylor Expansion and Power Series
• Notes
• Exercises
5. Measure Theory*
• Sigma Algebras*
• The Measure Function*
• Caratheodory's Extension Theorem*
• Dynkin's Theorem*
• Outer Measure*
• The Extension Theorem*
• Independent Sigma Algebras*
• Important Measure Functions*
• Discrete Measure Functions
• The Lebesgue Measure
• The Lebesgue-Stieltjes Measure
• Measurability of Functions
• Notes
• Exercises
6. Integration
• Riemann Integral
• Relationship between Integration and Differentiation
• The Lebesgue Integral*
• Relation between the Riemann and the Lebesgue Integrals*
• Transformed Measures*
• Product Measures*
• Important Measure Functions*
• Integration over Product Spaces*
• The Lebesgue Measure over Multivariate Euclidean Spaces
• Multivariate Differentiation and Integration
• Notes
• Exercises