.

Probability

The Analysis of Data, Volume 1

Table of Contents

1. Basic Definitions
   • Sample Space and Events
   • The Probability Function
   • The Classical Probability Model on Finite Spaces
   • The Classical Probability Model on Continuous Spaces
   • Conditional Probability and Independence
   • Basic Combinatorics for Probability
   • Probability and Measure Theory*
   • Notes
   • Exercises
2. Random Variables
   • Basic Definitions
   • Functions of a Random Variable
     • Discrete g(X)
     • Continuous g(X)
   • Expectation and Variance
   • Moment and Moment Generating Function
   • Random Variables and Measurable Functions*
   • Notes
   • Exercises
3. Important Random Variables
   • The Bernoulli Trial Distribution
   • The Binomial Distribution
   • The Geometric Distribution
   • The Hypergeometric Distribution
   • The Negative Binomial Distribution
   • The Poisson Distribution
   • The Uniform Distribution
   • The Exponential Distribution
   • The Gaussian Distribution
   • The Gamma and Chi-Squared Distribution
   • The t Distribution
   • The Beta Distribution
   • Mixture Distributions
   • The Empirical Distribution
   • The Smoothed Empirical Distribution
   • Notes
   • Exercises
4. Random Vectors
   • Basic Definitions
   • Joint PMF, PDF, and CDF
   • Marginal Random Vectors
   • Functions of a Random Vector
   • Conditional Probabilities and Random Vectors
   • Moments
   • Conditional Expectations
   • Moment Generating Function
   • Random Vectors and Independent Sigma-Algebras*
   • Notes
   • Exercises
5. Important Random Vectors
   • The Multinomial Random Vector
   • The Multivariate Normal Random Vector
   • The Dirichlet Random Vector
   • Mixture Random Vectors
   • The Exponential Family Random Vector
   • Notes
   • Exercises
6. Random Processes
   • Basic Definitions
   • Random Processes and Marginal Distributions
   • Moments
   • One Dimensional Random Walk
   • Random Processes and Measure Theory*
   • The Borel-Cantelli Lemmas and the Zero-One Law*
   • Notes
   • Exercises
7. Important Random Processes
   • Markov Chains
     • Basic Definitions
     • Examples
     • Transience, Persistence, and Irreducibility
     • Periodicity in Markov Chains
     • The Stationary Distribution
   • Poisson Processes
     • Postulates and Differential Equation
     • Relationship to Poisson Distribution
     • Relationship to Exponential Distribution
     • Relationship to Binomial Distribution
     • Relationship to Uniform Distribution
   • Gaussian Processes
     • The Wiener Process
   • Notes
   • Exercises
8. Limit Theorems
   • Modes of Stochastic Convergence
   • Relationships Between the Modes of Convergences
   • Dominated Convergence Theorem for Random Vectors*
   • Scheffe's Theorem
   • The Portmanteau Theorem
   • The Law of Large Numbers
   • The Characteristic Function*
   • Levy's Continuity Theorem and the Cramer-Wold Device*
   • The Central Limit Theorem
   • The Continuous Mapping Theorem
   • Slutsky's Theorem
   • Notes
   • Exercises

Mathematical Prerequisites

1. Set Theory
   • Basic Definitions
   • Functions
   • Cardinality
   • Limits of Sets
   • Notes
   • Exercises
2. Metric Spaces
   • Basic Definitions
   • Limits
   • Continuity
   • The Euclidean Space
   • Growth of Functions
   • Notes
   • Exercises
3. Linear Algebra
   • Basic Definitions
   • Rank
   • Eigenvalues, Determinant, and Trace
   • Positive Semi-Definite Matrices
   • Singular Value Decomposition
   • Notes
   • Exercises
4. Differentiation
   • Unvivariate Differentiation
   • Taylor Expansion and Power Series
   • Notes
   • Exercises
5. Measure Theory*
   • Sigma Algebras*
   • The Measure Function*
   • Caratheodory's Extension Theorem*
     • Dynkin's Theorem*
     • Outer Measure*
     • The Extension Theorem*
   • Independent Sigma Algebras*
   • Important Measure Functions*
     • Discrete Measure Functions
     • The Lebesgue Measure
     • The Lebesgue-Stieltjes Measure
   • Measurability of Functions
   • Notes
   • Exercises
6. Integration
   • Riemann Integral
   • Relationship between Integration and Differentiation
   • The Lebesgue Integral*
     • Relation between the Riemann and the Lebesgue Integrals*
     • Transformed Measures*
   • Product Measures*
   • Important Measure Functions*
   • Integration over Product Spaces*
     • The Lebesgue Measure over Multivariate Euclidean Spaces
   • Multivariate Differentiation and Integration
   • Notes
   • Exercises