Contents
1 Introduction to Probability 1
1.1 The History of Probability 1
1.2 Interpretations of Probability 2
1.3 Experiments and Events 5
1.4 Set Theory 6
1.5 The Definition of Probability 16
1.6 Finite Sample Spaces 22
1.7 Counting Methods 25
1.8 Combinatorial Methods 32
1.9 Multinomial Coefficients 42
1.10 The Probability of a Union of Events 46
1.11 Statistical Swindles 51
1.12 Supplementary Exercises 53
2 Conditional Probability 55
2.1 The Definition of Conditional Probability 55
2.2 Independent Events 66
2.3 Bayes’ Theorem 76
2.4 The Gambler’s Ruin Problem 86
2.5 Supplementary Exercises 90
3 Random Variables and Distributions 93
3.1 Random Variables and Discrete Distributions 93
3.2 Continuous Distributions 100
3.3 The Cumulative Distribution Function 107
3.4 Bivariate Distributions 118
3.5 Marginal Distributions 130
3.6 Conditional Distributions 141
3.7 Multivariate Distributions 152
3.8 Functions of a Random Variable 167
3.9 Functions of Two or More Random Variables 175
3.10 Markov Chains 188
3.11 Supplementary Exercises 202
4 Expectation 207
4.1 The Expectation of a Random Variable 207
4.2 Properties of Expectations 217
4.3 Variance 225
4.4 Moments 234
4.5 The Mean and the Median 241
4.6 Covariance and Correlation 248
4.7 Conditional Expectation 256
4.8 Utility 265
4.9 Supplementary Exercises 272
5 Special Distributions 275
5.1 Introduction 275
5.2 The Bernoulli and Binomial Distributions 275
5.3 The Hypergeometric Distributions 281
5.4 The Poisson Distributions 287
5.5 The Negative Binomial Distributions 297
5.6 The Normal Distributions 302
5.7 The Gamma Distributions 316
5.8 The Beta Distributions 327
5.9 The Multinomial Distributions 333
5.10 The Bivariate Normal Distributions 337
5.11 Supplementary Exercises 345
6 Large Random Samples 347
6.1 Introduction 347
6.2 The Law of Large Numbers 348
6.3 The Central Limit Theorem 360
6.4 The Correction for Continuity 371
6.5 Supplementary Exercises 375
7 Estimation 376
7.1 Statistical Inference 376
7.2 Prior and Posterior Distributions 385
7.3 Conjugate Prior Distributions 394
7.4 Bayes Estimators 408
7.5 Maximum Likelihood Estimators 417
7.6 Properties of Maximum Likelihood Estimators 426
7.7 Sufficient Statistics 443
7.8 Jointly Sufficient Statistics 449
7.9 Improving an Estimator 455
7.10 Supplementary Exercises 461
8 Sampling Distributions of Estimators 464
8.1 The Sampling Distribution of a Statistic 464
8.2 The Chi-Square Distributions 469
8.3 Joint Distribution of the Sample Mean and Sample Variance 473
8.4 The t Distributions 480
8.5 Confidence Intervals 485
8.6 Bayesian Analysis of Samples from a Normal Distribution 495
8.7 Unbiased Estimators 506
8.8 Fisher Information 514
8.9 Supplementary Exercises 528
9 Testing Hypotheses 530
9.1 Problems of Testing Hypotheses 530
9.2 Testing Simple Hypotheses 550
9.3 Uniformly Most Powerful Tests 559
9.4 Two-Sided Alternatives 567
9.5 The t Test 576
9.6 Comparing the Means of Two Normal Distributions 587
9.7 The F Distributions 597
9.8 Bayes Test Procedures 605
9.9 Foundational Issues 617
9.10 Supplementary Exercises 621
10 Categorical Data and Nonparametric Methods 624
10.1 Tests of Goodness-of-Fit 624
10.2 Goodness-of-Fit for Composite Hypotheses 633
10.3 Contingency Tables 641
10.4 Tests of Homogeneity 647
10.5 Simpson’s Paradox 653
10.6 Kolmogorov-Smirnov Tests 657
10.7 Robust Estimation 666
10.8 Sign and Rank Tests 678
10.9 Supplementary Exercises 686
11 Linear Statistical Models 689
11.1 The Method of Least Squares 689
11.2 Regression 698
11.3 Statistical Inference in Simple Linear Regression 707
11.4 Bayesian Inference in Simple Linear Regression 729
11.5 The General Linear Model and Multiple Regression 736
11.6 Analysis of Variance 754
11.7 The Two-Way Layout 763
11.8 The Two-Way Layout with Replications 772
11.9 Supplementary Exercises 783
12 Simulation 787
12.1 What Is Simulation? 787
12.2 Why Is Simulation Useful? 791
12.3 Simulating Specific Distributions 804
12.4 Importance Sampling 816
12.5 Markov Chain Monte Carlo 823
12.6 The Bootstrap 839
12.7 Supplementary Exercises 850
Tables 853
Answers to Odd-Numbered Exercises 865
References 879
Index 885