• Overview
• Details
• Exercises

The Law of Large Numbers (LLN) states that sample averages converge to the true mean. The Central Limit Theorem (CLT) states that properly normalized sums of i.i.d. variables approach a normal distribution, enabling approximate inference.

• Setup: i.i.d. X1, X2, ..., with mean μ and variance σ² < ∞. Define sample mean \(\bar{X}_n = \frac{1}{n}\sum_{i=1}^n X_i\).
• Weak LLN: \(\bar{X}_n \xrightarrow{p} \mu\) (convergence in probability).
• Strong LLN: \(\bar{X}_n \to \mu\) almost surely (stronger mode of convergence).
• CLT: \(\frac{\sqrt{n}(\bar{X}_n - \mu)}{\sigma} \xrightarrow{d} \mathcal{N}(0,1)\).
• Normal approximation: For large n, sums/means are approximately normal; use continuity corrections for discrete sums.
• Berry–Esseen: Rates of convergence depend on third absolute moment.
• Applications: Confidence intervals, A/B testing, error bars, bootstrap calibration.

1. Simulate coin flips to empirically illustrate LLN for the proportion of heads.
2. Use CLT to approximate \(\mathbb{P}(S_n \le k)\) for a Binomial(n,p) via a normal with continuity correction.
3. Derive a 95% confidence interval for μ from n i.i.d. samples with known σ.