Expectation, variance, covariance

Contents · Expectation, variance, covariance

Overview
Details
Exercises

Overview

Summary

Expectation is the average value of a random variable, variance measures dispersion around the mean, and covariance captures joint variability of two variables. These are the core moments used across probability and statistics.

Details

Concepts

Expectation (discrete): \(\mathbb{E}[X] = \sum_x x\,p_X(x)\); linearity: \(\mathbb{E}[aX+bY]=a\mathbb{E}[X]+b\mathbb{E}[Y]\).
Variance: \(\mathrm{Var}(X)=\mathbb{E}[X^2]-\mathbb{E}[X]^2\); scaling: \(\mathrm{Var}(aX+b)=a^2\mathrm{Var}(X)\).
Covariance: \(\mathrm{Cov}(X,Y)=\mathbb{E}[XY]-\mathbb{E}[X]\mathbb{E}[Y]\); independence ⇒ covariance 0 (but not conversely).
Variance of sums: \(\mathrm{Var}(\sum_i X_i)=\sum_i \mathrm{Var}(X_i) + 2\sum_{i.


            Correlation: \(\rho_{X,Y}=\frac{\mathrm{Cov}(X,Y)}{\sigma_X\sigma_Y}\), in [−1,1].
            Law of total expectation/variance: \(\mathbb{E}[X]=\mathbb{E}[\,\mathbb{E}[X\mid Y]~]\), \(\mathrm{Var}(X)=\mathbb{E}[\mathrm{Var}(X\mid Y)] + \mathrm{Var}(\mathbb{E}[X\mid Y])\).
            Indicators: \(\mathbb{E}[\mathbf{1}_A]=\mathbb{P}(A)\); useful for counting arguments.


      
      
        Exercises
        Hands-on
        
          
            Prove linearity of expectation and apply it to compute \(\mathbb{E}[\sum_{i=1}^n X_i]\) without independence.
            Show that \(\mathrm{Var}(X)=\mathbb{E}[(X-\mathbb{E}[X])^2]\) equals \(\mathbb{E}[X^2]-\mathbb{E}[X]^2\).
            Derive \(\mathrm{Var}(X+Y)=\mathrm{Var}(X)+\mathrm{Var}(Y)+2\mathrm{Cov}(X,Y)\) and specialize to independent variables.
            Use the law of total variance to compute the variance of a two-stage process (e.g., random number of coin flips).