Unit 2

Expectation, Continuous probability, CLT and Normal models

In this section we introduce expectation, an operation which takes a random variable and produces a deterministic quantity. The expectation of a random variable can be approximated with sample averages, and from them we can infer properties of the model (like parameters). Much of statistics relies on the fact that sample averages approximate expectations, and understanding how well these approximations work is a central goal of statistics. This will motivate us to study the probability distribution of sums of random variables, which leads to the CLT and the Normal distribution. By defining joint distribution where the conditionals are Normal, we will arrive at our first example of a linear regression model. We will study a linear regression model with a binary predictor and introduce the idea of a sample distribution in this context.

Material:

• Notes and exercises
• Colab notebook (2024)
• Colab notebook (2025)

Concepts

Expectation, conditional expectation, empirical averages, CLT, Normal random variables and their properties, coefficient of variation, sample distribution, linear regression model

Things to practice

• Calculate expectations by hand and in Python (including conditional expectations)
• Manipulate normal random variables (appling linearity of expectation and independence)
• How to use CLT to estimate probabilities of sums
• For a linear regression model with a binary predictor, understand how to estimate the regression coefficient as difference of conditional averages and derive the sample distribution (make sure you understand Example 12)

Wikipedia References

• Expectation
• Conditional expectation
• Coefficient of variation
• Law of large numbers
• Sampling distribution
• Central limit theorem
• Log Normal distribution