About the course

Instructor: Ethan Levien.

Meeting time: T/TH @ 10-12

Office Hours: T @ 1

Prerequisites: Technically math 10; however, there are many other paths to prepare you for this course. The import thing is that you have some exposure to probability theory and are comfortable coding.

Course objectives: You will learn how to build, fit and make predictions with regression models. Such models form the basis of many widely used data analysis techniques, including most machine learning algorithms. This is an applied course, and we won’t spend much time deriving equations or proving theorems. Instead, we will learn by playing around with real and simulated data. We will challenge the underlying assumptions of a method and see when things “break”. The goal is to help you develop an intuition about statistical inference, which can be generalized to real world settings where theorems and analytical formulas are not applicable.

Coding: All coding for the course will be done in Python. Some advantages of python (over R) are that: (1) We can easily run everything directly in the browser using google colab notebooks, so there is no need to download anything on your machine. (2) Python is widely used in data science and machine learning, both in academia and in industry. (3) While some very basic things are more difficult to implement, I think you’ll find it’s easier to generalize to more advanced methods. (4) I know python and I’ve basically never coded in R.

Course policies

Attendence: The course meets twice a week and attendance is mandatory.

Grading: Your grade will be based on the following assignments:

• Exercises (60%) - see links in schedule.
• Attendence (10%)
• Project (30%) - see project guidelines
• project proposal (5%)
• final writeup (25%)

Grades will be based on a combination of self-evaluation and my own assessment (more on this in class).

Exercises: Your “homework” is to submit solutions to a set of exercises. I say “homework” because I plan to incorperate problem solving sessions into the lectures, giving you more time to discuss problems with myself and your peers. You will submit you solutions to gradescope (approximately) every week before I release the solutions. Then, the following week you will self-evaluate (i.e. grade) you solutions and submit the evaluation. You should use the following point scale, which I will elaborate on in class:

• 0 - no work was done, or barely any effort was made.
• 1 - You put down partial work, but didn't put much effort in and didn't reach out if you needed help.
• 2 - You put in effort, but didn't get the problem exactly correct. You reached out at least once if you needed help.
• 3 - You got the problem correct, or made a very significant effort, including attending office hours and asking questions on slack if needed.

The # of points you get for each self-evaluations are simply the score you’ve given yourself plus an additional point for providing an explanation of what you did wrong. The graders will simply review your self-evaluations.

Resources

Textbooks: The course will be self-contained in the python notebooks and class notes. However, I will occasionally reference the following textbooks for additional readings (note that the first two texts use R rather than python, so the coding components can be skipped):

• Regression and other stories. The course is very roughly based on this text, which provides a non-technical introduction to regression analysis with many worked examples.
• Advanced Statistics with Applications in R. A more technical introduction to applied regression. I will refer to this text for some of the proofs and derivations.
• Probabilistic Programming and Bayesian Methods for Hackers I will reference this text when we cover probabilistic programming in pymc3. The pymc3 documentation itself is a bit dense.

Software: All coding will be done using python in colab notebooks. Within python there are a number of packages we will use throughout the course, including:

• numpy for working with arrays, linear algebra and generating random numbers.
• pandas for working with tabular data sets
• statsmodels for classical statistics
• pymc3 for Bayesian statistics

You do not need to be an expert in any of these packages and I will provide you with skeleton code for most tasks. That said, I expect you to have some basic proficiency debugging, which will occasionally involve referencing the documentation for these packages.

Schedule

Week	Topics	Reading	Assignments
1	Course policies Introduction python (during xhours) Using simulation to explore probability models	Models and simulation Python Notebook	Python for Scientific Computing (not graded)
2	Probability and random variables, CLT Linear regression model with a single predictor	Basic statistical models	Exercise Set 1
3	Estimators, bias, consistancy Statistical inference for regression modeling	Statistical inference	Exercise Set 1 Self-evaluation Exercise Set 2
4	Regression to the mean, AR processes Evaluating regression output p-values	Working with regression models	Exercise Set 2 Self-evaluation Exercise Set 3
5	Regression with multiple predictors Interpreting regression coefficients Working with categorical data Analysis of variance	Multiple predictors	Exercise Set 3 Self-evaluation Project proposal due
6	Performing regression on transformed variables Model building strategies Experimental design	Model evaluation	None! Day of caring
7	Adding features to a model Overfitting	Model evaluation	Exercise Set 4
8	Logistic regression	Model evaluation
9	Logistic regression	Bayes and regularization	Final project report (due during finals week)

Accessibility Needs

Students with disabilities who may need disability-related academic adjustments and services for this course are encouraged to see me privately as early in the term as possible. Students requiring disability- related academic adjustments and services must consult the Student Accessibility Services office (Carson Hall, Suite 125, 646-9900). Once SAS has authorized services, students must show the originally signed SAS Services and Consent Form and/or a letter on SAS letterhead to me. As a first step, if students have questions about whether they qualify to receive academic adjustments and services, they should contact the SAS office. All inquiries and discussions will remain confidential.