Module 3: Multivariate Regression

What this module covers

This module generalizes everything from Module 2 to much more complex problems. Multivariate linear regression allows for any number of features (independent input variables) while assuming the target depends linearly on each. Nonlinear regression then extends this to polynomial and other nonlinear models.

You’ll see how the same gradient descent framework handles both cases — the key ideas (cost function, gradient, iterative optimization) carry over directly. The main new challenges are feature scaling (making sure features on different scales don’t distort the optimization) and managing the complexity of polynomial feature expansions.

Materials

This module is split into two parts, each with its own lecture and practice notebook.

Slides: Multivariate (Non)Linear Regression (pdf)

Part 1: Linear regression with multiple features

Lecture: Multivariate Linear Regression — extending gradient descent to multiple features, the normal equation, and feature scaling.

Practice: Multivariate Linear Regression — applying multivariate linear regression to real geoscience data (seismic velocity prediction from well log measurements).

Part 2: Nonlinear models

Lecture: Multivariate Nonlinear Regression — polynomial features, model complexity, and the beginnings of the bias-variance tradeoff.

Practice: Multivariate Nonlinear Regression — fitting nonlinear models and comparing with linear approaches.

Data files

The practice exercises use several datasets that are loaded directly in the notebooks, but are also available for inspection:

• Training_Data_Vp_Case1.csv, WR313-G_Vp_Case1.csv, WR313-H_Vp_Case1.csv — seismic velocity data
• AmesHousing.txt — the Ames Housing dataset

Prerequisites

Module 2 — univariate linear regression and gradient descent.

---

Next module: Module 4: Classification — from predicting continuous values to classifying categories.