Week 13/Module 12 - Multiple Regression

From Simple to Multiple Regression

Introduction

Simple linear regression models an outcome using a single predictor, and it works well when one variable really does carry most of the explanatory weight. In engineering practice, that is rarely the case: fuel consumption depends on more than speed, concrete strength depends on more than cure time, and the price of a used car depends on more than mileage. When important variables are left out, the slope you estimate for the one predictor you kept can absorb the influence of everything you ignored, and your predictions and conclusions suffer.

Multiple regression extends the same least-squares idea to two or more predictors at once. The structure is familiar — an intercept, a slope for each predictor, and residuals — but the interpretation changes in an important way: each coefficient now describes the effect of its own predictor while the other predictors are held constant. That idea, often called controlling for other variables, is the conceptual bridge between what you learned last week and what you will do for the rest of this module.

As you watch these videos, focus on three things: how the simple regression equation is generalized to several predictors, what the phrase "holding the other variables constant" actually means for a fitted coefficient, and what signals tell you that one predictor is not enough. Keep notes on the vocabulary (predictor, response, coefficient, controlling for), because the later topics on R-squared, adjusted R-squared, and the F-test all build directly on it.

Video 1: Multiple Regression: An overview with an illustration predicting a price for a used car using Excel

Dr Nic's Maths and Stats · 7:46 · 10,918 views · 5y ago · captions — Watch on YouTube

About this video

A compact overview of multiple linear regression built around a worked illustration: predicting the price of a used car from several predictor variables using Excel. Based on its structure and timestamps, it introduces what multiple regression is, how the model is set up with more than one predictor, and how output is produced and read from a spreadsheet tool.

Why this video

Dr Nic's Maths and Stats is a well-regarded statistics education channel, and the used-car example makes the motivation for adding predictors concrete, which maps directly onto the "What Is Multiple Regression?" and "Why Add More Predictor Variables?" subtopics.

Video 2: Regression analysis | CLOSER Learning Hub

CLOSER · 3:51 · 230,126 views · 5y ago · captions — Watch on YouTube

About this video

A short animated explainer from CLOSER on regression analysis as a general tool for examining relationships between variables. The description indicates it covers relationships between two or more variables, so expect it to move from the single-predictor case toward the idea of modelling an outcome with multiple explanatory variables.

Why this video

Its brevity and animation make it a good conceptual primer on the continuity from simple to multiple regression, useful before you tackle the algebra and software output.

Video 3: Multiple Regression - Interpreting coefficients

Dr Nic's Maths and Stats · 7:57 · 19,708 views · 5y ago · captions — Watch on YouTube

About this video

A follow-on video focused specifically on how to read regression coefficients, in both the single-predictor and multiple-predictor settings. It is positioned as a continuation of the overview video, so it should address what each estimated slope means once other predictors are present in the model.

Why this video

Coefficient interpretation is exactly where simple and multiple regression diverge, making this the clearest fit for the "Controlling for Other Variables" subtopic and the best preparation for partial coefficients later in the module.

How These Videos Support the Learning Outcomes

Together these videos set up the conceptual foundation the module's outcomes rest on. The overview and the animated explainer show how a model with several predictors is constructed and why additional variables are worth including, which supports MO1 and informs the judgment calls behind MO5. The coefficients video speaks directly to MO2, since interpreting a slope as a unique effect while other predictors are held constant is the central skill there. These are introductory and conceptual treatments, so the detailed machinery for MO3 (R-squared versus adjusted R-squared) and MO4 (the overall F-test) is developed in the lectures, readings, and later topics of this module — use the videos to build intuition first, then return to the formal criteria.

About These Videos

These videos were chosen from YouTube on October 2, 2026. Each has captions; the player turns them on. Videos on YouTube can be changed or removed — if one no longer plays, tell your instructor. If a player does not appear, use its Watch on YouTube link.

Model Specification and Structure

Introduction

Multiple regression extends the simple linear model y = b0 + b1x + e by allowing several predictors to act on the response at once. Instead of asking how one variable relates to an outcome, we ask how a set of variables jointly explains it, and how much each one contributes once the others are accounted for. Model specification is the step where you decide which predictors belong in the equation, what form they take (continuous, categorical/indicator, possibly transformed), and what assumptions you are making about the error term.

This matters in engineering practice because real systems rarely depend on a single input. Yield depends on temperature and pressure and feed rate; travel time depends on distance, time of day, and road type. A badly specified model, one missing an important variable or padded with irrelevant ones, produces coefficients that are misleading even when the software reports impressive-looking output. Getting the specification right is what makes the later inference (t-tests on coefficients, the overall F-test, R-squared versus adjusted R-squared) meaningful.

As you watch, pay attention to three things. First, the language used to interpret a coefficient: a partial slope is the expected change in the response per unit change in that predictor while the other predictors are held fixed. Second, how predictors get chosen and compared, and why adding variables always raises R-squared but not necessarily adjusted R-squared. Third, how non-numeric predictors are brought into a model that is, mathematically, still linear in its coefficients.

Video 1: Multiple Regression Predictions, interpreting and understanding

Dr Nic's Maths and Stats · 9:34 · 6,320 views · 3y ago · captions — Watch on YouTube

About this video

Dr Nic works through prediction with a simple regression model and then with a multiple regression model, comparing the two side by side. Based on the title and description, expect an emphasis on reading the fitted equation, substituting predictor values, and interpreting what the output actually means rather than just computing it.

Why this video

It directly addresses the move from simple to multiple regression and the interpretation of the regression equation, and Dr Nic's channel is known for clear, careful statistical explanation aimed at students.

Video 2: Statistics 101: Multiple Regression, Best Subsets

Brandon Foltz · 11:59 · 16,183 views · 5y ago · captions — Watch on YouTube

About this video

This Statistics 101 video introduces best subsets regression, a systematic procedure for comparing candidate models built from different combinations of available predictors. It is likely to cover how competing subsets are evaluated using fit criteria and why a model with more predictors is not automatically a better model.

Why this video

Brandon Foltz's series is a widely used, slow-paced, example-driven introduction, and this episode speaks directly to the subtopic of selecting predictor variables and justifying what stays in the model.

Video 3: Fitting & interpreting regression models: Linear regression with categorical predictors

StataCorp LLC · 9:56 · 17,821 views · 4y ago · captions — Watch on YouTube

About this video

StataCorp demonstrates fitting a linear regression that includes a categorical predictor using factor-variable notation, and then interpreting the resulting coefficients. Expect discussion of how category levels are encoded relative to a baseline group and what each estimated coefficient means in that setup.

Why this video

It extends model specification beyond purely numeric predictors, which is essential for correct interpretation of coefficients in context, and it comes from the software developer itself, so the mechanics shown are authoritative.

How These Videos Support the Learning Outcomes

Together these videos build the specification skills the module outcomes require. The Dr Nic video supports MO1 and MO2 by showing how a multiple regression equation is assembled and how its coefficients are read as predictor-specific effects with the other variables held constant. The best subsets video supports MO3 and MO5 by framing model building as a comparison among candidate predictor sets judged on fit measures, which is exactly the reasoning behind preferring adjusted R-squared over raw R-squared and defending the inclusion or removal of a variable. The Stata demonstration reinforces MO1 and MO2 for categorical predictors, and the output shown in these examples is the same output you will use in lecture to carry out the overall F-test for MO4.

About These Videos

These videos were chosen from YouTube on October 2, 2026. Each has captions; the player turns them on. Videos on YouTube can be changed or removed — if one no longer plays, tell your instructor. If a player does not appear, use its Watch on YouTube link.

Interpreting Regression Coefficients

Introduction

In simple linear regression, the slope answers a straightforward question: when x goes up by one unit, how much does the predicted y change? In multiple regression we estimate several slopes at once, and each one is a partial regression coefficient. A partial coefficient tells you the expected change in the response for a one-unit increase in that predictor while all other predictors in the model are held fixed. That small phrase, "holding the others constant," is what makes multiple regression powerful and also what makes it easy to misread.

This matters in engineering work because real systems rarely depend on one input. Material strength, process yield, energy consumption, and travel time all respond to several factors simultaneously, and those factors are usually correlated with each other. Learning to read a coefficient as a unique contribution, net of the other predictors, is what lets you say something defensible about an individual factor instead of just quoting an overall fit statistic. You also need to recognize what the intercept means (and when it is meaningless), what the sign of a coefficient does and does not imply, and why coefficients measured in different units cannot be ranked by size alone.

As you watch, keep a few questions in front of you. What exactly is being held constant in each interpretation sentence? What are the units of the response and of each predictor, and does the stated interpretation carry those units? Does the intercept correspond to a physically possible setting of the predictors? And watch how each presenter phrases a conclusion out loud, because wording is where most interpretation errors show up on exams and in reports. Try writing your own one-sentence interpretation of a coefficient before the video gives one, then compare.

Video 1: Multiple Regression | Coefficients – Interpretation, C.I, Hypothesis Testing

Joshua Emmanuel · 8:06 · 132,587 views · 3y ago · captions — Watch on YouTube

About this video

A focused walkthrough of the coefficients section of typical multiple regression software output. Based on its title and description, it covers how to read each estimated coefficient, how to interpret the accompanying confidence intervals, and how hypothesis tests on individual coefficients are reported and read. It builds on a simple linear regression example so the move from one slope to several partial slopes is visible.

Why this video

Because most of your interpretation work will happen from a software output table, this video connects the concept of a partial coefficient directly to the numbers you will actually see, including the inference columns that tell you whether a predictor's unique contribution is statistically distinguishable from zero.

Video 2: Multiple Regression - Interpreting coefficients

Dr Nic's Maths and Stats · 7:57 · 19,708 views · 5y ago · captions — Watch on YouTube

About this video

Dr Nic works through how to interpret coefficients in both simple linear and multiple regression models, with attention to the wording used to state an interpretation. Expect explicit treatment of the "holding other variables constant" phrasing, the meaning of positive versus negative coefficients, and what the intercept represents in a model with multiple predictors. The description notes it continues from an earlier video in a regression series.

Why this video

This is the most directly on-topic video for the subtopics on partial coefficients, holding other variables constant, and sign interpretation, and Dr Nic's channel is well known for careful, plain-language statistical explanations that emphasize correct phrasing over mechanics.

Video 3: Multiple Regression Predictions, interpreting and understanding

Dr Nic's Maths and Stats · 9:34 · 6,320 views · 3y ago · captions — Watch on YouTube

About this video

This video uses fitted simple and multiple regression models to generate predictions, then discusses how to understand what the model is telling you. Substituting predictor values into the equation makes concrete how each coefficient contributes separately to a predicted value, and how the intercept enters the calculation. It is a natural companion to the interpretation video from the same channel.

Why this video

Prediction is a useful check on interpretation: seeing how changing one predictor by one unit shifts the prediction, with the others fixed, makes the partial-coefficient idea tangible rather than verbal, and it reinforces attention to units of measurement.

How These Videos Support the Learning Outcomes

These three videos sit squarely on MO2, the interpretation of partial regression coefficients as the unique effect of a predictor with the others held constant. Dr Nic's interpretation video supplies the conceptual language and the intercept discussion, the prediction video turns that language into arithmetic you can verify, and Joshua Emmanuel's walkthrough ties both to the coefficient table produced by statistical software. Together they also reinforce MO1 by showing a fitted multiple regression equation in use, and they touch MO5, since the hypothesis tests and confidence intervals on individual coefficients are part of the evidence you weigh when deciding whether a predictor earns a place in the model. Overall model fit and significance (MO3 and MO4) are handled in other topics of this module.

About These Videos

These videos were chosen from YouTube on October 2, 2026. Each has captions; the player turns them on. Videos on YouTube can be changed or removed — if one no longer plays, tell your instructor. If a player does not appear, use its Watch on YouTube link.

Assessing Model Fit

Introduction

Once you have fit a multiple regression model, the next question is how well it actually describes the data. Measures of model fit summarize how much of the variation in the response variable the model accounts for, and they give you a basis for comparing competing models built from different sets of predictors. In this topic we focus on the coefficient of determination, R-squared, and its companion statistic, adjusted R-squared.

R-squared is easy to compute and easy to over-trust. A key fact in multiple regression is that R-squared can never decrease when you add another predictor, even a useless one, so a high R-squared alone does not mean you have a good or parsimonious model. Adjusted R-squared addresses this by penalizing the model for the number of predictors relative to the sample size, so it can go down when an added variable does not earn its place. Understanding the difference is what lets you use these numbers responsibly when refining a model.

As you watch, pay attention to three things: the formulas and where the degrees of freedom enter, the behavior of R-squared versus adjusted R-squared as predictors are added, and how these statistics appear alongside the ANOVA table, the F-test, and the standard error of the estimate in typical regression output. Try to connect each number on the output to the sums of squares it comes from.

Video 1: Statistics 101: Multiple Regression, Adjusted R-square

Brandon Foltz · 16:38 · 24,827 views · 5y ago · captions — Watch on YouTube

About this video

A long-form lecture in Brandon Foltz's Statistics 101 series that works through adjusted R-squared in the multiple regression setting. Based on the title and description, it builds up from the regression model and ordinary R-squared to explain why an adjustment is needed and how the penalty for additional predictors works. Expect a slow, worked treatment with discussion of how the statistic behaves as variables are added to a model.

Why this video

The extended pace and step-by-step style make this a good primary explanation of the concept of fit, the limitations of R-squared, and the correction adjusted R-squared provides, covering several subtopics in one pass.

Video 2: Adjusted R-squared

LearnChemE · 6:26 · 5,790 views · 4y ago · captions — Watch on YouTube

About this video

A short, focused screencast from LearnChemE, produced by faculty in the Department of Chemical and Biological Engineering at the University of Colorado Boulder. In about six minutes it introduces adjusted R-squared, likely showing the formula with its degrees-of-freedom terms and illustrating how the value changes relative to ordinary R-squared when predictors are added.

Why this video

It is a concise, engineering-oriented refresher that pairs well with the longer lecture and directly targets the comparison between R-squared and adjusted R-squared.

Video 3: Multiple Regression | ANOVA Table | F-Test | R-square | Standard Error

Joshua Emmanuel · 4:05 · 105,910 views · 3y ago · captions — Watch on YouTube

About this video

A compact problem-solving video that works a multiple regression example with 40 observations, filling in an ANOVA table and computing the F-test, R-squared, and the standard error of the estimate. The emphasis is on reading and completing regression output rather than on theory.

Why this video

It shows where fit statistics sit within standard regression output and connects R-squared to the sums of squares and the overall F-test, which is exactly the context in which you will interpret model fit on exams and in practice.

How These Videos Support the Learning Outcomes

Together these videos center on MO3 by defining R-squared, exposing its limitation of never decreasing as predictors are added, and showing how adjusted R-squared corrects for model size. The Joshua Emmanuel example reinforces MO4 by placing R-squared in the ANOVA table alongside the overall F-test and standard error, and it assumes the fitted model framework from MO1 and MO2. The discussion of how adjusted R-squared responds to adding or dropping variables gives you the reasoning tool needed for MO5, where you must justify keeping or removing predictors during model refinement.

About These Videos

These videos were chosen from YouTube on October 2, 2026. Each has captions; the player turns them on. Videos on YouTube can be changed or removed — if one no longer plays, tell your instructor. If a player does not appear, use its Watch on YouTube link.

Building and Evaluating Multiple Regression Models

Introduction

Multiple regression extends simple linear regression by allowing several predictor variables to explain variation in a single response variable. In engineering settings you rarely have just one input driving an outcome: material strength may depend on temperature, curing time, and composition all at once. The multiple regression model expresses the response as an intercept plus a weighted sum of predictors, where each coefficient is estimated by ordinary least squares and interpreted as the effect of that predictor while the others are held constant.

Building a usable model is more than running the computation. You need to judge how well the model fits (R-squared versus adjusted R-squared), whether the model as a whole explains a significant share of variance (the overall F-test), whether individual coefficients are significant (t-tests), and whether the residuals behave as the assumptions of linearity, constant variance, independence, and normality require. Models with too many weak predictors can look impressive on R-squared alone while adding little real explanatory value, which is why refinement strategies matter.

The videos below walk through these decisions with worked examples. As you watch, pay attention to the reasoning used to decide whether a variable earns its place in the model, how adjusted R-squared responds when predictors are added or dropped, and how the F-test statistic is formed from the regression and error sums of squares. Keep a note of the vocabulary used for output tables, since that is the form in which you will see most software results.

Video 1: Statistics 101: Multiple Regression, Interactive Model Building

Brandon Foltz · 9:31 · 19,338 views · 5y ago · captions — Watch on YouTube

About this video

This video walks through the interactive, judgment-based side of model building: deciding when a predictor should be kept in a multiple regression model and when it should be dropped. Based on its title and description, it focuses on the decision logic behind including or excluding features rather than on mechanical computation alone. Expect discussion of fit measures and coefficient significance as the evidence behind those decisions.

Why this video

It speaks directly to the refinement subtopic, showing how adjusted R-squared and coefficient results are weighed in practice, and Brandon Foltz's Statistics 101 series is known for slow, example-driven explanations suited to first-time learners.

Video 2: Statistics 101: Multiple Regression, Stepwise Regression

Brandon Foltz · 11:44 · 43,835 views · 5y ago · captions — Watch on YouTube

About this video

This video introduces stepwise regression as a systematic model-building procedure, using a worked example to show how predictors enter or leave the model one at a time. It is likely to cover the criteria used at each step and how the model changes as variables are added or removed. It complements the more judgment-based approach by presenting an algorithmic alternative.

Why this video

Variable selection strategies are a named subtopic here, and this video gives the standard automated approach along with its logic, letting you compare it against manual, reasoning-driven refinement.

Video 3: F Test for Overall Significance

Jason Liu · 4:16 · 18,448 views · 5y ago · captions — Watch on YouTube

About this video

A short, focused clip on the F-test for overall significance of a multiple regression model. It explains what the test is asking, namely whether there is a statistically significant linear relationship between the response and the set of predictors taken together, and how to read the result. Expect the hypotheses, the test statistic, and the interpretation of the outcome.

Why this video

It isolates the overall F-test in a compact format, making it an efficient review of the distinction between testing the whole model and testing individual coefficients.

How These Videos Support the Learning Outcomes

Together these videos trace the full model-building cycle described in the module outcomes. The stepwise regression and interactive model-building videos support constructing a regression equation with appropriate predictors (MO1), interpreting what each coefficient contributes alongside the others (MO2), distinguishing R-squared from adjusted R-squared when predictors are added (MO3), and justifying inclusion or removal decisions during refinement (MO5). The F-test video targets MO4 directly by showing how to determine whether the model as a whole explains a significant proportion of variance. Use the videos as worked companions to the textbook treatment of assumption checking and software output interpretation.

About These Videos

These videos were chosen from YouTube on October 2, 2026. Each has captions; the player turns them on. Videos on YouTube can be changed or removed — if one no longer plays, tell your instructor. If a player does not appear, use its Watch on YouTube link.