Week 13/Module 12 - Multiple Regression — Topics & Learning Outcomes
1. From Simple to Multiple Regression
Description
This topic introduces multiple regression as an extension of simple linear regression, explaining why and when multiple predictor variables are needed to better model a continuous outcome.
Subtopics
- Limitations of Simple Linear Regression — Simple linear regression models the relationship between one predictor variable and one continuous outcome, which is often insufficient for real-world data.
- What Is Multiple Regression? — Multiple regression is an extension of simple linear regression that includes two or more predictor variables to explain variation in a single continuous outcome.
- Why Add More Predictor Variables? — Including additional predictors improves the model's ability to explain variance in the outcome and increases the accuracy of predictions.
- Controlling for Other Variables — A key advantage of multiple regression is the ability to statistically control for the influence of other predictors when estimating each variable's effect.
- When to Use Multiple Regression — Multiple regression is appropriate when a researcher wants to explain or predict a continuous outcome using more than one predictor variable.
- Continuity from Simple to Multiple Regression — Multiple regression builds directly on the concepts and mechanics of simple linear regression, making the transition a logical and incremental step.
Supporting Materials
- From Simple to Multiple Regression Topic Detail · Web page
This reading explains why simple linear regression is insufficient for real-world engineering and scientific problems and introduces multiple regression as the method for modeling outcomes that depend on more than one predictor variable; learners read it to build conceptual motivation before studying the technique formally.
- From Simple to Multiple Regression — Videos Topics and Embedded Video · Web page
A reading that motivates the move from single-predictor to multi-predictor models, using engineering examples like fuel consumption and concrete strength to show why one variable rarely explains an outcome on its own, serving as the conceptual entry point to the module.
2. Model Specification and Structure
Description
This topic covers how to properly specify a multiple regression model, including selecting predictor variables and understanding the mathematical structure of the regression equation.
Subtopics
- From Simple to Multiple Regression — Multiple regression extends simple linear regression by incorporating two or more predictor variables to explain variation in a single continuous outcome variable.
- The Multiple Regression Equation — The multiple regression model is expressed as a linear equation that combines a constant (intercept) with weighted contributions from each predictor variable.
- Selecting Predictor Variables — Choosing which variables to include in a multiple regression model is a critical step that should be guided by theory, prior research, and the research question.
- Interpreting Regression Coefficients in Context — Each coefficient in a multiple regression model has a specific conditional interpretation that differs from how coefficients are interpreted in simple regression.
- Assumptions Underlying Model Specification — A properly specified multiple regression model must satisfy several key assumptions for the estimates and inferences to be valid.
- The Role of the Error Term — The error term in a multiple regression model accounts for all variation in the outcome that is not explained by the included predictor variables.
Supporting Materials
- Model Specification and Structure Topic Detail · Web page
This reading covers how a multiple regression model is mathematically structured, including how predictors are combined with coefficients and an intercept to form the model equation; learners use it to understand how to correctly specify a multiple regression model before fitting it to data.
- Model Specification and Structure — Videos Topics and Embedded Video · Web page
A lesson on specifying the multiple regression model, extending y = b0 + b1x + e to several predictors and guiding learners through deciding which variables to include and how the model's structure represents their joint effect on the response.
3. Interpreting Regression Coefficients
Description
This topic focuses on how to interpret partial regression coefficients in a multiple regression context, distinguishing the unique contribution of each predictor while holding others constant.
Subtopics
- What Are Partial Regression Coefficients? — In multiple regression, each predictor has a partial regression coefficient that reflects its unique relationship with the outcome variable.
- Holding Other Variables Constant — The phrase 'holding other variables constant' is central to correctly interpreting coefficients in multiple regression.
- The Intercept in Multiple Regression — The intercept (b₀) in a multiple regression model represents the predicted value of the outcome when all predictor variables equal zero.
- Interpreting Positive and Negative Coefficients — The sign of a partial regression coefficient indicates the direction of the relationship between a predictor and the outcome, controlling for other variables.
- Unique Contribution of Each Predictor — Multiple regression allows researchers to assess the unique contribution of each predictor variable beyond what is explained by the other predictors.
- Units of Measurement and Coefficient Comparability — Raw (unstandardized) partial regression coefficients are expressed in the original units of the predictors, which can make direct comparison across predictors difficult.
- Common Misinterpretations to Avoid — Partial regression coefficients are frequently misread, especially when researchers conflate correlation with unique prediction or ignore the 'all else equal' condition.
Supporting Materials
- Interpreting Regression Coefficients Topic Detail · Web page
This reading explains how to interpret regression coefficients in a multiple regression context, emphasizing that each coefficient represents the effect of one predictor while holding all others constant; learners use it to correctly draw conclusions from estimated model parameters.
- Interpreting Regression Coefficients — Videos Topics and Embedded Video · Web page
A reading explaining partial regression coefficients — how each estimated slope represents the expected change in the response per one-unit increase in that predictor while the others are held constant — so learners can correctly interpret multi-predictor output.
4. Assessing Model Fit
Description
This topic examines statistical measures used to evaluate how well a multiple regression model fits the data, with emphasis on R-squared and adjusted R-squared and what they reveal about explanatory power.
Subtopics
- The Concept of Model Fit — Model fit refers to how well a multiple regression model explains the variation observed in the outcome variable using the selected predictors.
- R-Squared (R²): Definition and Interpretation — R-squared, also called the coefficient of determination, measures the proportion of total variation in the dependent variable that is explained by the regression model.
- Limitations of R-Squared in Multiple Regression — A key limitation of R-squared is that it never decreases when additional predictor variables are added to a model, even if those predictors are not meaningfully related to the outcome.
- Adjusted R-Squared: Correcting for Additional Predictors — Adjusted R-squared modifies the R² statistic by penalizing the addition of predictor variables that do not improve the model in a meaningful way.
- Comparing R-Squared and Adjusted R-Squared — Understanding the relationship between R² and adjusted R² helps researchers make informed decisions about model complexity and variable selection.
- Using Model Fit Statistics to Evaluate and Refine Models — R-squared and adjusted R-squared are practical tools for guiding decisions about which predictors to retain or remove during model building.
Supporting Materials
- Assessing Model Fit Topic Detail · Web page
This reading covers the metrics and methods used to evaluate how well a multiple regression model fits the data, including R-squared, adjusted R-squared, and overall model significance tests; learners use it to judge whether a fitted model adequately explains variation in the outcome variable.
- Assessing Model Fit — Videos Topics and Embedded Video · Web page
A lesson on evaluating how well a fitted model describes the data, covering R-squared and related fit measures and how to use them to compare competing models built from different predictor sets.
5. Building and Evaluating Multiple Regression Models
Description
This topic guides learners through the practical process of constructing, testing, and refining multiple regression models using real-world data to draw meaningful analytical conclusions.
Subtopics
- Specifying the Multiple Regression Model — Model specification involves selecting which predictor variables to include in the regression equation to best explain variation in the outcome variable.
- Estimating Regression Coefficients — Once the model is specified, regression coefficients are estimated using the method of ordinary least squares (OLS), which minimizes the sum of squared residuals.
- Assessing Model Fit with R-Squared and Adjusted R-Squared — R-squared (R²) measures the proportion of variance in the outcome variable explained by the set of predictor variables in the model.
- Testing Overall Model Significance with the F-Test — The F-test evaluates whether the overall multiple regression model explains a statistically significant amount of variance in the outcome variable.
- Checking Regression Assumptions — Valid interpretation of multiple regression results depends on satisfying key statistical assumptions about the data and residuals.
- Refining the Model: Variable Selection Strategies — After an initial model is built, researchers often refine it by adding, removing, or transforming predictors to improve interpretability and predictive accuracy.
- Interpreting and Communicating Results — The final step in building a multiple regression model is translating statistical output into meaningful, actionable conclusions for a real-world audience.
Supporting Materials
- Building and Evaluating Multiple Regression Models Topic Detail · Web page
This reading provides a practical, end-to-end walkthrough of constructing and evaluating multiple regression models, covering predictor selection, assumption checking, and interpretation of results; learners use it as an applied guide for carrying out a full multiple regression analysis.
- Building and Evaluating Multiple Regression Models — Videos Topics and Embedded Video · Web page
A summative reading that walks through the full workflow of constructing and assessing a multiple regression model with engineering examples such as material strength depending on temperature, curing time, and composition.
← Back to the module page
Student Learning Outcomes
By the end of this module, students will be able to:
MO1
Construct a multiple regression equation by selecting appropriate predictor variables and estimating coefficients using ordinary least squares
Level: ApplyType: CognitiveCourse mapping: —
MO2
Interpret partial regression coefficients in a multiple regression model as the unique effect of each predictor on the outcome while holding all other predictors constant
Level: UnderstandType: CognitiveCourse mapping: —
MO3
Differentiate between R-squared and adjusted R-squared as measures of model fit in the context of multiple predictor variables
Level: AnalyzeType: CognitiveCourse mapping: —
MO4
Evaluate overall multiple regression model significance using the F-test to determine whether the model explains a statistically significant proportion of variance in the outcome variable
Level: EvaluateType: CognitiveCourse mapping: —
MO5
Justify the inclusion or removal of predictor variables during model refinement based on adjusted R-squared, coefficient significance, and underlying regression assumptions
Level: EvaluateType: CognitiveCourse mapping: —
Course Outcomes (reference)
No course outcomes have been defined.