Week 13/Module 12 - Multiple Regression — Module Topics

From Simple to Multiple Regression

• Limitations of Simple Linear Regression — Simple linear regression models the relationship between one predictor variable and one continuous outcome, which is often insufficient for real-world data.
• What Is Multiple Regression? — Multiple regression is an extension of simple linear regression that includes two or more predictor variables to explain variation in a single continuous outcome.
• Why Add More Predictor Variables? — Including additional predictors improves the model's ability to explain variance in the outcome and increases the accuracy of predictions.
• Controlling for Other Variables — A key advantage of multiple regression is the ability to statistically control for the influence of other predictors when estimating each variable's effect.
• When to Use Multiple Regression — Multiple regression is appropriate when a researcher wants to explain or predict a continuous outcome using more than one predictor variable.
• Continuity from Simple to Multiple Regression — Multiple regression builds directly on the concepts and mechanics of simple linear regression, making the transition a logical and incremental step.

Model Specification and Structure

• From Simple to Multiple Regression — Multiple regression extends simple linear regression by incorporating two or more predictor variables to explain variation in a single continuous outcome variable.
• The Multiple Regression Equation — The multiple regression model is expressed as a linear equation that combines a constant (intercept) with weighted contributions from each predictor variable.
• Selecting Predictor Variables — Choosing which variables to include in a multiple regression model is a critical step that should be guided by theory, prior research, and the research question.
• Interpreting Regression Coefficients in Context — Each coefficient in a multiple regression model has a specific conditional interpretation that differs from how coefficients are interpreted in simple regression.
• Assumptions Underlying Model Specification — A properly specified multiple regression model must satisfy several key assumptions for the estimates and inferences to be valid.
• The Role of the Error Term — The error term in a multiple regression model accounts for all variation in the outcome that is not explained by the included predictor variables.

Interpreting Regression Coefficients

• What Are Partial Regression Coefficients? — In multiple regression, each predictor has a partial regression coefficient that reflects its unique relationship with the outcome variable.
• Holding Other Variables Constant — The phrase 'holding other variables constant' is central to correctly interpreting coefficients in multiple regression.
• The Intercept in Multiple Regression — The intercept (b₀) in a multiple regression model represents the predicted value of the outcome when all predictor variables equal zero.
• Interpreting Positive and Negative Coefficients — The sign of a partial regression coefficient indicates the direction of the relationship between a predictor and the outcome, controlling for other variables.
• Unique Contribution of Each Predictor — Multiple regression allows researchers to assess the unique contribution of each predictor variable beyond what is explained by the other predictors.
• Units of Measurement and Coefficient Comparability — Raw (unstandardized) partial regression coefficients are expressed in the original units of the predictors, which can make direct comparison across predictors difficult.
• Common Misinterpretations to Avoid — Partial regression coefficients are frequently misread, especially when researchers conflate correlation with unique prediction or ignore the 'all else equal' condition.

Assessing Model Fit

• The Concept of Model Fit — Model fit refers to how well a multiple regression model explains the variation observed in the outcome variable using the selected predictors.
• R-Squared (R²): Definition and Interpretation — R-squared, also called the coefficient of determination, measures the proportion of total variation in the dependent variable that is explained by the regression model.
• Limitations of R-Squared in Multiple Regression — A key limitation of R-squared is that it never decreases when additional predictor variables are added to a model, even if those predictors are not meaningfully related to the outcome.
• Adjusted R-Squared: Correcting for Additional Predictors — Adjusted R-squared modifies the R² statistic by penalizing the addition of predictor variables that do not improve the model in a meaningful way.
• Comparing R-Squared and Adjusted R-Squared — Understanding the relationship between R² and adjusted R² helps researchers make informed decisions about model complexity and variable selection.
• Using Model Fit Statistics to Evaluate and Refine Models — R-squared and adjusted R-squared are practical tools for guiding decisions about which predictors to retain or remove during model building.

Building and Evaluating Multiple Regression Models

• Specifying the Multiple Regression Model — Model specification involves selecting which predictor variables to include in the regression equation to best explain variation in the outcome variable.
• Estimating Regression Coefficients — Once the model is specified, regression coefficients are estimated using the method of ordinary least squares (OLS), which minimizes the sum of squared residuals.
• Assessing Model Fit with R-Squared and Adjusted R-Squared — R-squared (R²) measures the proportion of variance in the outcome variable explained by the set of predictor variables in the model.
• Testing Overall Model Significance with the F-Test — The F-test evaluates whether the overall multiple regression model explains a statistically significant amount of variance in the outcome variable.
• Checking Regression Assumptions — Valid interpretation of multiple regression results depends on satisfying key statistical assumptions about the data and residuals.
• Refining the Model: Variable Selection Strategies — After an initial model is built, researchers often refine it by adding, removing, or transforming predictors to improve interpretability and predictive accuracy.
• Interpreting and Communicating Results — The final step in building a multiple regression model is translating statistical output into meaningful, actionable conclusions for a real-world audience.