Week 12/Module 11 - Regression Analysis

Foundations of Regression Analysis

Introduction

Regression analysis is the set of tools engineers use to describe and quantify how one variable responds to one or more others. In its simplest form, you fit a straight line through a cloud of data points so that the line summarizes the relationship between an independent (predictor) variable and a dependent (response) variable. Once that line exists, it can be used for two distinct jobs: explaining how much the response changes when a predictor changes, and predicting the response for conditions you have not yet observed.

This topic matters because most engineering decisions involve more than one influence at a time. Yield depends on temperature and pressure; cycle time depends on batch size, operator experience, and machine age. Moving from simple regression (one predictor) to multiple regression (several predictors) lets you separate these effects. Along the way you need a way to judge whether the model is any good, which is where model fit measures such as R-squared, the standard error, and the F-test come in, and whether the model is trustworthy, which is where the underlying assumptions (linearity, independence, constant variance, and approximately normal errors) come in.

As you watch the videos below, keep three questions in mind. First, what exactly is being predicted and what is doing the predicting? Second, what number in the output tells you how well the model fits, and what does that number not tell you? Third, how does the mathematics scale up when you go from one predictor to many? The videos move roughly in that order, from conceptual framing to reading output to the underlying matrix machinery.

Video 1: Why Linear regression for Machine Learning?

IBM Technology · 3:59 · 100,299 views · 2y ago · captions — Watch on YouTube

About this video

A short conceptual overview from IBM that frames linear regression as a method for modeling the relationship between a dependent variable and one or more independent variables, and explains why it is such a common starting point in machine learning and data analysis. Based on its title and description, it is likely to cover the idea of fitting a line to data, the distinction between explanation and prediction, and typical use cases. Expect vocabulary and framing rather than hand calculations.

Why this video

It is a clean, plain-language entry point to the subtopics What Is Regression Analysis?, Dependent and Independent Variables, and Purpose of Regression in Decision-Making, produced by a well-known technical publisher whose explainer series is aimed at newcomers.

Video 2: Multiple Regression | ANOVA Table | F-Test | R-square | Standard Error

Joshua Emmanuel · 4:05 · 105,910 views · 3y ago · captions — Watch on YouTube

About this video

A worked problem that takes a multiple regression scenario with 40 observations and walks through the ANOVA table, filling in degrees of freedom and sums of squares, then computing the F-statistic, R-squared, and the standard error of the estimate. It shows where each quantity in a regression printout comes from and how the pieces of the table relate to one another. The focus is on reading and completing output rather than on deriving the model.

Why this video

It directly addresses The Concept of Model Fit and the simple-versus-multiple distinction by showing how fit statistics are computed and interpreted when there are several predictors, and Joshua Emmanuel's channel is widely used for concise, step-by-step business-statistics problem solutions.

Video 3: Matrix Approach to Multiple Linear Regression

LearnChemE · 10:14 · 99,137 views · 4y ago · captions — Watch on YouTube

About this video

A longer treatment from LearnChemE that presents multiple linear regression in matrix form, setting up the design matrix and response vector and using the normal equations to obtain the coefficient estimates. The description points to an accompanying spreadsheet, so the presentation is likely to connect the matrix algebra to a practical computational setup. This is the general machinery that simple regression is a special case of.

Why this video

It shows why multiple regression is a natural extension of simple regression rather than a separate technique, and LearnChemE is a university-produced engineering education channel, so the notation and worked approach fit an engineering audience.

How These Videos Support the Learning Outcomes

Together these videos build the conceptual base the module outcomes rest on. The IBM overview establishes what regression is and why engineers use it for decision-making, supporting the framing needed for MO1 and MO5. The ANOVA and F-test walkthrough gives direct practice reading the fit statistics named in MO2 and makes clear how output changes as predictors are added, which supports MO3. The matrix-based treatment generalizes the least squares fit from one predictor to many, reinforcing MO1 and MO3. None of the three focuses on residual plots or assumption checking, so MO4 will be addressed primarily through the module readings, lecture material, and your software-based assignments.

About These Videos

These videos were chosen from YouTube on October 2, 2026. Each has captions; the player turns them on. Videos on YouTube can be changed or removed — if one no longer plays, tell your instructor. If a player does not appear, use its Watch on YouTube link.

Simple Linear Regression

Introduction

Simple linear regression is the technique we use to describe the straight-line relationship between one predictor variable (x) and one response variable (y). In engineering practice, this is how you turn a scatter of measured data — load versus deflection, temperature versus reaction rate, flow rate versus pressure drop — into an equation you can use to explain behavior and predict new outcomes. The model has the familiar form y-hat = b0 + b1*x, where b0 is the intercept and b1 is the slope.

The core computational idea is the least squares method: among all possible lines, we choose the one that minimizes the sum of squared vertical distances (residuals) between the observed data points and the line. From that fitted line we then ask three separate questions. First, what do the coefficients mean physically — what does a one-unit change in x do to y, and is the intercept meaningful within the range of the data? Second, how well does the line fit — this is where R-squared, the proportion of variation in y explained by x, comes in. Third, is the relationship statistically real — this is the t-test on the slope (equivalently the F-test for the overall model) and its p-value.

As you watch the videos below, keep a running list of the formulas and what each quantity represents: the slope and intercept estimators, SST/SSR/SSE, R-squared, and the standard error. Pay attention to how each presenter moves from raw data to a fitted equation, and note the warning about extrapolating predictions outside the range of x values actually observed.

Video 1: Intro to Simple Linear Regression | Slope, y-intercept, R-square

Joshua Emmanuel · 19:21 · 10,355 views · 3y ago · captions — Watch on YouTube

Video 2: Simple Linear Regression Using Least Squares Estimators

LearnChemE · 10:59 · 9,729 views · 4y ago · captions — Watch on YouTube

Video 3: The Least Squares Regression Line

Dan Kernler · 9:49 · 2,938 views · 4y ago · captions — Watch on YouTube

About These Videos

These videos were chosen from YouTube on October 2, 2026. Each has captions; the player turns them on. Videos on YouTube can be changed or removed — if one no longer plays, tell your instructor. If a player does not appear, use its Watch on YouTube link.

Multiple Regression Models

Introduction

Multiple regression extends the simple linear model by allowing two or more predictor variables to explain variation in a single response. Instead of fitting a line through a scatterplot, you fit a surface described by an equation of the form y = b0 + b1x1 + b2x2 + ... + bkxk, where each slope coefficient describes the effect of one predictor while the other predictors are held constant. That "holding the others constant" idea is the single most important conceptual shift in this module, and it is also the source of most misinterpretation.

For engineers, multiple regression is the everyday tool for modeling outcomes that depend on several inputs at once: strength as a function of mix proportions and curing time, energy use as a function of temperature and occupancy, process yield as a function of several operating settings. Along with building the model, you need tools for judging it: R-squared and adjusted R-squared for overall fit, t-tests and p-values for individual coefficients, the F-test for the model as a whole, and diagnostics such as the variance inflation factor (VIF) when predictors are correlated with each other.

As you watch the videos below, focus on how each piece of software output maps to a decision. Ask yourself: what does this coefficient mean in the units of the problem? Why does adjusted R-squared behave differently from R-squared when a predictor is added? And when two predictors carry overlapping information, how does multicollinearity distort the coefficient estimates and their standard errors? Keep a worked example of your own nearby so you can follow along with the reasoning rather than just the arithmetic.

Video 1: Multiple Regression | Coefficients – Interpretation, C.I, Hypothesis Testing

Joshua Emmanuel · 8:06 · 132,587 views · 3y ago · captions — Watch on YouTube

About this video

A compact walkthrough of the coefficient section of a multiple regression software output. Based on its title and description, it covers how to read and interpret each estimated coefficient, how to construct confidence intervals for the coefficients, and how to carry out hypothesis tests on individual predictors using the reported standard errors, t-statistics and p-values.

Why this video

It targets the interpretation subtopic directly and in a short, focused format, which makes it a good first stop for learning to read regression output the way you will see it in software rather than in formula form.

Video 2: Statistics 101: Multiple Regression, Adjusted R-square

Brandon Foltz · 16:38 · 24,827 views · 5y ago · captions — Watch on YouTube

About this video

This Statistics 101 lesson focuses on adjusted R-squared as a model assessment statistic for multiple regression. It is likely to contrast adjusted R-squared with ordinary R-squared, explain why R-squared never decreases when predictors are added, and show how the adjustment penalizes unnecessary predictors when comparing candidate models.

Why this video

Brandon Foltz's Statistics 101 series is known for slow, example-driven explanations, and this video addresses both the model fit subtopic and the model building and variable selection subtopic, since adjusted R-squared is a common criterion for deciding which predictors to keep.

Video 3: L33: Multiple linear regression | variance inflation factor | part 1

IIT Madras - B.S. Degree Programme · 19:45 · 10,845 views · 4y ago · captions — Watch on YouTube

About this video

A university lecture from the IIT Madras B.S. Degree Programme covering multiple linear regression with an emphasis on the variance inflation factor. Expect a treatment of what multicollinearity is, how VIF is computed from regressions of each predictor on the remaining predictors, and how to use VIF values to flag predictors that are redundant or unstable.

Why this video

It is the most formal and detailed of the three and is the designated resource for the multicollinearity subtopic, giving you a diagnostic tool to use when coefficient estimates look unstable or contradict expectations.

How These Videos Support the Learning Outcomes

Together these videos carry you from reading output to judging it. The coefficient walkthrough supports MO2 and MO3 by showing how slope estimates, confidence intervals and p-values are interpreted when several predictors are present, which is exactly what distinguishes multiple from simple regression. The adjusted R-squared lesson reinforces MO2 and MO5 by giving you a defensible basis for choosing among competing models and for explaining to a non-technical audience why a bigger model is not automatically a better one. The VIF lecture supports MO4 by adding multicollinearity to your model-checking routine alongside the residual-based assumption checks, so that a model you report is one you have actually validated. MO1 remains the simple-regression foundation on which all of this is built.

About These Videos

These videos were chosen from YouTube on October 2, 2026. Each has captions; the player turns them on. Videos on YouTube can be changed or removed — if one no longer plays, tell your instructor. If a player does not appear, use its Watch on YouTube link.

Evaluating Regression Model Performance

Introduction

Fitting a regression line is only the first step. Once you have an equation, you have to ask harder questions: How much of the variation in the response does the model actually explain? Are the individual predictors contributing anything beyond chance? Does the model as a whole do better than simply predicting the mean? And, just as important, are the assumptions behind the least squares procedure reasonable for this data set? This topic gathers the standard tools engineers use to answer those questions — R-squared and adjusted R-squared, MSE and RMSE, coefficient p-values, the overall F-statistic, and residual diagnostics.

These ideas matter in this course because a regression model that looks good on a summary line can still be wrong in ways that cause bad engineering decisions. A high R-squared does not guarantee linearity, constant variance, or independent errors; adding predictors always raises R-squared, which is why adjusted R-squared exists; and residual plots often reveal curvature or funnel shapes that no single number will show you. Understanding the decomposition of total variability into explained and unexplained parts ties all of these metrics together, since R-squared, MSE, and the F-statistic are all built from sums of squares.

As you watch, keep track of three things. First, the vocabulary and arithmetic of sums of squares (total, regression/explained, error/residual) and how each metric is computed from them. Second, what a "good" residual plot looks like versus plots that signal a violated assumption. Third, the distinction between a model that fits the data you have and a model that will predict well on new data — that distinction motivates adjusted R-squared and RMSE as honest measures of error in the units of the response.

Video 1: Evaluating Regression Models

Leslie Myint · 10:30 · 10,225 views · 5y ago · captions — Watch on YouTube

About this video

This lecture-style video builds on the idea of examining regression residuals and moves into how we evaluate and compare regression models overall. Based on its title and description, it covers quantifying the errors a model makes and using those error summaries as criteria for judging model quality, which is the conceptual home of measures such as MSE, RMSE, and R-squared.

Why this video

It frames model evaluation as a systematic process rather than a single number, which directly supports the R-squared, error-metric, and residual-analysis subtopics in this module.

Video 2: Analyzing Residuals for Model Accuracy

LearnChemE · 10:00 · 3,544 views · 4y ago · captions — Watch on YouTube

About this video

Produced by LearnChemE at the University of Colorado Boulder, this short video focuses specifically on reading residuals to judge whether a fitted model is accurate. Expect discussion of plotting residuals, recognizing patterns that indicate a poor or incorrectly specified model, and using that visual evidence to decide whether the fit should be trusted.

Why this video

It is an engineering-faculty-produced treatment of residual analysis, so the examples and framing align well with how you will check linearity, homoscedasticity, and overall model validity in EGN3443.

Video 3: Decomposition of Variability: Sum of Squares | Statistics Tutorial

365 Data Science · 3:20 · 62,251 views · 5y ago · captions — Watch on YouTube

About this video

A concise tutorial from 365 Data Science that breaks total variability in the response into its components — the sum of squares total, the sum of squares due to regression, and the sum of squares of errors. It explains what each piece represents and how they relate to one another in a fitted regression.

Why this video

Nearly every metric in this topic (R-squared, adjusted R-squared, MSE, and the F-statistic) is assembled from these sums of squares, so this short, clearly presented video gives you the algebraic foundation the other videos build on.

How These Videos Support the Learning Outcomes

Together these videos move you from the mechanics of fitting (MO1) to the interpretation and judgment skills the module emphasizes. The sum-of-squares decomposition video supplies the quantities behind R-squared, adjusted R-squared, and the F-statistic, supporting your ability to read and interpret standard regression output (MO2). The model-evaluation video treats error metrics and comparison criteria, which is also how you decide whether extra predictors in a multiple regression are earning their place (MO3). The LearnChemE residual video targets MO4 directly by showing how residual plots expose violations of linearity and constant variance. Finally, being able to explain why a model is or is not trustworthy — not just report a number — is exactly the communication skill called for in MO5.

About These Videos

These videos were chosen from YouTube on October 2, 2026. Each has captions; the player turns them on. Videos on YouTube can be changed or removed — if one no longer plays, tell your instructor. If a player does not appear, use its Watch on YouTube link.

Interpreting Regression Results

Introduction

Fitting a regression line is only half the job. The other half is reading the output: what each coefficient means in the units of the problem, whether a predictor is statistically distinguishable from zero, how much of the variation in the response the model actually explains, and whether the model as a whole is doing anything useful. Statistical software will happily print a table of numbers for any dataset you hand it, so the engineer's value lies in interpretation, not computation.

In this course, regression output is the bridge between data and decisions. A slope coefficient tells you the expected change in the response per one-unit change in a predictor, holding the other predictors fixed; a p-value tells you how strong the evidence is against the hypothesis that the true coefficient is zero; R-squared and Adjusted R-squared describe how closely the fitted values track the observed data; and the F-test asks whether the predictors, taken together, explain more than chance would. Each piece answers a different question, and confusing them is the most common error students make.

As you watch the videos below, keep a blank regression output table in front of you and label the parts: coefficient, standard error, t statistic, p-value, confidence interval, SSR/SSE/SST, degrees of freedom, mean squares, F, R-squared, and standard error of the estimate. Ask yourself after each number: what practical statement about the engineering system could I write using this value, and what statement would be an overreach? Remember throughout that statistical significance is not the same as practical importance, that a high R-squared does not prove causation or guarantee valid assumptions, and that extrapolating beyond the range of your data is risky.

Video 1: Multiple Regression | Coefficients – Interpretation, C.I, Hypothesis Testing

Joshua Emmanuel · 8:06 · 132,587 views · 3y ago · captions — Watch on YouTube

About this video

A short walkthrough of the coefficients section of a multiple regression software output, using a worked problem. Based on the title and description, it covers how to interpret each estimated coefficient, how to build and read confidence intervals for the coefficients, and how to carry out hypothesis tests on individual predictors using the reported t statistics and p-values.

Why this video

It targets the first two subtopics directly, understanding coefficients and assessing statistical significance, and Joshua Emmanuel's worked-problem style is well suited to reading a real output table line by line.

Video 2: Multiple Regression | ANOVA Table | F-Test | R-square | Standard Error

Joshua Emmanuel · 4:05 · 105,910 views · 3y ago · captions — Watch on YouTube

About this video

This video works through the ANOVA portion of a multiple regression output for a stated problem involving 40 observations. It is likely to show how sums of squares and degrees of freedom produce the F-statistic for the overall model, and how R-squared and the standard error of the estimate are computed and interpreted from the same table.

Why this video

It is the natural companion to the coefficients video, covering the overall F-test and model fit measures, and it connects the ANOVA table arithmetic to the summary statistics students must interpret.

Video 3: Hypothesis Test for Simple Linear Regession

LearnChemE · 5:46 · 58,006 views · 4y ago · captions — Watch on YouTube

About this video

A concise treatment of hypothesis testing in the simple linear regression setting, organized by textbook with an accompanying spreadsheet. Expect it to set up the null and alternative hypotheses for the slope, compute the test statistic from the estimate and its standard error, and reach a conclusion about whether the predictor is significant.

Why this video

LearnChemE produces accurate, engineering-oriented tutorials, and this one grounds significance testing in the single-predictor case before you generalize the same logic to multiple regression output.

How These Videos Support the Learning Outcomes

Together these videos move from the simple-regression significance test to the full multiple regression output, which is exactly the progression the module outcomes require. The coefficients walkthrough and the simple linear regression hypothesis test support MO2 by showing how to interpret slope estimates, standard errors, confidence intervals, and p-values, and they reinforce MO1 by keeping the fitted equation in view. The ANOVA and F-test video completes MO2 with R-squared, Adjusted R-squared, the standard error of the estimate, and the overall model test, while the contrast between the simple and multiple regression examples supports MO3. Interpreting these numbers in the units of a stated problem is the first step toward MO5, communicating findings as actionable insights, and the cautions raised about significance, fit, and extrapolation point forward to the assumption checking required by MO4.

About These Videos

These videos were chosen from YouTube on October 2, 2026. Each has captions; the player turns them on. Videos on YouTube can be changed or removed — if one no longer plays, tell your instructor. If a player does not appear, use its Watch on YouTube link.

Applying Regression to Real-World Datasets

Introduction

So far in this module you have worked with regression in a fairly controlled setting: a clean dataset, a clear predictor, and a tidy output table. Real engineering data rarely arrives that way. This topic walks through the full applied workflow — choosing and cleaning a dataset, deciding which variables matter, framing a question the data can actually answer, fitting the model, reading the output in the context of the problem, checking whether the assumptions hold, refining the specification, and finally explaining what you found to someone who does not read regression tables.

This matters because almost every engineering role that uses statistics uses it this way. A coefficient only becomes useful when you can say what a one-unit change means in physical or financial terms, and a model only becomes trustworthy when you have looked at the residuals and admitted where it falls short. The judgment calls — which predictors to keep, whether an R-squared of 0.62 is good enough, whether an outlier is an error or a real signal — are the part that cannot be memorized.

As you watch, pay attention to the decisions being made rather than just the commands being typed. Notice how each presenter moves from data to a fitted model to a statement about the world, what evidence they use to justify keeping or dropping a variable, and how they describe uncertainty. Try to follow along in MATLAB, Python, or whatever tool your section uses, and ask yourself at each step how you would explain that step to a project manager.

Video 1: How to Fit a Linear Regression Model in MATLAB

MATLAB · 5:28 · 30,847 views · 3y ago · captions — Watch on YouTube

About this video

A short, practical walkthrough from MathWorks showing how to carry a linear regression problem through a typical workflow in MATLAB — loading data, fitting a model, and examining the results the software returns. The video is oriented around the mechanics of the tool rather than the underlying derivations, so expect to see the commands and output objects you would actually use on an assignment or project.

Why this video

It comes directly from the producer of the software many engineering students use, so the syntax and workflow shown are authoritative and current, and it maps cleanly onto the "building the regression model on real data" and "interpreting regression output" subtopics.

Video 2: Statistics 101: Multiple Regression, Interactive Model Building

Brandon Foltz · 9:31 · 19,338 views · 5y ago · captions — Watch on YouTube

About this video

Brandon Foltz works through the decision logic of multiple regression model building: when to keep a predictor in the model and when to remove it, and how that choice changes the fit and the interpretation. The emphasis is on the reasoning behind each step rather than on a single software package.

Why this video

This is the clearest match for the "refining and improving the model" and "identifying variables" subtopics, and Foltz's Statistics 101 series is known for slow, explicit explanations of why a decision is made, which is exactly what students struggle with when moving beyond a single predictor.

Video 3: Linear Regression Explained | Python Machine Learning Tutorial with Real Data

Coursera · 5:57 · 1,373 views · 1y ago · captions — Watch on YouTube

About this video

A hands-on Python tutorial from Coursera framed around a concrete applied question — predicting sales from advertising spend — that takes you from a real dataset to a fitted linear model. Because it is built around an actual business scenario, it also illustrates how a regression result gets stated as a practical conclusion.

Why this video

The applied framing supports the "formulating a research question" and "communicating findings" subtopics, and seeing the same ideas in a second language reinforces that regression is a method, not a software feature.

How These Videos Support the Learning Outcomes

Together these three videos take the mechanics you learned earlier in the module and place them in an applied workflow. The MATLAB walkthrough reinforces constructing and fitting a least-squares model and reading its output (MO1, MO2), while the Coursera tutorial shows the same process in Python anchored to a real predictive question, which supports applying regression to a real dataset and stating the result as an actionable insight (MO1, MO5). Foltz's model-building video addresses the move from one predictor to several and the judgment involved in keeping or excluding variables, directly supporting the distinction between simple and multiple regression and the interpretation of significance measures (MO2, MO3). For MO4, use the videos as a starting point but rely on your course materials and lab work for the formal residual and assumption checks, since none of these short videos is primarily a diagnostics tutorial.

About These Videos

These videos were chosen from YouTube on October 2, 2026. Each has captions; the player turns them on. Videos on YouTube can be changed or removed — if one no longer plays, tell your instructor. If a player does not appear, use its Watch on YouTube link.