Assessing Regression Model Assumptions

Linear regression is one of the most widely used statistical modeling techniques, but its validity depends critically on a set of underlying assumptions. When these assumptions hold, ordinary least squares (OLS) estimation produces the best linear unbiased estimates (BLUE) of the regression coefficients—a property guaranteed by the Gauss-Markov theorem. When they are violated, the consequences range from inefficient estimates to severely biased coefficients, rendering the model's predictions, hypothesis tests, and confidence intervals untrustworthy. A rigorous analyst does not simply fit a regression model and report results; they systematically diagnose whether the model's assumptions are satisfied and take corrective action when they are not. The four core assumptions are linearity, homoscedasticity, independence of errors, and normality of residuals. Each assumption addresses a different facet of the relationship between predictors, outcomes, and error terms, and each has its own diagnostic tools and remedies.

Linearity Assumption

The linearity assumption states that the expected value of the outcome variable is a linear function of the predictors. In practical terms, this means that a one-unit change in a predictor produces a constant change in the outcome regardless of where along the predictor's range you are. This assumption is foundational: the entire mathematical machinery of OLS—minimizing the sum of squared residuals to find a straight-line fit—only makes sense if the underlying relationship actually is linear.

When the linearity assumption is violated, the regression coefficients become biased. Bias means the estimated coefficients do not converge to the true population parameters even with infinitely large samples. More concretely, the model will systematically overpredict in some regions of the predictor space and underpredict in others. For example, if the true relationship between hours of study and exam score follows a diminishing-returns curve (each additional hour yields smaller gains than the last), a straight-line model will underestimate scores at moderate study hours and overestimate them at very high hours. This pattern of systematic inaccuracy across the range of the predictor is the hallmark of a violated linearity assumption.

Before fitting a model, a straightforward first step is to examine scatterplots of each predictor against the outcome variable. A roughly elliptical cloud of points suggests linearity; a curved band, a fan shape, or a cluster pattern suggests otherwise. These pre-modeling plots cost nothing and can save significant effort by identifying transformations needed before estimation begins. For example, a scatterplot of income versus spending might reveal a curved relationship that becomes approximately linear after taking the logarithm of income.

Transformations are the primary remedy for nonlinearity. Common approaches include:

• Logarithmic transformation: Appropriate when the outcome or predictor spans several orders of magnitude, or when the relationship shows rapid growth that levels off. For instance, modeling log(salary) as a function of years of experience often yields a more linear relationship than modeling salary directly.
• Square-root transformation: Useful for count data or variables with right skew, where the curvature is less extreme than a log relationship would imply.
• Polynomial terms: Adding a squared or cubic term for a predictor (e.g., including both x and x²) allows the regression line to bend, capturing quadratic or cubic curvature while remaining within the linear regression framework.
• Splines and piecewise regression: For more complex nonlinear patterns, spline models fit separate polynomial segments across ranges of the predictor, joined smoothly at knot points.

It is worth noting that "linear regression" refers to linearity in the parameters, not necessarily in the predictors themselves. A model like Y = β₀ + β₁x + β₂x² is still a linear regression model because it is linear in the β coefficients, even though it models a curved relationship in x.

Homoscedasticity Assumption

Homoscedasticity (from Greek: homos = same, skedasis = dispersion) requires that the variance of the residuals is constant across all levels of the fitted values and predictors. In other words, the spread of the errors around the regression line should be roughly the same whether the predicted value is small, medium, or large. When this assumption is satisfied, every observation contributes equally reliable information to the estimation of the regression coefficients.

Heteroscedasticity—the violation of this assumption—occurs when the error variance systematically changes. A classic real-world example: modeling household expenditure as a function of income typically reveals that wealthier households show much more variability in spending than lower-income households. The errors are small and tightly clustered at low income levels, then fan out dramatically at high income levels. This "fan" or "cone" shape is the most common visual signature of heteroscedasticity.

The consequences of heteroscedasticity are important to understand precisely. OLS coefficient estimates remain unbiased under heteroscedasticity—the estimates still point in the right direction on average—but they are no longer efficient. Efficiency means having the smallest possible variance among unbiased estimators, and OLS loses this property when error variances differ. More critically, the standard errors computed by standard OLS software are incorrect under heteroscedasticity. They may be too small or too large depending on the pattern of variance, which means that t-statistics, p-values, and confidence intervals are all distorted. A predictor might appear statistically significant when it is not (false positive) or fail to reach significance when it truly matters (false negative).

The most common remedies include:

• Weighted least squares (WLS): Rather than treating all observations equally, WLS assigns weights inversely proportional to the estimated error variance of each observation. Observations with high variance (less reliable) receive lower weight, and those with low variance (more reliable) receive higher weight. This restores efficiency and produces correct standard errors.
• Heteroscedasticity-consistent (HC) standard errors: Also called "robust standard errors" or "sandwich estimators," these adjust the standard error calculations to account for non-constant variance without changing the coefficient estimates. They are widely used in econometrics and available in most statistical packages.
• Variance-stabilizing transformations: Applying a transformation to the outcome variable can sometimes equalize the variance. The log transformation is particularly effective when variance increases proportionally with the mean. The Box-Cox procedure provides a systematic way to identify the optimal power transformation.

Visually, heteroscedasticity is detected in a residuals vs. fitted values plot by looking for a systematic change in the spread of residuals as fitted values increase. To formally confirm suspicions raised by visual inspection, the Breusch-Pagan test provides a statistical hypothesis test. It works by regressing the squared residuals from the original model on the predictors; if the predictors significantly explain the variation in squared residuals, the null hypothesis of homoscedasticity is rejected. The White test is a related but more general alternative that also detects nonlinear forms of heteroscedasticity.

Independence of Errors Assumption

The independence assumption requires that the residuals (errors) from the regression model are uncorrelated with each other. Each observation's error should carry no information about any other observation's error. This assumption is most naturally satisfied when data are collected through random sampling from a large population, where there is no reason to expect one person's error to be related to another's.

The assumption is most commonly violated in two settings. First, in time-series data, observations are collected sequentially over time, and adjacent observations often share common influences—economic conditions, seasonal effects, or carry-over from previous periods—that induce correlation in the errors. Second, in clustered or hierarchical data (students within schools, patients within hospitals), observations within the same cluster tend to be more similar to each other than to observations from other clusters, which creates within-cluster correlation in the errors.

When errors are correlated—a condition called autocorrelation in time series or intraclass correlation in clustered data—the standard errors computed by OLS are underestimated. This happens because correlated observations contain less independent information than the sample size suggests: 100 highly correlated observations do not provide 100 independent pieces of information. The artificially small standard errors make coefficients look more statistically significant than they truly are, inflating the Type I error rate. In practice, this means a researcher might conclude that a predictor significantly explains variation in the outcome when, after properly accounting for error dependence, the evidence is far weaker.

The Durbin-Watson statistic is the classic diagnostic for detecting first-order autocorrelation (correlation between adjacent errors) in time-series regression. The statistic ranges from 0 to 4, with a value of 2 indicating no autocorrelation. Values substantially below 2 suggest positive autocorrelation (common in economic time series), while values substantially above 2 suggest negative autocorrelation. Critical values depend on the sample size and number of predictors, and tables or software provide the thresholds. For more complex autocorrelation structures (e.g., seasonal patterns), the Ljung-Box test on the residuals is more appropriate.

Remedies for violated independence depend on the source of the problem:

• Time-series models: Autoregressive integrated moving average (ARIMA) models or generalized least squares with an autocorrelated error structure explicitly model the dependence between observations.
• Mixed-effects (multilevel) models: For clustered data, mixed-effects models include random effects for clusters, which absorbs the within-cluster correlation and produces correct standard errors.
• Lagged variables: In time-series contexts, adding lagged values of the outcome or predictors as regressors can capture the temporal dependency and reduce residual autocorrelation.
• Cluster-robust standard errors: Analogous to heteroscedasticity-robust standard errors, these correct standard errors for within-cluster correlation without changing the model structure.

Normality of Residuals

The normality assumption states that the residuals of the regression model are normally distributed. It is important to clarify what this assumption does—and does not—require. It does not require that the outcome variable or the predictors themselves be normally distributed. It requires only that the errors (residuals) around the regression line follow a normal distribution. Furthermore, this assumption is primarily needed for inference—that is, for the validity of p-values, t-tests, F-tests, and confidence intervals—not for the point estimates of the coefficients themselves, which remain unbiased regardless of the error distribution.

In large samples, the central limit theorem provides some protection: even if individual residuals are not perfectly normal, the sampling distributions of the coefficient estimates will be approximately normal for large n, making inference approximately valid. This is why normality violations are generally more concerning in small samples, where the central limit theorem has not yet kicked in, and less critical in large datasets.

Departures from normality can take several forms, each with different implications:

• Heavy tails (leptokurtosis): More extreme values than a normal distribution predicts, inflating the apparent variance of residuals and making standard errors too large.
• Skewness: Asymmetry in the residual distribution, often caused by a skewed outcome variable or the presence of influential outliers.
• Multimodality: Two or more peaks in the residual distribution, which may suggest that important group differences or categorical variables have not been accounted for in the model.

The standard visual tools for assessing normality are the Q-Q plot (quantile-quantile plot) and the histogram of residuals. In a Q-Q plot, the quantiles of the observed residuals are plotted against the theoretical quantiles of a standard normal distribution. If residuals are normally distributed, the points fall along a straight diagonal line. Systematic deviations reveal the nature of the departure: an S-shaped curve indicates skewness, while points curving upward at both ends indicate heavy tails. Histograms are more intuitive but less sensitive for detecting subtle deviations, particularly in the tails.

For formal testing, the Shapiro-Wilk test is widely regarded as one of the most powerful tests for normality in small to moderate samples. It tests the null hypothesis that the residuals are drawn from a normal distribution; a significant p-value (typically p < 0.05) provides evidence against normality. The Kolmogorov-Smirnov test and the Anderson-Darling test are alternatives, with Anderson-Darling placing greater weight on the tails. A practical caution: in very large samples, these formal tests will detect even trivially small departures from normality as statistically significant, even when the deviation has no practical effect on inference. Conversely, in very small samples, they may lack the power to detect meaningful non-normality. This is why experienced analysts typically rely primarily on visual methods and use formal tests as supplementary confirmation.

Consequences of Assumption Violations on Model Integrity

It is useful to synthesize how each assumption violation specifically damages a model's integrity, distinguishing between effects on coefficient estimates versus effects on inference.

This table makes clear that linearity violations are the most damaging because they bias the coefficient estimates themselves—the model is learning the wrong relationships. The other three violations primarily damage inference rather than the coefficients, but this is still serious: a model with unbiased coefficients but unreliable p-values and confidence intervals cannot support trustworthy conclusions about which predictors matter and by how much. Biased coefficients from violated linearity also directly harm predictive performance: the model will make systematically wrong predictions in identifiable regions of the predictor space, which is detectable through cross-validation on held-out data.

Diagnosing Assumptions Through Residual Analysis

The residuals of a fitted model—the differences between observed and predicted values—are the primary diagnostic evidence for assumption assessment. A well-specified model with satisfied assumptions should produce residuals that are random noise: no patterns, no trends, no systematic structure. Any detectable pattern in the residuals is a signal that something about the model or its assumptions needs attention.

The residuals vs. fitted values plot is arguably the single most informative diagnostic plot available. It displays the raw residuals on the vertical axis against the fitted (predicted) values on the horizontal axis. Under ideal conditions, the points scatter randomly around the horizontal zero line with no discernible pattern and roughly constant spread. Specific patterns to watch for include:

• Curved or nonlinear pattern: A U-shaped or arc-shaped band of residuals indicates that the linearity assumption is violated—the model is missing a nonlinear component.
• Fan or cone shape: Residuals that spread out as fitted values increase (or decrease) signal heteroscedasticity.
• Alternating positive and negative residuals in sequence: In time-ordered data, this suggests autocorrelation.

The scale-location plot (also called the spread-location plot) is a refinement specifically for assessing homoscedasticity. It plots the square root of the absolute standardized residuals against the fitted values. Because the square root compresses the scale and absolute values eliminate sign, this plot makes it easier to see whether the average magnitude of the residuals is constant across the fitted range. A flat, horizontal smoother line through the points indicates homoscedasticity; an upward or downward trend indicates that variance changes systematically with the level of the fitted values.

The Q-Q plot of residuals, as described above, is the standard tool for assessing normality. In the context of a full diagnostic suite, it complements the residuals vs. fitted plot by examining the distributional shape of residuals rather than their relationship to fitted values.

In practice, a comprehensive regression diagnostic workflow combines all three plots—residuals vs. fitted, scale-location, and Q-Q—along with a fourth plot showing residuals vs. leverage to identify influential observations. Together, these plots give a complete picture of linearity, homoscedasticity, normality, and the influence of individual data points. Statistical software such as R's plot(model) function, Python's statsmodels diagnostic plots, or SPSS's regression diagnostics output all these plots with a single command, making systematic assumption checking a routine rather than a laborious addition to analysis.

The discipline of assumption checking is not merely a statistical formality. It is the mechanism by which an analyst ensures that the numbers produced by a regression model can be defended, communicated, and acted upon with confidence. A regression model whose assumptions have not been examined is a model whose results cannot be fully trusted, regardless of how large the dataset or how sophisticated the software used to estimate it.