3 Properties of the Least Squares Estimators and Model Assumptions

“The most important questions of life are, for the most part, really only problems of probability.” - Pierre Simon, Marquis de Laplace

In the simple linear regression model \[\begin{align*} y = \beta_0 + \beta_1 x + \varepsilon \end{align*}\] we have already discussed how to estimate \(\beta_0\) and \(\beta_1\) from sample data using least squares. In this chapter, we take the next step: we ask what kind of estimators least squares produces, what assumptions make those estimators reliable, and how we estimate the amount of random variation around the regression line.

Example 3.1

Fitting the `mtcars` regression line in R

The mtcars data were extracted from the 1974 Motor Trend US magazine and contain fuel consumption and several automobile design and performance measurements for 32 automobiles from the 1973-1974 model years.

We first make a scatterplot of mpg versus wt.

library(tidyverse)

data("mtcars")

mtcars |> 
  ggplot(aes(x = wt, y = mpg)) +
  geom_point()

A linear relationship appears reasonable for modeling mpg based on wt.

Now fit the least squares line using the lm() function in R. The model is specified with a formula: the response variable appears to the left of ~, and the predictor variable appears to the right.

fit <- lm(mpg ~ wt, data = mtcars)
fit


Call:
lm(formula = mpg ~ wt, data = mtcars)

Coefficients:
(Intercept)           wt  
     37.285       -5.344

The fitted line, or prediction equation, is approximately \[\begin{align*} \hat{y} = 37.285 - 5.344x \end{align*}\] where \(x\) is automobile weight, measured in thousands of pounds, and \(\hat y\) is the predicted miles per gallon.

The slope tells us that heavier cars tend to have lower fuel economy: for each additional thousand pounds of weight, the predicted fuel economy decreases by about 5.344 mpg.

3.1 Properties of the Least Squares Estimators

3.1.1 Linear Estimators

The least squares estimators are linear functions of the observations \(y_1,\ldots,y_n\). That means both \(b_0\) and \(b_1\) can be written as linear combinations of the observed response values.

Since \(y\) is the variable we want to model, an estimator for a parameter that takes the form of a linear combination of the \(y\) values is called a linear estimator.

Two meanings of linear

The phrase linear estimator does not mean the same thing as linear relationship.

A linear relationship means the mean response follows a straight-line pattern, such as \(E(y)=\beta_0+\beta_1x\).
A linear estimator means an estimate can be written as a weighted sum of the observed response values, such as \(\sum a_i y_i\).

In this chapter, when we say \(b_0\) and \(b_1\) are linear estimators, we mean the second idea.

3.1.2 \(b_1\) as a Linear Estimator

We can express \(b_1\) as \[ \begin{align} b_1 & = \sum k_i y_i \end{align} \tag{3.1}\] where \[\begin{align*} k_i & = \frac{x_i-\bar{x}}{\sum \left(x_i-\bar{x}\right)^2} \end{align*}\]

For those who want to see the math:

We first note that \(\sum(x_i-\bar{x})=0\).

We now rewrite \(b_1\) as \[\begin{align*} b_1 &= \frac{\sum (x_i-\bar{x})(y_i-\bar{y})}{\sum (x_i-\bar{x})^2}\\ &= \frac{\sum (x_i-\bar{x})y_i-\bar{y}\sum (x_i-\bar{x})}{\sum (x_i-\bar{x})^2}\\ &= \frac{\sum (x_i-\bar{x})y_i}{\sum (x_i-\bar{x})^2}\\ &= \left(\frac{1}{\sum (x_i-\bar{x})^2}\right)\sum (x_i-\bar{x})y_i. \end{align*}\]

From Equation 3.1, \(b_1\) is a linear combination of the \(y\) values because the \(k_i\) values are known constants. In regression, the observed \(x_i\) values are treated as fixed once the data have been collected.

3.1.3 \(b_0\) as a Linear Estimator

We can write \(b_0\) as \[ \begin{align} b_0 & = \sum c_i y_i \end{align} \tag{3.2}\] where \[\begin{align*} c_i & = \frac{1}{n}-\bar{x}k_i. \end{align*}\]

Therefore, \(b_0\) is also a linear combination of the observed response values.

For those who want to see the math:

We can rewrite \(b_0\) as \[\begin{align*} b_0 &= \bar{y}-b_1\bar{x}\\ &= \frac{1}{n}\sum y_i-\bar{x}\sum k_i y_i\\ &= \sum\left(\frac{1}{n}-\bar{x}k_i\right)y_i. \end{align*}\]

Example 3.2: Seeing the intercept and slope as weighted sums

Suppose we have three observations with predictor values \[\begin{align*} x_1=1,\qquad x_2=2,\qquad x_3=4. \end{align*}\]

Then \(\bar{x}=7/3\), and \[\begin{align*} \sum (x_i-\bar{x})^2 = \frac{14}{3}. \end{align*}\]

The weights for \(b_1\) are \[\begin{align*} k_1=-\frac{2}{7},\qquad k_2=-\frac{1}{14},\qquad k_3=\frac{5}{14}. \end{align*}\]

So the slope estimator is \[\begin{align*} b_1 = -\frac{2}{7}y_1-\frac{1}{14}y_2+\frac{5}{14}y_3. \end{align*}\]

If the observed response values are \(y_1=3\), \(y_2=5\), and \(y_3=8\), then \[\begin{align*} b_1 &= -\frac{2}{7}(3)-\frac{1}{14}(5)+\frac{5}{14}(8)\\ &= \frac{23}{14}\\ &\approx 1.64. \end{align*}\]

For the intercept, \[\begin{align*} c_i=\frac{1}{n}-\bar{x}k_i, \end{align*}\] which gives \[\begin{align*} c_1=1,\qquad c_2=\frac{1}{2},\qquad c_3=-\frac{1}{2}. \end{align*}\]

Thus \[\begin{align*} b_0 = y_1+\frac{1}{2}y_2-\frac{1}{2}y_3 = 3+\frac{1}{2}(5)-\frac{1}{2}(8) = 1.5. \end{align*}\]

The key idea is that the \(x\) values determine the weights. Once those weights are fixed, \(b_0\) and \(b_1\) are built from the observed \(y\) values.

The practical payoff is that \(b_0\) and \(b_1\) are themselves random variables. If we repeatedly collected new samples from the same population, the observed \(y\) values would change, so the fitted intercept and slope would also change. The model assumptions below help us describe that sample-to-sample variation.

3.2 Model Assumptions

Now let’s focus on the random component \(\varepsilon\) in the probabilistic model and its connection to the errors in estimating \(\beta_0\) and \(\beta_1\).

Specifically, we want to understand how the probability distribution of \(\varepsilon\) influences how accurately the fitted model represents the true relationship between the response variable \(y\) and the predictor variable \(x\).

We make four key assumptions about the probability distribution of \(\varepsilon\):

The mean of the error distribution is 0. This means that, on average, the errors cancel out for each value of the predictor variable \(x\). Consequently, the mean value of \(y\) for a given \(x\) is \[\begin{align*} E(y) = \beta_0 + \beta_1x. \end{align*}\]
The variance of the error distribution is constant across all values of \(x\). For our linear model, this means the variance of \(\varepsilon\) is the same value, \(\sigma^2\), regardless of the value of \(x\).
The probability distribution of \(\varepsilon\) is normal.
The errors for different observations are independent. In other words, the error for one observation does not influence the errors for other observations.

Example 3.3: Translating the model assumptions into plain language

Suppose \(x\) is study time and \(y\) is exam score. The simple linear regression model says that students with the same study time may still have different exam scores because of random error.

The four assumptions would mean:

Mean zero: Among all students who studied the same number of hours, the positive and negative errors balance out around the mean score.
Constant variance: The spread of exam scores around the mean score is about the same for students who studied 2 hours, 6 hours, or 10 hours.
Normality: At each study-time value, the errors are approximately mound-shaped and symmetric around 0.
Independence: One student’s error does not determine another student’s error.

The assumptions are not saying that all exam scores are normally distributed. They are statements about the errors around the regression line.

The implications of the first three assumptions can be seen in Figure 3.1, which shows distributions of errors for four particular values of \(x\).

Figure 3.1: The probability distribution of \(\varepsilon\). At every value of \(x\), there is a distribution of \(y\) values that satisfy the four assumptions listed above. The red points are observed values.

From Figure 3.1, the error distributions are normal, centered at 0, and have constant variance \(\sigma^2\).

The line in the middle of each curve represents the mean value of \(y\) for a given value of \(x\). The line of means is given by \[ E(y) = \beta_0 + \beta_1x. \]

These assumptions allow us to create measures of reliability for the least squares estimators and to develop hypothesis tests for evaluating the usefulness of the least squares line.

Various diagnostic techniques are available for checking the validity of these assumptions, and those diagnostics suggest remedies when the assumptions seem unreasonable. Therefore, it is important to use diagnostic tools in every regression analysis. We will discuss these techniques in detail later.

In practice, the assumptions do not need to hold exactly for least squares estimators and test statistics to have the reliability we expect from a regression analysis. The assumptions will be sufficiently satisfied for many real-world applications.

Example 3.4: Recognizing assumption problems from a residual plot

Suppose a residual plot shows small residuals for low values of \(x\) but much larger residuals for high values of \(x\). The plot has a funnel shape.

This pattern mainly raises concern about the constant variance assumption. The model may still capture the general direction of the relationship, but our usual formulas for standard errors, confidence intervals, and hypothesis tests may no longer be trustworthy without adjustment.

If the residual plot instead showed a curved pattern, the concern would be different: it would suggest that the mean relationship may not be linear. If residuals from nearby time periods tended to have the same sign, the concern would be independence.

3.3 An Estimator of \(\sigma^2\)

The variability of random error, measured by its variance \(\sigma^2\), plays a crucial role in the accuracy of estimating model parameters \(\beta_0\) and \(\beta_1\), as well as in the precision of predictions when using \(\hat{y}\) to estimate \(y\) for a given value of \(x\). As a result, \(\sigma^2\) appears in the formulas for confidence intervals and test statistics.

In most real-world scenarios, \(\sigma^2\) is unknown and must be estimated using the available data. The best estimate for \(\sigma^2\) is \(s^2\), calculated by dividing the sum of squared residuals by the associated degrees of freedom. The sum of squared residuals is \[ SSE = \sum \left(y_i-\hat{y}_i\right)^2. \]

In a simple linear regression model, 2 degrees of freedom are used to estimate the y-intercept and slope, leaving \(n-2\) degrees of freedom for estimating the error variance. Thus, the estimate of \(\sigma^2\) is \[ \begin{align*} s^2 &= \frac{SSE}{n-2}\\ &= \frac{\sum \left(y_i-\hat{y}_i\right)^2}{n-2}. \end{align*} \]

The value of \(s^2\) is called the mean square error (MSE).

The value \[\begin{align*} s &= \sqrt{s^2} \end{align*}\] is called the standard error of the regression model or the root mean square error (RMSE).

Units matter

SSE and MSE are measured in squared response units because residuals are squared. RMSE is measured in the original response units, which often makes it easier to interpret.

For example, if the response variable is mpg, then RMSE is also measured in mpg.

Using the empirical rule, when the model assumptions are reasonable, we expect approximately 95% of observed \(y\) values to lie within about \(2s\) of their respective least squares predicted values, \(\hat y\).

Review: The Empirical Rule

The empirical rule applies to distributions that are mound-shaped and symmetric. It states that approximately 68% of the distribution is within one standard deviation of the mean, approximately 95% is within two standard deviations of the mean, and approximately 99.7% is within three standard deviations of the mean. Since we assume \(\varepsilon\) is normally distributed, the empirical rule can help us interpret the typical size of prediction errors.

Example 3.5: Estimating the standard deviation with the `mtcars` model

We can use the summary() function with the fitted lm() object to obtain summary statistics for the regression model.

fit <- lm(mpg ~ wt, data = mtcars)

fit |> summary()


Call:
lm(formula = mpg ~ wt, data = mtcars)

Residuals:
    Min      1Q  Median      3Q     Max 
-4.5432 -2.3647 -0.1252  1.4096  6.8727 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  37.2851     1.8776  19.858  < 2e-16 ***
wt           -5.3445     0.5591  -9.559 1.29e-10 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 3.046 on 30 degrees of freedom
Multiple R-squared:  0.7528,    Adjusted R-squared:  0.7446 
F-statistic: 91.38 on 1 and 30 DF,  p-value: 1.294e-10

The residual standard error, or RMSE, is approximately 3.046 mpg.

We can obtain the ANOVA table for the fitted model with the following code:

anova(fit)

Analysis of Variance Table

Response: mpg
          Df Sum Sq Mean Sq F value    Pr(>F)    
wt         1 847.73  847.73  91.375 1.294e-10 ***
Residuals 30 278.32    9.28                      
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

The MSE is approximately 9.28. Notice that \[\begin{align*} \sqrt{9.28} \approx 3.05, \end{align*}\] which matches the residual standard error reported by summary().

In context, the fitted model’s typical prediction error is about 3 mpg. Using the rough \(2s\) guideline, many observed cars would be expected to have fuel economies within about \(2(3.046)=6.092\) mpg of their fitted values, assuming the model assumptions are reasonable.

3.4 Recap

In this chapter, we moved from fitting a least squares line to understanding why that line can be used for inference.

Idea	Meaning
Least squares estimators	The formulas used to estimate \(\beta_0\) and \(\beta_1\) from sample data.
Linear estimator	An estimator that can be written as a weighted sum of the observed response values.
Sampling variation	Different samples usually produce different fitted intercepts and slopes.
Mean-zero error	The regression line describes the mean response because errors average to 0 at each value of \(x\).
Constant variance	The spread of errors is the same across values of \(x\).
Normal errors	The error distribution is assumed to be approximately normal, which supports common inference procedures.
Independent errors	The error from one observation does not influence the error from another observation.
MSE	The estimate \(s^2=SSE/(n-2)\) of the error variance \(\sigma^2\).
RMSE	The estimate \(s=\sqrt{s^2}\) of the error standard deviation, measured in the original units of \(y\).

3.5 Check your understanding

Problems

Why are \(b_0\) and \(b_1\) considered random variables, even though they are calculated from formulas?
What does it mean to say that \(b_1\) is a linear estimator? Does this mean the relationship between \(x\) and \(y\) must be linear?
In the expression \(b_1=\sum k_i y_i\), why are the \(k_i\) values treated as constants?
Explain why the assumption \(E(\varepsilon)=0\) connects the regression line to the mean response \(E(y)\).
A residual plot has a clear funnel shape. Which regression assumption is most directly in question, and why does that matter?
Why is the normality assumption mainly important for inference rather than for simply drawing the least squares line?
Why do we divide SSE by \(n-2\) rather than by \(n\) when estimating \(\sigma^2\) in simple linear regression?
A model has RMSE \(s=4.2\). Interpret this value in context-free language. Why is RMSE often easier to explain than MSE?

Solutions

They vary from sample to sample. Once a sample is collected, \(b_0\) and \(b_1\) are fixed numbers. But before sampling, the response values are random, so the fitted intercept and slope are random too. A different sample would usually produce a different fitted line.
A weighted sum of responses. Saying \(b_1\) is a linear estimator means it can be written as a weighted sum of the observed response values, \(b_1=\sum k_i y_i\). This is different from saying the mean relationship between \(x\) and \(y\) is linear. One statement is about the form of the estimator; the other is about the form of the model.
The weights come from the predictor values. The \(k_i\) values depend only on the observed \(x\) values. Once the data’s predictor values are fixed, the weights are fixed, so the randomness in \(b_1\) comes from the response values.
Mean-zero error centers the model on the regression line. If \(y=\beta_0+\beta_1x+\varepsilon\) and \(E(\varepsilon)=0\), then the average value of \(y\) at a given \(x\) is \(E(y)=\beta_0+\beta_1x\). The error term explains why individual observations vary around that mean line.
Constant variance. A funnel shape suggests that the spread of residuals changes as \(x\) changes. This matters because standard errors, confidence intervals, and hypothesis tests rely on assumptions about the variability of the errors.
The line can be fit without normal errors. Least squares can compute a fitted line whether or not the errors are normal. The normality assumption becomes more important when we use the fitted model to make probability-based statements, such as confidence intervals and hypothesis tests.
Two parameters have been estimated. In simple linear regression, we estimate both an intercept and a slope. That uses 2 degrees of freedom, leaving \(n-2\) degrees of freedom to estimate the error variance.
Typical error size. An RMSE of 4.2 means that observed responses typically differ from their fitted values by about 4.2 response units. RMSE is easier to explain than MSE because RMSE is in the original units of the response variable, while MSE is in squared units.