Wiki Page, Math 13 - Perez Sandbox : Module 9 - Linear Regression (4 of 4)

Wiki Page
2026-01-17T21:15:56-08:00

Assessing the Fit of the Regression Line

How do statisticians assess the fit of the least-squares regression line?

When a scatterplot has a linear form, the least-squares regression line is the line that best fits the data. However, that does not mean that the regression line is a good fit (it is the best we can do, but it may not be that good.)

Statisticians use two different measures to assess the fit of the regression line:

R-sq : the square of the correlation coefficient

Standard error of the regression : the standard deviation of the prediction errors

R-squared

R-sq gives us information about whether we have the right explanatory variable in our linear regression model.

A large R-sq value tells us that the explanatory variable explains a large proportion of the changes in Y; our linear model based on X is good at explaining the variability in Y.

A small R-sq value tells us that changes in X account for a small proportion of the changes in Y. In other words, the linear model based on X is not doing a very good job accounting for the variability we see in Y. This suggests that we should look for another explanatory variable.

R-sq is . We usually interpret R-sq as a percentage but StatCrunch gives it decimal form, so it is a number between 0 and 1. R-sq does not have units; it is a percentage.

Standard error of the regression

Standard error is the standard deviation of the prediction errors. It is a non-negative number (like all standard deviations.) It's units are the units of the Y-variable (the response variable.)

is roughly a measure of the average amount that the predicted Y-values deviate from the observed Y-values, which is an estimate of the average distance that the data points are from the line. In other words, is roughly the average amount of prediction error for the regression line.

For those who like formulas, the standard error is related the sum of squared errors (SSE): , where n is the sample size.

EXAMPLE Interpreting R-sq and standard error

Previously, we analyzed the winning Olympic times in the 1500-meter run for men, X=years from 1896-2000. We observed a strong linear form with negative association in the scatterplot. Below is the complete StatCrunch print-out for the linear regression model.

Simple linear regression results:

Dependent Variable: Time
Independent Variable: Year
Time = 994.19341 - 0.39304496 Year
Sample size: 24
R (correlation coefficient) = -0.89075356
R-sq = 0.7934419
Estimate of error standard deviation: 6.7634302

Parameter estimates

Parameter | Estimate | Std. Err. | Alternative | DF | T-Stat | P-value
Intercept | 994.19341 | 83.377999 | 0 | 22 | 11.92393 | <0.0001
Slope | -0.39304496 | 0.042755741 | 0 | 22 | -9.1927996 | <0.0001

Analysis of variance table for regression model

Source | DF | SS | MS | F-stat | P-value
Model | 1 | 3865.7131 | 3865.7131 | 84.507565 | <0.0001
Error | 22 | 1006.3678 | 45.743989 | |
Total | 23 | 4872.0809 | | |

The bottom part of this print-out (below the title "Parameter estimates") contains information that we will not use in this course. Let's focus on the top part of the print-out, above the title "Parameter estimates."

First, a few notes to connect the StatCrunch print-out to familiar concepts:

StatCrunch uses the term "dependent variable" instead of "explanatory variable", and "independent variable" instead of "response variable."

The linear regression equation is Time = 994.19341 - 0.39304496 Year. Time is actually predicted Time.

Sample size: 24 tells us that there are 24 data points. We have winning times for 24 years.

R (correlation coefficient) = -0.89075356 is the familiar r-value, a measure of the strength and direction of the linear pattern in the data.

Now, let's interpret the meaning of R-sq and the standard error:

R-sq = 0.7934419.
Interpretation: 79% of the variability in winning times is explained by changes in the year of the Olympics. In other words, the passage of time is explaining a significant proportion of the improvement in winning times for the men's 1500-meter race. The remaining 21% (100-79%=21%) is due to other factors that are not part of our linear regression model, such as improvements in running shoes or more rigorous training regimes, etc.

Estimate of error standard deviation: 6.7634302
Interpretation: In StatCrunch the "estimate of error standard deviation" is the standard error of the regression. For our linear regression model, the average amount of prediction error is approximately 6.76 seconds. On average our predictions from the regression line will have an error of 6.76 seconds, which means a prediction will on average be about 6.76 seconds too high or too low.

EXAMPLE Interpreting R-sq and standard error for a weaker fit

Here again is the data on highway sign visibility and driver age.

The least-squares regression line is Distance = 576 − 3 * Age.
StatCrunch gives r = –0.793 and = 51.35.

We will use R-sq and to assess the fit of the regression line.

R-sq: r= −0.793, so

The linear relationship between age and sign legibility distance accounts for 63% of the variation in sign legibility distance. Other variables besides age account for 37% of the variation in sign legibility distance, such as sign size, letter height, or the contrast between the letters and their background.

Note that the R-sq value here is smaller (63%) than the R-sq value for the Olympics data (79%). This means that this linear regression model does not fit the data as well as the regression model for the Olympics data.

Standard error of the regression:

Previously, we used the regression equation to predict the sign legibility distance for a 60-year-old driver:
Predicted distance = 576 + (−3 * 60) = 396
How good is this prediction?

Unfortunately, there is no 60-year-old driver in the original data set of 30 drivers, so we cannot determine the prediction error. Instead, we use the as a measurement of average error.

StatCrunch gives = 51.35 feet as the average amount of prediction error. Based on the for this data, we estimate that our prediction of 396 feet has an error of 51 feet.

Note that the value here is larger (51.35) than the value for the Olympics data (6.76). This is further evidence that this linear regression model does not fit the data as well as the regression model for the Olympics data.

When we compare the two examples, which regression line is a better fit for the data?

Answer: The regression line for the Olympic data is a better fit because that line has a larger R-sq and a smaller .

A larger R-sq value tells us that the regression line accounts for a larger proportion of the variation in Y. In other words, the relationship between X and Y is doing a better job predicting Y.

The smaller value tells us that the regression line has smaller prediction errors on average. Smaller average error is always better.

Mark this page as done, then click the "Next" or > button to continue.

Content by the Open Learning Initiative and licensed under CC BY.