Wiki Page, Math 13 - Perez Sandbox : Module 9 - Linear Regression (2 of 4)

Wiki Page
2026-01-17T21:15:56-08:00

The line of best fit

How do statisticians find the best line to summarize a linear pattern in data?

The line that best fits the data is the line that makes the best predictions. The closer the line is to the data, the better the predictions from the line.

EXAMPLE Prediction error and the line of best fit

Let’s begin with a simple data set with only five data points.

Which line appears to be a better summary of the linear pattern in the data?

Here are some initial observations:

The line on the left comes very close to passing through two of the five points, with the point (12, 7) also very close to the line. For these points, the predicted y-values from the line are very close to the observed y-values, which means very small prediction error. The points (8, 13) and (17, 14) are relatively far from the line, which translates into larger amounts of prediction error.

The line on the right does not pass through any of the points. It appears to pass through the middle of the distribution of the data. The predictions from this line will all have some error, but none of the predictions will have a large amount of error.

To visually represent the prediction errors, we drew a vertical line segment from each point to the line. The length of each vertical line segment is the amount that the predicted y-value deviates from the actual y-value for that data point. This length is the prediction error.

Which line best fits the data?

The line that best fits the data will have the smallest overall prediction error. At this point, identifying the line with the least amount of prediction error is a judgment call. We need a more precise way to measure the overall prediction error.

The most common measurement of overall prediction error is the sum of the squares of the errors, or SSE (sum of squared errors). The line with the smallest SSE is called the least-squares regression line. We call this line the “line of best fit.”

EXAMPLE Sum of squared errors (SSE)

We want to adjust the line until the overall prediction error for all points together is as small as possible.

When we locate the line of best fit, the sum of the errors (deviations) above the line will equal the sum of the errors (deviations) below the line.

Sound familiar? This is similar to what we discovered about deviations from the mean during our development of the standard deviation. The sum of the errors (deviations) above the mean equal the sum of the errors (deviations) below the mean.

To fix this issue, statisticians squared the deviations from the mean in the calculation of the standard deviation and they do the same thing here: square the errors.

Below are the scatterplots again. As before, each vertical line represents the error in a prediction. For each data point, the squared error is equal to the area of a yellow square. The SSE is the sum of these areas. The least-squares regression line is the line with the smallest SSE.

When we compare the sum of the areas of the yellow squares, the line on the left has an SSE of 57.8. The line on the right has a smaller SSE of 43.9; therefore, using the least-squares measurement, the line on the right is the better fit. It has a smaller sum of squared errors.

But is the line on the right the best fit? The answer is no. We can use StatCrunch to find the line with the smallest SSE for this data; it is y = 6.72 +0.26x. The SSE is 41.79.

The main point

The line of best fit is the line that comes as close as possible to all of the data points. "As close as possible" means that the sum of the squared errors is as small as possible. The line with the smallest sum of squared errors (smallest SSE) is appropriately named the least squares regression line.

Mark this page as done, then click the "Next" or > button to continue.

Content by the Open Learning Initiative and licensed under CC BY.