Wiki Page, Math 13 - Perez Sandbox : Module 20 - Introduction to Hypothesis Testing (5b of 5)

Wiki Page
2026-01-17T21:15:31-08:00

What Is the Probability That We Will Make a Type I Error?

If the significance level is 5% ( = 0.05), then 5% of the time we will reject the null hypothesis (when it is true!). Of course we will not know if the null is true. But if it is, the natural variability that we expect in random samples will produce "highly unlikely" results 5% of the time. This makes sense because we assume the null hypothesis is true when we create the sampling distribution. We look at the variability in random samples selected from the population described by the null hypothesis.

Similarly, if the significance level is 1%, then 1% of the time sample results will be rare enough for us to reject the null hypothesis. In this case, if the null hypothesis is actually true, then by chance alone, 1% of the time we will reject a true null hypothesis. The probability of a Type I error is therefore 1%.

In general, the probability of a Type I error is .

What Is the Probability That We Will Make a Type II Error?

The probability of a Type I error, if the null hypothesis is true, is equal to the significance level. The probability of a Type II error is much more complicated to calculate and beyond the scope of this course.

We can reduce the risk of a Type I error by using a lower significance level. The best way to reduce the risk of a Type II error is by increasing the sample size. In theory, we could also increase the significance level, but doing so would increase the likelihood of a Type I error at the same time. We discuss these ideas further in a later module.

Mark this page as done, then click the "Next" or > button to continue.

Content by the Open Learning Initiative and licensed under CC BY.