Wiki Page, Math 13 - Perez Sandbox : Module 20 - Introduction to Hypothesis Testing (4a of 5)

Wiki Page
2026-01-17T21:15:30-08:00

What does the P-value really mean?

Hypothesis testing appears in all upcoming modules. The process and the logic of the hypothesis test will always be the same, but the details will differ somewhat.

Every hypothesis test will use a P-value to make a decision about the population(s). The P-value is the connection between probability and decision-making in inference. Now we discuss the P-value in more depth and relate it to our previous work with sampling distributions, simulations and the normal model.

EXAMPLE What is a P-value?

Let’s return to the familiar example of the 2008 presidential election. In that election, newspapers reported that Obama received 40% of the white male vote. We wonder if a smaller percentage of white males will support Obama in the 2012 election. We define the following hypotheses and conduct a hypothesis test.

: The proportion of white males voting for Obama in 2012 is 0.40.

: The proportion of white males voting for Obama in 2012 is less than 0.40.

We select a random sample of 200 white male voters and find that 35% plan to vote for Obama in 2012. Clearly 35% is less than 40%. But is the difference statistically significant or due to chance? If the population proportion is 0.40, we expect to see sample proportions vary from this. But will sample proportions as small as or smaller than 0.35 occur very often? What’s the probability?

The probability (P-value) is about 0.078.

The P-value is the chance that a random sample of 200 white males will have, at most, 35% supporting Obama if 40% of this population supports Obama. It is a probability, a relative frequency.

This is quite a mouthful. We find that visualizing the sampling distribution helps us understand the P-value. Here is a diagram that may be helpful in interpreting the P-value.

Notice that the simulation is set to select samples from a population of p=0.40, which is the null hypothesis.

Notice that 117 out of 1,500 samples have sample proportions of 35% or less. 117 out of 1,500 is 0.078, the P-value.

In general, the P-value is the probability that sample results are as extreme as or more extreme than the result observed in the data if the null hypothesis is true. The phrase “as extreme as or more extreme than” means further from the center of the sampling distribution in the direction of the alternative hypothesis.

Note: You may recall the concept of a conditional probability. The P-value is a conditional probability. The condition is “the null hypothesis is true.”

Note: We can also look at the P-value in terms of error in the sample proportion. If 40% of this population support Obama, then our sample with 35% supporting Obama has a 5% error. From this perspective, the P-value is the chance that sample proportions supporting the alternative hypothesishave as much as or more error than the data. For this example, the P-value describes sample proportions less than 0.40 that deviate 0.05 or more from 0.40. This describes sample proportions at or below 0.35.

Comment

Recall when we investigated the conditions that make a normal model a good fit for the sampling distribution. When a normal model is a good fit, we use it to find probabilities.

In the Obama example, we can see that a normal model is a good fit for the sampling distribution, so we can find the P-value by calculating the z-score and using an applet. Below we created a diagram to remind you how the sampling distribution relates to the standard normal model of z-scores. We will find P-values using the normal model (if conditions are met), but, for now, we focus on what the P-value is and how to use it to make decisions.

Notice that all three models assume that Ho is true; in each case, we are randomly sampling from a population where p=0.40. In the Standard Normal curve, the mean p=0.40 becomes Z=0.

Notice that in each model we estimate the probability that a sample proportion is 0.35 or less. (Why? Because the alternative hypothesis is that the population proportion is less than 0.40.) In the Standard Normal curve the Z-score for 0.35 is -1.44.

Notice that the relative frequency from the simulation is approximately equal to the area under the normal curve, whether we are using a normal curve to represent the sampling distribution or whether we are standardizing scores and using the Standard Normal curve.

Notice that for our example the P-value from the standard normal curve (0.074) is not exactly equal to the relative frequency in the simulated sampling distribution (0.078), but it is close. Both of these values represent estimates of the probability we want.

In hypothesis testing you use the standard normal curve (or a similar model) to find P-values. For this reason, you will frequently see the P-value defined in terms of the “test statistic,” which is the z-score in our example. Here is a common definition: The P-value is the probability of obtaining a test statistic at least as extreme as the one that was actually observed, assuming the null hypothesis is true.

Mark this page as done, then click the "Next" or > button to continue.

Content by the Open Learning Initiative and licensed under CC BY.