Wiki Page, Math 13 - Perez Sandbox : Module 24 - Matched Pairs Practice Solutions (4b of 5)

Wiki Page
2026-01-17T21:15:43-08:00

This page contains the entire matched pairs hypothesis test that was the focus of the previous quiz. We also use this as an opportunity to review the types of conclusions that can be drawn from experiments.

Scenario: An instructor at Los Medanos College conducted an experiment with her statistics class to study the effect of texting on concentration. She created two audio clips in which she read two different lists of words. The treatment required students to send a short text message to a friend while listening to one of the audio clips. In the control setting, students simply listened to one of the audio clips. Everyone wore earphones and listened to the audio clips in the same order. But a coin flip determined who was and was not texting each time. After each listening session, students had 2 minutes to write down all the words they could remember. Twenty three students participated in the experiment.

Other information we are given:

The mean of the differences for students in the sample is -6.96 and the standard deviation is 3.88.

Here are the distribution of word counts:

Hypotheses:

The null hypothesis is a claim of “no difference” or "no treatment effect." A student who performs the same with and without texting has a difference in word counts of zero. Similarly, if the students overall do not perform differently in the control and treatment settings, the mean of their differences is 0.

:

The alternative hypothesis is that texting (the treatment) reduces concentration, i.e. students will remember fewer words when texting. Let's define the variable to be "word count treatment minus word count control." With this definition, if students remember fewer words when texting, the difference will be negative. Therefore, the alternative hypothesis says the mean is negative.

: , where is the mean of the differences (treatment minus control).

Checking conditions:

Is there a random process that allows probability computations? Yes, the treatment order is randomly determined. So we are willing to proceed with the t-test.

Next we consider whether conditions are met for use of a t-model. The sample size is only 23. But the distribution of differences in the sample is not heavily skewed and does not contain outliers. So we assume that these differences are normally distributed in general when comparing these two treatments. We use the t-test with caution and plan to include a disclaimer with the results.

Other important observations are as follows:

Why do we use only one of the three graphs of the data to check conditions? In this situation the only distribution that is relevant to checking the guidelines for t-test is the distribution of differences. Why? Because this is a matched-pairs design. The hypotheses are about “the effect” of texting. So we take two measurements (with and without texting) and subtract to look at the effect.

Does the experiment control confounding variables? The distributions of word counts for the Treatment and Control settings are skewed, but the distribution of differences is not. This is an illustration of how a matched-pairs design in which treatment order is randomized can control the confounding effects of individual characteristics, such as natural ability with word puzzles. For example, there was one student who was very good at recalling words. She recalled 12 words when texting and 18 words when not texting. Her scores will skew the mean of the Treatment scores and the mean of the Control scores. But her difference of −6 fits in with differences in other students’ scores.

Test statistic and P-value:

The T-score is

Set the degrees of freedom to 23 − 1 = 22. With a t-test statistic this far from 0, we know the P-value will be very small. The applet shows that the P-value is essentially 0. Here is a picture of the applet showing our results.

Conclusion: Since the P-value is essentially zero, we reject and accept .
Conclusion in context: This experiment provides strong evidence that texting weakens college students’ concentration (P-value essentially zero). Students’ ability to recall words was significantly worse when they were texting, compared to their recall when they were not texting.

Disclaimer: The sample was too small to formally meet the requirements for a t-test. Based on the data, we are assuming that the differences in word recall would be normally distributed in general when comparing these two treatments in the population of all college students. The conclusion from this test is valid only if this assumption is true.

Following are important observations about this conclusion:

Why did we include the disclaimer? It is good practice to include a disclaimer with this conclusion since the sample is small.

Can did we use cause-and-effect language? We use cause-and-effect language here: “texting weakens concentration.” This is valid because there is random assignment of treatment order in the experiment. Recall that a coin flip determined the treatment order for each student.

Can we generalize to all college students? In our conclusion we also generalize to the population of “college students.” Technically, this is acceptable only if the sample is a random sample from this population. Can we view the students in a statistics class as a random sample of college students for this experiment? This is debatable. But we think that this class is a representative sample of all college students when considering the effect of texting on concentration.

Mark this page as done, then click the "Next" or > button to continue.

Content by the Open Learning Initiative and licensed under CC BY.