Wiki Page, Math 13 - Perez Sandbox : Module 16 - Introduction to Inference (2 of 2)

Wiki Page
2026-01-17T21:15:20-08:00

Probability plays a role in ALL statistical inference. Before we dive into this idea, we will investigate a real court case. In this case the prosecution used an argument based on probability (statistical inference), as well as an argument based only on data analysis (not statistical inference.)

EXAMPLE Using inference to detect cheating—a real case

How can a prosecutor use data to detect cheating? The details of this case appeared in Chance magazine in 1991.

Overview of the case:

The Case: During an exam at a university in Florida in 1984, the proctor suspected that one student, whom we will call Student C, was copying answers from another student, whom we will call Student A. The proctor accused Student C of cheating, and the case went to the university’s supreme court.

The Evidence: At the trial, the prosecution introduced evidence based on data. Here is the evidence: On the 16 questions missed by both Student A and Student C, 13 of the wrong answers were the same.

The Argument: The prosecutor used the data to draw an inference based on probability. He asked the question: Could 13 out of 16 matches be due to chance?

He argued that a match of 13 out of 16 was very unlikely with random guessing; therefore, there had to be another explanation besides chance, and the prosecutor said the explanation was cheating. Based on this evidence, Student C was found guilty of academic dishonesty.

The Role of Random Chance

Using Simulation to Produce a Probability Distribution

To decide if we agree with this argument, we need to determine if it would be unusual to get 13 matches on 16 questions by chance alone if students just guess.

Let’s assume that each question had four options: A, B, C, or D.

We use a computer program to randomly assign answers to each question, which mimics what happens when someone randomly guesses. Using software to imitate chance behavior is called simulation.

The table lists the 16 questions missed by both students and shows how each student answered the 16 questions. Answers with a check mark ( ) show the 13 questions for which Student C's wrong answers match Student A's wrong answers.

The computer simulation generated the last three columns. Each of these columns shows randomly chosen answers for the 16 questions. Matches to Student A indicated with a check mark ( ).

Students' wrong answers, together with simulated random answers

16 Questions missed by both students | Student A's wrong answers | Student C's wrong answers | Random answers #1 | Random answers #2 | Random answers #3
1 | A | A | B | D | A
2 | C | B | D | A | B
3 | B | B | B | A | B
4 | B | B | A | C | D
5 | B | B | C | C | C
6 | D | D | B | B | C
7 | D | D | D | B | B
8 | C | C | C | B | B
9 | B | B | D | B | A
10 | C | A | C | B | A
11 | A | A | A | C | A
12 | B | B | D | B | A
13 | B | B | B | B | B
14 | B | B | D | D | B
15 | D | D | B | C | A
16 | B | A | A | D | D

We can see that random guessing produces far fewer matches to Student A.

To make this more precise, let's determine the proportion of answers that match Student A.

Student C: 13 ÷ 16 = 0.8125. About 81% of the time, Student C’s answers matched Student A’s answers on questions that they both missed.

For the random answers generated by “guessing,” we see the proportion of matches is much lower:

Set 1 has 6 matches with Student A: 6 ÷ 16 = 0.375 as a proportion, or about 38%.

Set 2 has 3 matches with Student A: 3 ÷ 16 = 0.1875 as a proportion, or about 19%.

Set 3 has 5 matches with Student A: 5 ÷ 16 = 0.3125 as a proportion, or about 31%.

The proportion of matches in the randomly generated answers vary quite a bit, from 0.1875 to 0.3125, but none are close to the proportion of matches seen in Student C’s paper, 0.8175.

Let's look at these results in a dotplot. The blue line is the proportion of matches for Student C.

Now we repeat the simulation of random sets of answers on the 16 questions, each time determining the proportion of matches to the wrong answers on Student A’s paper.

On the left is a dotplot for the proportion of matches in 100 random sets of answers. Each dot represents a set of 16 answers generated by random guessing. On the right is a histogram which shows the long-run pattern for the proportion of matches in 1,025 random sets of answers.

Analysis of the results from the simulation:

In the histogram, we see that typical results fall between about 0.1 and 0.4. More specifically, if a student randomly guessed on these 16 questions, it would not be surprising to see from 2 to 6 matches with Student A’s wrong answers. Translated into proportions, this is 2 ÷ 16 = 0.125 (about 0.1) to 6 ÷ 16 = 0.375 (about 0.4).

What Can We Conclude From the Simulation?

Is Student C unusual?

Yes. Notice that no randomly generated set of answers comes close to the proportion of matches on Student C’s exam. A proportion of 0.8125 would be very unusual if guessing, so we conclude that Student C was not guessing.

The prosecution argued that a match of 13 out of 16 wrong answers (a proportion of 0.8125) was unusual and could not be explained by random chance. Our simulation agrees with this observation. When we created answer sets by randomly guessing, we never saw more than 9 matches out of 16, which is a proportion of 0.5625. We agree with the prosecution that there has to be another explanation besides chance.

However, could there be another explanation besides cheating?

Yes. If you don’t know the answer to a question on an exam, you rarely guess at random. It is more likely that you will make an educated guess. Some wrong answers might be more logical than others. This could also explain the large proportion of matches on wrong answers between the two students. So this evidence is not convincing evidence that Student C cheated, but we know that he did not just guess.

EXAMPLE New argument based on data analysis (no use of probability)

Student C appealed his case. A second trial was held. This time the prosecution made a new argument using the data. This argument did NOT use probability. Therefore, this argument is not an example of statistical inference.

Here is the new argument: The prosecution created a new measurement. They compared every student’s paper to Student A’s paper. For the 40-multiple choice questions on the test, they counted the number of matches to Student A’s paper and divided by 40. This new measurement was also a proportion.

Analysis: There were 88 students. Here are the results:

Each dot represents a student who took the exam. A dot with a proportion of 0.6 means that 60% of this student’s answers matched Student A’s answers.

Note: Student A is included in the data. Of course, Student A is the dot with the proportion of 40 out of 40 = 1.0 This makes sense because all of Student A’s answers matched Student A. (This does not mean that Student A did well on the exam.)

We see that many of the proportions fall between between 0.40 (which is 16 matches out of 40) and 0.60 (which is 24 matches out of 40). So it is not surprising if between 40% and 60% of a student’s answers matched Student A’s answers.

Student C had 32 matches out of 40, which is a proportion of 0.80. (This does not mean that Student C made an 80% on the exam. It means that 80% of Student C’s answers matched Student A’s answers.) Student C is once again an unusual data point.

Conclusion: This time it is harder to argue that Student C is not cheating. When we compare him to the rest of the class, his paper had an unusually high number of matches with Student A’s answers. This data together with the proctor’s testimony is fairly convincing evidence for the prosecution’s claim that Student C cheated by copying from Student A’s paper.

(Source: Phillip J. Boland and Michael Proschan, “The Use of Statistical Evidence in Allegations of Exam Cheating,” Chance 3(3):10-14, 1991.)

What’s the Main Point?

Statistical inference always involves an argument based on probability.

In this court case, the prosecution used two different types of arguments to provide evidence of cheating. The first argument is an example of statistical inference because it is based on probability. We set up a simulation to reflect an assumption that the prosecutor made. The assumption is that answer sets come from random guessing. We simulated over a thousand answer sets with randomly chosen answers to investigate the long-run behavior of simulated answer sets. We then compared Student C to the distribution of randomly generated answer sets. Student C was unusual. We concluded that Student C was not randomly guessing.

The second argument is an example of data analysis with no statistical inference. The prosecutor designed a measurement and collected data from every student in the class. He compared Student C’s measurement to the measurements of the other students. Probability did not have a role in this analysis. Probability statements require a random event and a look at long-run behavior of random events, so this is not an example of statistical inference.

Content by the Open Learning Initiative and licensed under CC BY