Home > RippleStat > Sampling Distributions


Instructional Software for
Statistics & Experimental Psychology

RippleSoft Software

How inferential Statistical Decisions Operate

An additional tutorial which develops this example in more detail is available by clicking here.

The Situation

Simple Visual Reaction Time (SVRT) tells the length of time it takes to start a response (a key press, for instance) after a visual stimulus appears. The typical reaction time is about 1/5 second (200 milliseconds; 200 ms).

It is well documented that females have a slightly longer SVRT than males.

"Empirical" Results (aka "Descriptive Statistics")

Use the 9 males and 9 females in our Experimental Psychology course to check the validity of the assertion that males have shorter reaction times than females. Assume we discover then 9 females have an average SVRT of 190 ms and the males have an average of 180. The difference in average reaction times is 10 ms (female Mn - male Mn). This difference is plotted as an orange square in the card image shown below.

Descriptive Statistics is the term used to indicate I am telling you what the subjects of the research did--what the results of the study are. I am not generalizing or predicting what the results might be if the research were repeated with another set of subjects.

Inferential Statistics or "Suppose I repeat the research with different subjects?"

If I were to repeat the research with another 18 subjects (9 males and 9 females)? Would these descriptive statistical results be repeated? Would I find again (and again) that the female reaction time was longer? When I make predictions about the results of research with another group of subjects I am asking an Inferential Statistics question.

Two Interpretations of a Descriptive Statistical Result

Descriptive Statistics of the Original Study
Results that might happen in the future with different subjects
Females have a longer reaction time than the males. The descriptive statistics of the repeated study shows that females have longer reaction times then males.

Confirms the original study: Females have shorter reaction times.

The descriptive statistics of the replicated studies do not support the notion that females have longer reaction times than males.

No evidence females have shorter reaction times.

How can I decide which interpretation of the Descriptive Statistic is better?

  • One thing I could do is to repeat the research many times using different subjects each time. This is time consuming and expensive.

Another possibility is to think outside the box. Let's reverse the question: What would data "look like" if females and males had the same average reaction time?

  • Because now I am assuming "no difference" or "the difference is zero," it's possible to run a simulation and see what kind of results I get.
  • Then we can compare the results we got when measuring RT in males and females with the results from the simulation which assumed "No Difference".
  • Are the data we got similar to the data that occurs under the assumption that females and males have the same mean RT?

So, let's run a simulation where we draw repeatedly draw two samples of 9 observations from one population--which is what is meant when we assume males and females have the same reaction time.

The Process: Draw two samples of 9 (pretend the first sample is the measurement of female SVRT). Calculate the means of the two samples and find the difference in the two means. Plot the result in the image below as a black triangle. Repeat the sampling procedure many times (100 times in this instance). The orange square indicates the results of the result measuring 9 males and females in class.

What do you decide?

Figure 1. Results of the replications of the simulated experiment.

Click on the card to see a larger image.

Compare the location of the orange square, which shows the difference in average male and female reaction time for the data we collected from students with the black triangles.

Each black triangle represents the results of simulating a research study where we measure reaction time from two groups of 9 subjects, find the average RT and calculate the difference in the average RT.

For the data summarized in the black triangles, the two groups of subjects were the same. If it had not been a simulation, we could have measured reaction time on two groups of males or two groups of females. That's basically what was done except using a mathematical model instead of actual subjects.

I conclude the orange square is NOT like the results I get from two groups that have the same average reaction time. Therefore, I am also concluding males and females have different reaction times.

Inferential Statistical Tests

When a researcher analyzes the data of an experiment with a statistical test, she is wanting to know whether or not the results are reliable--would she get similar results if the experiment were repeated with different subjects.

When the researcher calculates the value of statistic for an inferential statistical test and then compares that value to tabled values, she is asking, "Are the results I actually obtained similar to what I might have gotten if there NO real difference between the groups of subjects.


If you look at the simulation screen you'll notice there are a number of settings that can be changed. When these values are changed, you might end up making a different decision about whether or not the results of the empirical research are reliable.

These settings (aka "parameters") are assumptions that are made when inferential statistical tests are performed.

Here are some examples of the impact changes in parameters can have on the decisions based on simulation. The first 3 examples are based on a 10 ms difference in reaction time and change the sample size (4, 16, or 25). The second three examples are based on a 5 ms difference and have sample sizes of 9, 16, and 36.

© 2005 by Burrton Woodruff. All Rights Reserved. Modified Thu, Dec 27, 2007