Statistical Hypothesis Testing

• Big Ideas
• What you need to know
• Demonstration
• More Web stuff

### next

Big Ideas

• Hypotheses:

Theories or ideas you are trying to test with data. Usually these boil down to particular values for the mean of the distribution of the data .

• "Null" and "Alternative" Hypotheses

In the simplest case, the two possibilities for the real mean of data. Often one is zero , and thus is called the "Null". The "Alternative" is usually larger.

next

What you need to know

• The distribution of statistics such as means or proportions depends on whether the null or alternative hypothesis is true.

• When we perform a statistical test, we decide between the null and alternative hypotheses depending on whether the mean exceeds a certain "critical value"

• The "significance level" of our test is the probability that we choose the alternative hypothesis when the null hypothesis is true.

• The "power" of our test is the probability that we choose the alternative hypothesis when the alternative hypothesis is true

next

p-values: Take the test!

# True or False?

1. The p-value is the probability that the null hypothesis is true.

2. The p-value is The probability that the alternative hypothesis is true.

3. The p-value is the likelihood of the observed data, given that the null hypothesis is true.

4. The p-value is, in future experiments, the probability of obtaining results as "extreme" or more "extreme" given that the null hypothesis is true.

5. The p-value is a convenient test statistic.

next

p-values: Take the test!

# True or False?

1. The p-value is the probability that the null hypothesis is true.

2. The p-value is The probability that the alternative hypothesis is true.

3. The p-value is the likelihood of the observed data, given that the null hypothesis is true.

4. The p-value is, in future experiments, the probability of obtaining results as "extreme" or more "extreme" given that the null hypothesis is true.

5. The p-value is a convenient test statistic.

next

Properties of "p-values"

The Good
Tend to be small when the alternative hypothesis is true. Easier to digest than "t-statistics".
The Ugly
Oversized studies lead to "significant" p-values with tiny alternatives. Undersized studies lead to "non-significant" p-values with large alternatives.
next

• An interactive,Web-based program illustrating concepts in hypothesis testing
• Based on statistics for proportions
• Lets you control the sample size and definitions of the hypotheses
• Simulates data and calculates p-values
• Example: 20 people are randomly selected for a survey of HIV status. A statistical test is performed to see whether the population proportion is .1 or .3. If it is .3 or higher, a risky live vaccine will be used.
• Local machine, X10

next

Other Web Stuff

The X10 Project