Type I and Type II Errors

Summary

Every hypothesis test can make two kinds of errors. A Type I error rejects a true null hypothesis; its probability is $\alpha$ . A Type II error fails to reject a false null hypothesis; its probability is $\beta$ . Power, defined as $1-\beta$ , is the probability of correctly rejecting a false null.^[1]

Prerequisites

• Hypothesis Testing
• p-values
• Probability

Definition / Notation

Parameters / Assumptions

• The test, sample size $n$ , and significance level $\alpha$ are fixed.
• Computing $\beta$ requires a specific alternative parameter value.
• The sampling distributions under both $H_0$ and the chosen alternative are known or approximated.

Essential Result

For a fixed sample size, $\alpha$ and $\beta$ are inversely related: lowering $\alpha$ raises $\beta$ , and vice versa. Increasing $n$ reduces both error probabilities for a fixed alternative. Power increases with larger effect size, larger $\alpha$ , smaller variability, and larger sample size.^[2]

Worked Example

Test $H_0:\mu =100$ against $H_a:\mu =105$ with known $\sigma =10$ , $n=25$ , and $\alpha =0.05$ for an upper-tailed test. Under $H_0$ the rejection region is

If $\mu =105$ , then $\overline{X}\sim N(105,4)$ . Thus

The power is $1-\beta \approx 0.805$ .

Common Mistakes

• Thinking $\alpha +\beta =1$ .
• Ignoring that $\beta$ depends on the specific alternative value.
• Believing a non-significant result means the effect is exactly zero.
• Choosing sample size without considering the desired power.

Connections

• Hypothesis Testing
• p-values
• Confidence Intervals

References