Type I & Type II Errors

What are Type I & Type II errors?

There are four possible outcomes of a hypothesis test
- Two good outcomes
  - H₀ was false and H₀got rejected
  - H₀ was true and H₀was not rejected
- Two bad outcomes (errors)
  - H₀ was true but H₀was rejected (a Type I error)
  - H₀ was false yet H₀was not rejected (a Type II error)
Type I errors occur when a hypothesis test gives sufficient evidence to reject H₀despite it being true
- This is sometimes called a “false positive”
- In a court case this would be when the defendant is found guilty despite being innocent
Type II errors are when a hypothesis test gives insufficient evidence to reject H₀despite it being false
- This is sometimes called a “false negative”
- In a court case this would be when the defendant is found innocent despite being guilty

How do I find the probability of a Type I or Type II error?

The probability of a Type I error is the probability of being in the critical region (rejecting H₀) given H₀was true
- P(Type I error) = P(in the critical region | H₀ is true)
The critical region itself was set up based on a significance level, α%, which assumed H₀ was true
- So for continuous distributions (normal, $χ^{2}$ )
  - P(Type I error) = α%
  - It's exactly equal to the significance level
  - You can often write this down with no calculation
- For discrete distributions (binomial, Poisson, Geometric)
  - It's as close as you can get to α% whilst still being critical
  - So P(Type I error) is the actual significance level (≤ α%)
The probability of a Type II error is the probability of not being in the critical region (not rejecting H₀) given H₀was false
- You need to be given the actual population parameter to find this
  - For example, H₀ assumed $p = \frac{1}{2}$ but actually $p = \frac{1}{3}$
  - This is more helpful than just saying $p \neq \frac{1}{2}$ (though, in practice, harder to know)
- P(Type II error) = P(not in the critical region | actual population parameter)
Either error is not desirable in a hypothesis test
- Ideally you want to know these probabilities before doing a test

Can I reduce the probabilities of making a Type I or Type II error?

You can reduce the probability of a Type I error by reducing the significance level

However this will increase the probability of a Type II error

You can reduce the probability of a Type II error by increasing the significance level

However this will increase the probability of a Type I error

The only way to reduce both probabilities is by increasing the size of the sample

Worked example

Lucy can hit the target 70% of the time when she throws an axe with her right hand. She claims that the proportion, p, of her throws that hit the target is higher than 70% when she uses her left hand. Lucy uses the hypotheses $H_{0} : p = 0.7$ and $H_{0} : p > 0.7$ to test her claim. Lucy makes 100 throws and will reject the null hypothesis if the axe hits the target more than 77 times.

Find the probability of a Type I error.

4-12-7-ib-ai-hl-type-i--ii-errors-a-we-solution

Given that Lucy actually hits the target 80% of the time with her left hand, find the probability of a Type II error.

4-12-7-ib-ai-hl-type-i--ii-errors-b-we-solution

Type I & Type II Errors (Edexcel A Level Further Maths: Further Statistics 1)

Revision Note