What are type I and type II errors?

According to Banerjee et al. (2009), rejecting a null hypothesis that is in fact true is a type I error (false positive), and failing to reject one that is in fact false is a type II error (false negative). The probability of a type I error is α, the maximum the investigator sets before the study; the probability of a type II error is β, and power is 1 − β. The same source says many studies set α at 0.05 and β at 0.20 (power 80%), with conventional ranges of 0.01–0.10 for α and 0.05–0.20 for β. False results caused by bias are not called type I and type II errors.

Banerjee et al. note that neither error can be avoided entirely, that increasing the sample size can reduce their likelihood, and that the research question decides which error matters more to avoid. Strictly, at a fixed significance level a larger sample lowers β (it raises power) while the type I error probability stays at α; a larger sample lets the investigator keep both α and β low.

Multiple testing raises the risk of a type I error: according to Bland and Altman (1995), when 20 independent true null hypotheses are each tested at the 0.05 level, the probability of at least one significant result is 1 − 0.95²⁰ = 0.6415 (they report 0.64). They describe the Bonferroni method as dividing α by the number of tests k (α/k).