KONSİLWhat is statistical power?
Statistical power is the probability that a study yields a statistically significant result when a true effect of a given size exists (1 − β). A power analysis, done before data collection, links effect size, significance level (α), power and variability to the sample size.
Power is the complement of the type II error probability β (power = 1 − β). It depends on the sample size, the target difference (effect size), the significance level α and, for continuous outcomes, the standard deviation. With the others fixed, either the required sample size or the achievable power can be calculated. The DELTA2 guidance gives typical values as two-sided 5% significance with 80% or 90% power; according to Sullivan and Feinn (2012), levels other than the β of 0.20 chosen by Cohen need to be explained.
Banerjee et al. (2009) describe the choice of a suitable effect size as the hardest part of planning sample size. According to DELTA2, at least one key stakeholder group, such as patients, health professionals, regulators or funders, should regard the target difference as important, yet in many cases it appears to be set by convenience or budget.
According to Sullivan and Feinn, power must be calculated before the study starts. ‘Observed power’, calculated after a non-significant result, is criticised: Hoenig and Heisey (2001) show that it is a one-to-one function of the p-value and so adds nothing to interpretation, and Greenland et al. (2016) call it a transformation of the null P value. The CONSORT 2010 explanation document sees little merit in such a calculation; power is then indicated by the confidence interval.
Example
In a hypothetical study comparing mean systolic blood pressure between two groups, the target difference is 5 mmHg and the standard deviation 10 mmHg (d = 0.5). With a two-sided α of 0.05 and 80% power, an independent-samples t test needs 63.77, rounded up to 64, participants per group; if the target difference falls to 2.5 mmHg (d = 0.25), it needs 252.13, rounded up to 253.
A point to watch
According to Hoenig and Heisey (2001), ‘observed power’ calculated after a non-significant result is a one-to-one function of the p-value and does not change how the p-value is interpreted. According to the CONSORT 2010 explanation document, power at that stage is indicated by confidence intervals.
Related terms
Guides and tools
Sources
- The Abuse of Power: The Pervasive Fallacy of Power Calculations for Data Analysis — The American Statistician (American Statistical Association), 2001
- CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials — BMJ, 2010
- DELTA2 guidance on choosing the target difference and undertaking and reporting the sample size calculation for a randomised controlled trial — BMJ, 2018
- Hypothesis testing, type I and type II errors — Industrial Psychiatry Journal, 2009
- Using Effect Size—or Why the P Value Is Not Enough — Journal of Graduate Medical Education, 2012
- Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations — European Journal of Epidemiology (Springer), 2016
- Statistics review 4: sample size calculations — Critical Care (BioMed Central), 2002