Quant #17: Statistical Power and Power Analysis

One of the goals of conducting a power analysis is to determine the power of a statistical test of a study. Statistical power refers to the probability of rejecting the null hypothesis when it is false – that is, the probability of detecting an effect when it does exist. In other words, statistical power reflects the probability of accepting the alternative hypothesis if it is true. For example, if there exists a relationship between x and y, the probability of detecting that relationship (given that it exists) is the statistical power. The higher the statistical power, the lower the probability of making a Type II error (false negative) – i.e., the higher the probability of detecting an effect when the effect does exist. In fact, statistical power can be thought of as the opposite of the probability of a Type II error.

Another common reason to conduct power analysis is to determine the sample size. Power analysis is often done before the study is conducted, but can also be done after the study has started. Factors that can influence the sample size include statistical power, effect size, and significance criterion. When power analysis is conducted before data collection, it is known as a prospective (or a prior) power analysis. Conversely, power analysis is referred to as post hoc when it is performed after data collection. Although a prospective power analysis is not the only power analysis that can be employed, it is the most common (and most recommended) as it aids the determination of the sample size beforehand.