Quant #19: Significance Level

The significance level is one of the factors that must be determined before a researcher can conduct sampling. The significance level is also known as alpha. The significance level can be defined as the probability of rejecting the null hypothesis when it is true. For example, in determining whether one variable has an effect on another, a significance level of 0.05 indicates a 5% risk of concluding that there exists an effect when, in fact, the effect does not exist. Lower significance levels imply stronger evidence to reject the null hypothesis. Thus, the significance level reflects the strength of the required evidence to reject the null hypothesis.

It should be noted that the significance level (alpha) refers to a pre-chosen value whereas the p-value indicates a value calculated after running the hypothesis test. Once the significance level is determined, it can be compared to the p-value that is found during hypothesis testing. If the p-value is equal to or below the significance level, the null hypothesis can be rejected as there is sufficient evidence against it. The conclusion is that the detected effect is statistically significant.