When a p-value is smaller than the alpha level, what does this indicate?

• A. The result is statistically significant; reject the null
• B. The result is not significant; fail to reject
• C. The alternative hypothesis is proven
• D. The test has no Type I error

Answer: (A)

When the p-value is below the pre-set alpha level, the data are unlikely to occur if the null hypothesis were true, so we reject the null at that significance level. The p-value shows the strength of evidence against the null: a smaller p-value means stronger evidence. This indicates statistical significance, meaning the observed effect is unlikely to be due to chance alone. It doesn’t prove the alternative hypothesis beyond doubt, and even with a small p-value there’s still a nonzero chance of a Type I error in repeated experiments, since alpha governs that long-run rate.