Hypothesis Test

A hypothesis test starts with two competing statements. The null hypothesis usually states that there is no effect or no difference, for example, that a new trading strategy performs no better than the market. The alternative hypothesis states that there is an effect, for example, that the strategy does perform better.

Data are collected and a test statistic is calculated. This is then used to find a p-value: the probability of seeing results at least as extreme if the null hypothesis were true. If the p-value is below a chosen threshold, called the significance level and often set at 5%, the null hypothesis is rejected in favour of the alternative.

For example, a bank might test whether a new marketing campaign increased average account balances. If the p-value is 0.02, it rejects the null hypothesis of no change and concludes there is evidence of an increase.

Hypothesis tests can be wrong in two ways. A false positive rejects a true null hypothesis, finding an effect that is not real. A false negative fails to reject a false null hypothesis, missing a real effect. Choosing the significance level and sample size involves balancing these risks.