Formulating Null & Alternative Hypotheses & p-values

Reading Time: 8 Mins | Level: Intermediate

1. What is Hypothesis Testing?

In business analytics, we constantly test assumptions (e.g. "Did our new website design increase conversion rates?", "Is supplier A producing fewer defective parts than supplier B?").

Hypothesis Testing is a formal statistical framework for evaluating whether an observed difference is genuine or merely random sample fluctuation.

2. Setting Up Hypotheses

1. 1
   Null Hypothesis ($H_0$): The baseline assumption of no effect, no change, or no difference. (e.g. $H_0: \mu_{new} = \mu_{old}$).
2. 2
   Alternative Hypothesis ($H_1$ or $H_a$): The claim we are trying to prove—that there is a statistically significant effect (e.g. $H_1: \mu_{new} \ne \mu_{old}$).

3. The P-Value & Significance Level ($\alpha$)

• Significance Level ($\alpha$): The probability threshold of committing a Type I error (false positive). By convention, $\alpha = 0.05$ (5%).
• P-Value: The probability of observing a result at least as extreme as the sample data, assuming the Null Hypothesis ($H_0$) is true.

The Decision Rule:

4. Type I vs Type II Errors

• Type I Error ($alpha$, False Positive): Rejecting $H_0$ when it was actually true (e.g., convicting an innocent person).
• Type II Error ($eta$, False Negative): Failing to reject $H_0$ when it was actually false (e.g., letting a guilty person walk free).