A hypothesis test is a statistical method for making decisions about a population parameter based on sample data. It involves two competing hypotheses:
Null Hypothesis (H₀): A statement of no effect or no difference, representing the default assumption.
Alternative Hypothesis (H₁ or Hₐ): A statement that challenges the null hypothesis, suggesting there is an effect or difference.
Medical Research
H₀: A new drug has no effect on patient recovery time
H₁: The new drug significantly reduces recovery time
Quality Control
H₀: The manufacturing process produces parts with a mean diameter of 10 cm
H₁: The mean diameter is different from 10 cm
Consider a quality control scenario where we want to test if the mean diameter of manufactured parts differs from 10 cm.
H₀: μ = 10 cm
H₁: μ ≠ 10 cm
Type I Error (α): Rejecting the null hypothesis when it is actually true (false positive)
Type II Error (β): Failing to reject the null hypothesis when it is actually false (false negative)
|
Hypothesis State |
Fail to Reject H₀ |
Reject H₀ |
|
H₀ is True |
Correct Decision |
Type I Error (α) |
|
H₀ is False |
Type II Error (β) |
Correct Decision |
Suppose a medical test for a disease:
α = 0.05 means 5% chance of wrongly concluding the treatment works
β depends on sample size and effect size
p-value: The probability of obtaining test results at least as extreme as the observed results, assuming the null hypothesis is true
Significance Level (α): The threshold for rejecting the null hypothesis, typically 0.05 or 0.01
If p-value ≤ α, reject the null hypothesis.
Example calculation:
p-value = 0.03
Significance level (α) = 0.05
0.03 < 0.05, so reject H₀
Power analysis determines the sample size needed to detect a statistically significant effect, considering:
Significance level (α)
Desired power (1 - β)
Expected effect size
Power = 1 - β β = Probability of Type II Error
Sample size calculation:
Choose α (typically 0.05)
Specify desired power (usually 0.80 or 80%)
Estimate effect size
Use power analysis calculators or statistical software
Statistical Significance: Determined by p-values and hypothesis testing
Practical Significance: The real-world importance or meaningful impact of the results
A study shows a statistically significant 0.5 cm height difference between two groups. However, this may not be practically significant in most contexts.
Effect size
Sample size
Real-world implications
Cost-benefit analysis