Module 9: Hypothesis Tests for Single Samples
Supporting Lectures:
EGN3443 Module 9 - Hypothesis Testing for Single Samples
1. Introduction to Hypothesis Testing
Hypothesis testing is a fundamental statistical method used to make inferences about a population parameter based on sample data. It provides a systematic approach to decision-making under uncertainty.
Key Concepts
- Null Hypothesis (H₀): A statement of no effect or no difference
- Alternative Hypothesis (Hₐ): A statement challenging the null hypothesis
- Significance Level (α): The probability of making a Type I error
- P-value: The probability of obtaining results at least as extreme as the observed results, assuming the null hypothesis is true
Summary of Methods
| Statistic Measured in Null Hypothesis |
Test Method to Use |
| Mean |
T-Test or Z-Test |
| Proportion |
Z-Test |
| Variance |
Chi-Squared |
| Sample Size |
Power |
2. Z-Tests and T-Tests for Means
Z-Test
Definition: A parametric test used when the population standard deviation is known and the sample size is large (n ≥ 30) or the population is normally distributed.
Use of a z-test:
- Determine whether two population means are different when the variances are known and the sample size is large
- Compare or determine the significance of statistical measures, particularly the mean in a sample from a normally distributed population or between two independent samples
- Test whether a sample mean significantly differs from a known population mean, especially when the sample size is large and the population variance is known
- Determine if results from a test are valid or repeatable
- Test whether the mean of a population is less than, greater than, or equal to a specific value, assuming the population standard deviation is known
Calculation Formula: Z = (x̄ - μ₀) / (σ / √n)
- x̄ = Sample mean
- μ₀ = Hypothesized population mean
- σ = Population standard deviation
- n = Sample size
Example: A manufacturing company claims its machines produce parts with a mean diameter of 10 cm. A sample of 40 parts has a mean diameter of 9.8 cm with a known population standard deviation of 0.2 cm. Test at α = 0.05.
Web Reference:
T-Test
Definition: Used when the population standard deviation is unknown and must be estimated from the sample, typically for smaller sample sizes.
Types of T-Tests:
- One-sample t-test
- Paired t-test
- Independent two-sample t-test
Use of a T-Test:
- Compare the means of two groups
- Determine if there is a significant difference between the means of two groups
- Quantify the difference between the mean of a variable from up to two samples
- Assess whether observed differences are significant or could have occurred by chance
Calculation Formula: t = (x̄ - μ₀) / (s / √n)
- x̄ = Sample mean
- μ₀ = Hypothesized population mean
- s = Sample standard deviation
- n = Sample size
Example: A researcher wants to test if a new study technique improves test scores. The average score for 15 students is 78, with a standard deviation of 6. The hypothesized population mean is 75. Test at α = 0.05.
Web Reference:
3. Tests for Proportions
Definition: Hypothesis tests used to make inferences about population proportions.
Uses of the test for Proportion:
- Determining whether a predictor variable has a statistically significant relationship with an outcome variable
- Estimating the difference between two or more groups
- Testing the equality of two proportions against the alternative that they are not equal
- Making inferences about a population proportion
Calculation Formula: Z = (p̂ - p₀) / √[p₀(1-p₀)/n]
- p̂ = Sample proportion
- p₀ = Hypothesized population proportion
- n = Sample size
Example: A political poll claims 50% of voters support a candidate. A sample of 500 voters shows 260 supporting the candidate. Test at α = 0.05.
Web Reference:
4. Chi-Square Tests for Variance
Definition: A test used to compare the variance of a sample to a hypothesized population variance.
Calculation Formula: χ² = (n - 1)s² / σ₀²
- s² = Sample variance
- σ₀² = Hypothesized population variance
- n = Sample size
Example: A quality control engineer wants to verify if the variability in a manufacturing process meets specifications with a maximum variance of 4.
Correct Situations for Chi-Squared Tests for Variance
The chi-squared test for variance is a statistical method with specific applications. Here are the correct situations for using this test:
Testing a Single Sample Variance Against a Known Value
When to use:
- You have a single sample and want to test if its variance differs significantly from a known or theoretical variance
- You need to determine if a process has become more or less variable than a historical standard
- Quality control scenarios where consistency is measured through variance
Example: Testing if the precision of a manufacturing process (measured through variance) matches specifications.
Testing Hypotheses About Population Variance
When to use:
- H₀: σ² = σ₀² (null hypothesis that population variance equals a specific value)
- H₁: σ² ≠ σ₀² (two-sided alternative)
- H₁: σ² > σ₀² or H₁: σ² < σ₀² (one-sided alternatives)
Key Requirements for Valid Application
- Random sampling: The sample must be randomly drawn from the population
- Independence: Observations must be independent of each other
- Normality: The underlying population should be normally distributed
- This assumption is crucial for the chi-squared test for variance
- The test is sensitive to departures from normality
Important Distinctions and Limitations
- Not for comparing multiple variances directly: The basic chi-squared test is for a single sample against a reference value
- For comparing multiple sample variances: Use specialized tests like:
- Bartlett's test (assumes normality)
- Levene's test or Brown-Forsythe test (more robust against non-normality)
- F-test (only for comparing exactly two variances)
- Not for categorical data: Despite the name similarity, this is different from the chi-squared test of independence used for categorical data
Applications in Different Fields
- Engineering: Testing if component variability meets specifications
- Manufacturing: Monitoring process stability and consistency
- Scientific research: Verifying measurement precision or experimental consistency
- Finance: Analyzing volatility of financial instruments against benchmarks
- Healthcare: Ensuring consistency in diagnostic measurements
Common Misapplications
- Using it when data is clearly non-normal
- Applying it to small samples (generally n < 20) where test power is reduced
- Confusing it with other chi-squared tests (like goodness-of-fit or independence tests)
- Using it when the variances being compared aren't independent
Web Reference:
5. Engineering Case Studies with Single Sample Tests
Case Study 1: Manufacturing Tolerance
Scenario: Verifying if machine-produced parts meet specified dimensional tolerances using hypothesis testing.
Case Study 2: Material Strength
Scenario: Testing whether a new alloy meets minimum strength requirements through statistical inference.
Examples:
- ASQ Six Sigma Case Studies
6. Sample Size and Power Calculations
Sample Size Determination
Factors Influencing Sample Size:
- Desired significance level (α)
- Desired power (1 - β)
- Expected effect size
- Population standard deviation
Power Calculation Formula: Power = 1 - β = Probability of correctly rejecting a false null hypothesis
Example: Calculating sample size needed to detect a 0.5 standard deviation difference with 80% power at α = 0.05.
Additional Resources
- OpenIntro Statistics Textbook
Practical Tips
- Always state your hypotheses clearly
- Choose the appropriate test based on your data characteristics
- Check test assumptions before conducting the analysis
- Interpret results in the context of your research question
Scenarios for Using Sample Size and Power Calculations
Sample size and power calculations are crucial in experimental design across many fields. Here are the key scenarios where they should be used:
Research and Clinical Trials
- Clinical drug trials: Determining how many patients are needed to reliably detect treatment effects while minimizing unnecessary exposure to experimental treatments
- Medical device testing: Ensuring sufficient statistical power to demonstrate safety and efficacy to regulatory bodies like FDA or EMA
- Phase I-IV clinical trials: Each phase requires appropriate sample sizes based on different aims (safety, efficacy, or post-market surveillance)
Engineering and Product Development
- Product reliability testing: Calculating how many units to test to accurately estimate failure rates and confidence intervals
- Materials testing: Determining sample quantities needed when testing new materials or components to detect meaningful differences in properties
- Quality control processes: Establishing appropriate sampling plans for production lines to maintain quality standards
Environmental and Field Studies
- Environmental impact assessments: Determining how many sampling locations and measurements are needed to detect potential contamination or ecological changes
- Wildlife population studies: Calculating required observation counts to accurately estimate population sizes, trends, or behaviors
- Agricultural field trials: Determining plot numbers and sizes needed to detect yield differences between treatments
Market Research and Business
- A/B testing: Calculating minimum visitor numbers needed for each website variant to detect conversion rate differences
- Customer satisfaction surveys: Determining survey recipient counts needed to achieve representative results with desired confidence levels
- Market acceptance testing: Determining sample sizes for consumer product testing to detect preference differences
Academic Research
- Social science studies: Calculating participant numbers needed for surveys, interviews, or experimental groups
- Psychology experiments: Determining how many subjects are required to detect hypothesized behavioral or cognitive effects
- Educational interventions: Calculating class or student numbers needed to evaluate teaching methods or curriculum changes
Critical Decision Scenarios
- When resources are limited: Optimizing sample sizes to balance statistical confidence with budget, time, or material constraints
- When testing is destructive or expensive: Minimizing sample sizes while maintaining adequate power
- When effect sizes are expected to be small: Ensuring sufficient samples to detect subtle but important differences
- When consequences of incorrect decisions are severe: Using power calculations to minimize both Type I and Type II errors in high-stakes decisions
- When planning grant proposals: Justifying sample sizes to funding agencies based on scientific principles rather than arbitrary numbers
Regulatory and Compliance Contexts
- Regulatory submissions: Meeting statistical requirements from bodies like FDA, EPA, or other regulatory agencies
- Standards compliance testing: Determining appropriate sample sizes for certification to ISO, ASTM, or other industry standards
- Legal proceedings: Establishing scientifically valid sampling plans for evidence in litigation or regulatory disputes
In all these scenarios, power and sample size calculations provide scientific legitimacy to studies, optimize resource use, and ensure that experiments have a reasonable chance of detecting true effects when they exist.