Week 10/Module 9 - Hypothesis Testing II — Topics & Learning Outcomes
1. Two-Sample Hypothesis Tests
Description
Introduces hypothesis testing procedures for comparing two independent groups, covering the logic, assumptions, and application of two-sample z-tests and t-tests.
Subtopics
- Logic of Two-Sample Hypothesis Testing — Two-sample hypothesis tests are used to determine whether there is a statistically significant difference between the means (or proportions) of two independent groups.
- Independence Assumption and Sampling — A fundamental requirement of two-sample tests is that the two groups must be independent of each other, meaning observations in one group do not influence or relate to observations in the other.
- Two-Sample Z-Test — The two-sample z-test compares the means of two independent groups when population standard deviations are known and/or sample sizes are large.
- Two-Sample T-Test — The two-sample t-test is used when population standard deviations are unknown and sample sizes are small, relying on estimated standard errors and the t-distribution.
- Assumptions Underlying Two-Sample Tests — Both two-sample z-tests and t-tests rely on a set of statistical assumptions that must be evaluated before the results can be considered valid.
- Interpreting Results and Making Decisions — After computing the test statistic, the result is compared to a critical value or evaluated using a p-value to determine whether to reject the null hypothesis.
Supporting Materials
- Two-Sample Hypothesis Tests Topic Detail · Web page
This reading introduces two-sample hypothesis tests, explaining how to compare means or other statistics from two independent populations to determine whether observed differences are statistically significant, and serves as a foundational reference for learners setting up and interpreting such tests.
- Two-Sample Hypothesis Tests — Videos Topics and Embedded Video · Web page
A lesson page introducing two-sample hypothesis tests for comparing means from two independent groups, covering the two-sample z-test and pooled versus unpooled t-tests with engineering examples such as comparing alloys or suppliers, which learners read to set up and compute comparative tests.
2. Paired Sample Comparisons
Description
Explores methods for analyzing data collected from matched or repeated-measures designs, emphasizing how pairing reduces variability and strengthens inferential conclusions.
Subtopics
- What Are Paired Sample Designs? — Paired sample designs involve collecting two related measurements from the same subject or from matched subjects, rather than from two independent groups.
- Why Pairing Reduces Variability — The primary statistical advantage of pairing is that it removes between-subject variability from the error term, making the test more sensitive to true differences.
- Computing the Paired Difference Score — The foundation of all paired-sample inference is the difference score D, calculated for each pair as the value in condition one minus the value in condition two.
- The Paired-Sample t-Test — The paired-sample t-test evaluates whether the mean of the difference scores is significantly different from zero, using a t-distribution with n − 1 degrees of freedom.
- Assumptions of the Paired-Sample t-Test — Like all parametric tests, the paired-sample t-test rests on several assumptions that must be reasonably satisfied for results to be valid.
- Interpreting and Reporting Results — Proper interpretation of a paired-sample t-test includes reporting the test statistic, degrees of freedom, p-value, and a measure of effect size to convey practical significance.
- When to Choose a Paired vs. Independent Design — Selecting the correct design and corresponding test depends on the research question, how data were collected, and the nature of the relationship between observations.
Supporting Materials
- Paired Sample Comparisons Topic Detail · Web page
This reading explains paired sample designs, where each observation in one group is linked to a corresponding observation in the other, covering how the pairing structure changes the appropriate statistical method and how learners should apply paired tests rather than independent-samples tests in these situations.
- Paired Sample Comparisons — Videos Topics and Embedded Video · Web page
A lesson page on paired or matched-design experiments where the same specimen, machine, or site is measured twice, explaining why such data require the paired-sample t-test on differences rather than an independent-samples analysis.
3. Selecting the Appropriate Statistical Test
Description
Guides learners through a decision-making framework for choosing the correct hypothesis test based on data type, sample size, independence, and research context.
Subtopics
- The Decision-Making Framework Overview — Selecting the correct statistical test requires a structured decision process rather than guesswork. A systematic framework helps researchers avoid errors that lead to invalid conclusions.
- Identifying Data Type and Measurement Level — The scale of measurement of your outcome variable is the first and most critical factor in test selection. Tests designed for continuous data cannot be validly applied to categorical data, and vice versa.
- Assessing Sample Size and Distributional Assumptions — Many parametric tests assume the sampling distribution of the statistic is approximately normal, an assumption that depends heavily on sample size and the underlying population distribution. Evaluating these conditions guides the choice between parametric and non-parametric methods.
- Determining Independence vs. Dependence of Samples — Whether your samples are independent or related (paired/matched) is a fundamental branching point in test selection. Using an independent-samples test on paired data wastes statistical power and may produce incorrect results.
- Considering the Number of Groups or Samples — The number of groups being compared directly determines the class of test to apply. Two-group comparisons and multi-group comparisons require fundamentally different procedures.
- Aligning Test Choice with the Research Question — Beyond data characteristics, the specific inferential goal—testing differences, associations, or relationships—shapes which test is appropriate. A test that answers the wrong question produces irrelevant results even if technically applied correctly.
- Practical Checklist for Final Test Selection — A concise checklist consolidates all decision criteria into a quick-reference tool that can be applied before any analysis begins. Running through each checkpoint reduces the risk of selecting an inappropriate test.
Supporting Materials
- Selecting the Appropriate Statistical Test Topic Detail · Web page
This reading provides a structured decision-making framework for choosing the correct statistical test based on study design, data type, sample size, and other conditions, helping learners and instructors systematically avoid common errors in test selection.
- Selecting the Appropriate Statistical Test — Videos Topics and Embedded Video · Web page
A decision-framework lesson that helps learners choose among the two-sample z-test, pooled and unpooled t-tests, and the paired t-test by working through questions about data structure, sample size, and known variance.
4. Assumptions and Conditions for Validity
Description
Examines the underlying assumptions required for each inferential technique and discusses how to verify whether those conditions are met before drawing conclusions.
Subtopics
- Why Assumptions Matter in Inferential Testing — Every inferential technique rests on a set of underlying assumptions; violating these can lead to invalid p-values, incorrect confidence intervals, and misleading conclusions.
- Independence of Observations — Most parametric and many non-parametric tests require that observations be independent of one another, meaning the value of one data point does not influence another.
- Normality Assumption and How to Verify It — Parametric tests such as the t-test assume that the population distribution (or the sampling distribution of the statistic) is approximately normal.
- Homogeneity of Variance (Equal Variances) — Independent two-sample t-tests in their classical form assume that the two populations have equal variances, a condition known as homoscedasticity.
- Sample Size and the Conditions for Each Test — Adequate sample size is a practical condition that affects whether the theoretical assumptions of a test are reasonably satisfied and whether the test has sufficient power.
- Conditions Specific to Paired Comparisons — The paired t-test requires that the differences between paired observations follow an approximately normal distribution, rather than requiring normality of the original measurements themselves.
- Using Non-Parametric Methods When Assumptions Fail — Non-parametric tests make fewer distributional assumptions and serve as valid alternatives when parametric conditions cannot be met.
Supporting Materials
- Assumptions and Conditions for Validity Topic Detail · Web page
This reading details the underlying assumptions required for inferential tests to produce valid probability statements—such as normality, independence, and equal variances—and guides learners on how to check these conditions before drawing conclusions from their analyses.
- Assumptions and Conditions for Validity — Videos Topics and Embedded Video · Web page
A lesson page reviewing the assumptions behind each hypothesis test — independence, normality or sufficient sample size, and variance conditions — and explaining how to check them and what happens to p-values when they are violated.
5. Introduction to Non-Parametric Methods
Description
Introduces non-parametric alternatives to traditional hypothesis tests, explaining when and why they are used when parametric assumptions cannot be satisfied.
Subtopics
- What Are Non-Parametric Methods? — Non-parametric methods are statistical tests that do not rely on assumptions about the underlying population distribution, such as normality.
- When to Use Non-Parametric Tests — Non-parametric tests are used when the assumptions required by parametric tests — such as normality or homogeneity of variance — cannot be reasonably satisfied.
- Advantages of Non-Parametric Methods — Non-parametric methods offer flexibility and robustness, particularly in real-world datasets that do not conform to idealized statistical assumptions.
- Limitations and Trade-offs — While non-parametric tests are versatile, they come with trade-offs, most notably reduced statistical power compared to their parametric counterparts when parametric assumptions are actually met.
- Common Non-Parametric Alternatives to Parametric Tests — For most standard parametric tests, there exists a non-parametric equivalent that can be applied when the necessary assumptions are not met.
- Selecting the Right Test: Parametric vs. Non-Parametric — Choosing between a parametric and non-parametric test requires careful consideration of the data type, sample size, and the degree to which distributional assumptions are satisfied.
Supporting Materials
- Introduction to Non-Parametric Methods Topic Detail · Web page
This reading introduces non-parametric statistical methods as alternatives to parametric tests when standard assumptions like normality cannot be met, explaining the rationale for and basic application of these techniques for learners working with real-world data that violates parametric requirements.
- Introduction to Non-Parametric Methods — Videos Topics and Embedded Video · Web page
An introductory lesson on non-parametric alternatives for data that violate normality or involve small, skewed, or contaminated samples, showing when rank-based methods are preferable to mean-based parametric tests.
6. Interpreting and Communicating Results
Description
Focuses on accurately interpreting test statistics, p-values, and confidence intervals, and on communicating findings from hypothesis tests in a statistically sound and meaningful way.
Subtopics
- Understanding the Test Statistic — A test statistic summarizes how far the observed sample data deviates from what would be expected under the null hypothesis, expressed in standardized units.
- Interpreting P-Values Correctly — The p-value represents the probability of obtaining a test statistic at least as extreme as the one observed, assuming the null hypothesis is true.
- Significance Levels and Decision Thresholds — The significance level (α) is the pre-determined threshold used to decide whether a p-value is small enough to reject the null hypothesis.
- Using Confidence Intervals to Complement Hypothesis Tests — Confidence intervals provide a range of plausible values for a population parameter and offer additional context beyond a simple reject-or-fail-to-reject decision.
- Distinguishing Statistical Significance from Practical Significance — A statistically significant result does not necessarily mean the finding is large enough to matter in practice; practical significance depends on the size and real-world relevance of the effect.
- Reporting Hypothesis Test Results Clearly — Communicating hypothesis test results requires reporting all relevant statistical information in a structured and transparent manner so that readers can evaluate the evidence independently.
- Common Misinterpretations to Avoid — Several widespread misconceptions about hypothesis testing can lead to flawed conclusions and poor scientific communication.
Supporting Materials
- Interpreting and Communicating Results Topic Detail · Web page
This reading explains how to correctly interpret and communicate the outputs of hypothesis tests—including test statistics, p-values, degrees of freedom, and confidence intervals—helping learners avoid common misinterpretations and report results accurately in engineering contexts.
- Interpreting and Communicating Results — Videos Topics and Embedded Video · Web page
A lesson on interpreting and reporting results, tracing the reasoning from test statistic to p-value to decision at a given significance level, and on communicating conclusions clearly in engineering reports and design reviews.
← Back to the module page
Student Learning Outcomes
By the end of this module, students will be able to:
MO1
Select the appropriate hypothesis test (two-sample z-test, two-sample t-test, or paired-sample t-test) for a given engineering scenario based on data type, sample size, and sample independence
Level: AnalyzeType: CognitiveCourse mapping: —
MO2
Compute the test statistic for two-sample and paired-sample hypothesis tests using the correct formula and degrees of freedom
Level: ApplyType: CognitiveCourse mapping: —
MO3
Evaluate whether the assumptions underlying two-sample and paired-sample tests are satisfied for a given dataset
Level: EvaluateType: CognitiveCourse mapping: —
MO4
Interpret p-values and confidence intervals from two-sample and paired-sample tests to distinguish between statistical significance and practical significance
Level: EvaluateType: CognitiveCourse mapping: —
MO5
Identify an appropriate non-parametric alternative when the assumptions of a parametric two-sample test cannot be satisfied
Level: ApplyType: CognitiveCourse mapping: —
Course Outcomes (reference)
No course outcomes have been defined.