Week 9/Module 8 - Hypothesis Testing — Topics & Learning Outcomes
1. Foundations of Hypothesis Testing
Description
Introduces the core concepts and logic underlying hypothesis testing, including the purpose of statistical hypotheses and how they relate to real-world claims.
Subtopics
- What Is Hypothesis Testing? — Hypothesis testing is a formal statistical procedure used to evaluate claims or assumptions about a population based on sample data.
- The Role of Statistical Hypotheses — A statistical hypothesis is a formal statement about a population parameter that can be tested using sample data.
- The Null Hypothesis (H₀) — The null hypothesis is the default assumption that there is no effect, no difference, or no relationship in the population being studied.
- The Alternative Hypothesis (H₁ or Hₐ) — The alternative hypothesis represents the claim or effect the researcher believes may be true if the null hypothesis is rejected.
- The Logic of Evidence and Decision-Making — Hypothesis testing operates on the principle of indirect proof: we assume the null hypothesis is true and then assess how compatible the sample data are with that assumption.
- Connecting Real-World Claims to Statistical Hypotheses — One of the most important skills in hypothesis testing is translating a practical question or claim into a properly structured pair of statistical hypotheses.
Supporting Materials
- Foundations of Hypothesis Testing Topic Detail · Web page
A reading that introduces hypothesis testing as a framework for using sample data to make decisions about populations, covering core concepts such as the logic of statistical inference, burden of proof, and the structured steps of a hypothesis test — intended as foundational background for learners beginning the topic.
- Foundations of Hypothesis Testing — Videos Topics and Embedded Video · Web page
A conceptual reading that introduces hypothesis testing as a formal decision procedure, reframing engineering questions as "how surprising would this data be if the status quo were true?" and setting up the vocabulary used throughout the week.
2. Formulating Null and Alternative Hypotheses
Description
Covers how to correctly define and distinguish between null and alternative hypotheses, including directional and non-directional hypothesis forms.
Subtopics
- The Purpose of Hypothesis Formulation — Hypothesis formulation is the foundational step in hypothesis testing, providing a clear framework for evaluating statistical claims about a population.
- The Null Hypothesis (H₀) — The null hypothesis represents the default assumption — typically a statement of no effect, no difference, or no relationship between variables.
- The Alternative Hypothesis (H₁ or Hₐ) — The alternative hypothesis is the claim a researcher seeks to support, representing a deviation from the null hypothesis in a specified or unspecified direction.
- Non-Directional (Two-Tailed) Hypotheses — A non-directional hypothesis tests for any difference from the null value, regardless of direction, and is used when the researcher has no specific prediction about the direction of the effect.
- Directional (One-Tailed) Hypotheses — A directional hypothesis specifies the expected direction of the effect — either greater than or less than the null value — and corresponds to a one-tailed test.
- Choosing Between Directional and Non-Directional Forms — Selecting the correct hypothesis form requires careful consideration of the research question, theoretical background, and the consequences of testing in the wrong direction.
- Common Errors in Hypothesis Formulation — Incorrectly stated hypotheses can invalidate an entire study, making it essential to verify that both H₀ and H₁ are mutually exclusive, exhaustive, and properly structured.
Supporting Materials
- Formulating Null and Alternative Hypotheses Topic Detail · Web page
A reading that explains how to correctly formulate null and alternative hypotheses before collecting data, covering the roles of each hypothesis, directional versus non-directional tests, and common pitfalls in hypothesis specification — used by learners to practice translating research questions into formal statistical claims.
- Formulating Null and Alternative Hypotheses — Videos Topics and Embedded Video · Web page
A lesson on translating engineering claims into competing statements about a population parameter, explaining why the null hypothesis always carries the equality and how to choose one- versus two-sided alternatives before any calculation begins.
3. Test Statistics and Sampling Distributions
Description
Explains how to select and calculate appropriate test statistics for different scenarios, and how these relate to underlying sampling distributions.
Subtopics
- What Is a Test Statistic? — A test statistic is a numerical value calculated from sample data that is used to decide whether to reject the null hypothesis.
- Sampling Distributions and Their Role — A sampling distribution describes how a test statistic would behave across all possible random samples of the same size if the null hypothesis were true.
- The Z-Test Statistic — The Z-test statistic is used when testing a population mean and the population standard deviation is known, or when the sample size is sufficiently large.
- The t-Test Statistic — The t-test statistic is used when the population standard deviation is unknown and must be estimated from the sample, which is the more common real-world scenario.
- Choosing the Right Test Statistic — Selecting an appropriate test statistic depends on the type of data, the parameter being tested, the number of groups, and the assumptions that can be met.
- Degrees of Freedom — Degrees of freedom (df) are a parameter that determines the exact shape of several sampling distributions, including the t, chi-square, and F distributions.
- Connecting the Test Statistic to a P-Value — Once a test statistic is calculated, its position within the sampling distribution determines the p-value, which quantifies the probability of observing a result as extreme as the sample under the null hypothesis.
Supporting Materials
- Test Statistics and Sampling Distributions Topic Detail · Web page
A reading that explains what a test statistic is, how it compresses sample data into a single value, and how sampling distributions (such as z, t, and chi-square) are used to evaluate that value under the null hypothesis — used by learners to understand the mathematical bridge between raw data and a hypothesis test decision.
- Test Statistics and Sampling Distributions — Videos Topics and Embedded Video · Web page
A reading that defines the test statistic as a standardized measure of distance between the sample result and the null-predicted value, and shows which reference sampling distribution (standard normal, t with given degrees of freedom, etc.) applies to each situation.
4. P-Values and Significance Levels
Description
Explores how to interpret p-values in context, set significance thresholds, and use these tools to make statistically grounded decisions.
Subtopics
- What Is a P-Value? — A p-value is the probability of observing a test statistic as extreme as, or more extreme than, the one calculated from sample data, assuming the null hypothesis is true.
- Setting the Significance Level (α) — The significance level, denoted α, is a pre-defined threshold that researchers set before conducting a test to determine when results will be considered statistically significant.
- Comparing the P-Value to α — The core decision rule in hypothesis testing is to compare the calculated p-value to the chosen significance level α to determine whether to reject the null hypothesis.
- Interpreting P-Values in Context — Statistical significance does not automatically imply practical significance; p-values must always be interpreted within the real-world context of the research question.
- Common Misinterpretations of P-Values — P-values are among the most frequently misunderstood statistics; recognizing common errors in interpretation is essential for drawing sound conclusions.
- Using P-Values to Make Statistically Grounded Decisions — Hypothesis testing with p-values provides a structured framework for making data-driven decisions while acknowledging the role of chance and uncertainty.
Supporting Materials
- P-Values and Significance Levels Topic Detail · Web page
A reading that explains p-values and significance levels (α), how they interact to form a decision rule for rejecting or failing to reject the null hypothesis, and how to correctly interpret p-values — used by learners to understand and apply the core decision-making mechanism of hypothesis testing.
- P-Values and Significance Levels — Videos Topics and Embedded Video · Web page
A lesson explaining the p-value as the probability of a result at least as extreme as the observed one when the null is true, and how to compare it to a pre-chosen significance level alpha to reach a reject / fail-to-reject decision.
5. Applying Common Hypothesis Tests
Description
Guides learners through the practical application of widely used hypothesis tests to real-world data sets through examples and exercises.
Subtopics
- One-Sample t-Test in Practice — The one-sample t-test is used to determine whether a sample mean differs significantly from a known or hypothesized population mean.
- Two-Sample t-Test for Comparing Groups — The two-sample t-test evaluates whether the means of two independent groups differ significantly from one another.
- Paired t-Test for Before-and-After Data — The paired t-test is applied when the same subjects are measured twice, such as before and after an intervention, to control for individual variability.
- Chi-Square Test for Categorical Data — The chi-square test assesses whether observed frequencies in categorical data differ significantly from expected frequencies, or whether two categorical variables are independent.
- ANOVA for Comparing Multiple Group Means — Analysis of Variance (ANOVA) extends hypothesis testing to situations where three or more group means must be compared simultaneously.
- Selecting the Right Test for Real-World Data — Choosing the appropriate hypothesis test depends on the data type, number of groups, sample size, and whether observations are independent or paired.
- Interpreting and Communicating Test Results — Correctly interpreting test outcomes and communicating findings clearly is as important as performing the calculations themselves.
Supporting Materials
- Applying Common Hypothesis Tests Topic Detail · Web page
A reading that walks through the step-by-step application of the most commonly used hypothesis tests — including one-sample z and t tests, two-sample tests, and chi-square tests — providing worked examples so learners can see how to select and execute the appropriate test for a given engineering or research scenario.
- Applying Common Hypothesis Tests — Videos Topics and Embedded Video · Web page
A worked application page that puts the testing procedure into practice across common engineering data situations — one-sample, two-sample, and related comparisons — so learners can match a data scenario to the correct test and carry it through.
6. Type I and Type II Errors
Description
Examines the nature and consequences of errors in hypothesis testing, including how to identify, minimize, and communicate the risk of false conclusions.
Subtopics
- Defining Type I Error (False Positive) — A Type I error occurs when the null hypothesis is true but is incorrectly rejected, producing a false positive conclusion.
- Defining Type II Error (False Negative) — A Type II error occurs when the null hypothesis is false but fails to be rejected, resulting in a missed detection or false negative.
- The Trade-off Between Type I and Type II Errors — There is an inherent inverse relationship between Type I and Type II errors; reducing one typically increases the other.
- Statistical Power and Its Role in Minimizing Type II Errors — Statistical power is the probability of correctly rejecting a false null hypothesis, and it directly reflects the ability to avoid Type II errors.
- Consequences of Each Error Type in Practice — The real-world consequences of Type I and Type II errors vary widely depending on the field and the decision being made.
- Communicating Error Risk in Statistical Findings — Clearly reporting the risk of both error types is essential for transparent and trustworthy communication of hypothesis test results.
Supporting Materials
- Type I and Type II Errors Topic Detail · Web page
A reading that defines Type I errors (false positives) and Type II errors (false negatives), explains their relationship to α and statistical power, and discusses the consequences of each error type — used by learners to understand the trade-offs and risks inherent in any hypothesis testing decision.
- Type I and Type II Errors — Videos Topics and Embedded Video · Web page
A reading contrasting Type I (false positive, probability alpha) and Type II (false negative, probability beta) errors, used to help learners reason about the trade-off between the two and its connection to power and sample size.
7. Interpreting and Communicating Results
Description
Focuses on how to draw statistically sound conclusions and present hypothesis testing findings clearly, accurately, and with appropriate confidence.
Subtopics
- Drawing Statistically Sound Conclusions — After conducting a hypothesis test, the conclusion must be grounded in the statistical evidence rather than assumptions or desired outcomes.
- Interpreting P-Values in Context — The p-value represents the probability of obtaining results at least as extreme as the observed data, assuming the null hypothesis is true.
- Communicating Findings Clearly and Accurately — Presenting hypothesis testing results requires precise language that accurately reflects the statistical process and its limitations.
- Distinguishing Statistical Significance from Practical Significance — A result can be statistically significant without being meaningful in practice, and this distinction is critical when communicating findings.
- Acknowledging and Communicating Potential Errors — Every hypothesis test carries the risk of Type I and Type II errors, and honest communication of results acknowledges these limitations.
- Presenting Results with Appropriate Confidence — Confidence intervals complement hypothesis test results by providing a range of plausible values for the parameter of interest, adding depth to the conclusion.
Supporting Materials
- Interpreting and Communicating Results Topic Detail · Web page
A reading that addresses how to accurately interpret hypothesis test results and communicate findings without overstating conclusions, covering common misinterpretations of p-values, the distinction between statistical and practical significance, and best practices for reporting results — used by learners and instructors to ensure correct understanding and professional communication of statistical analyses.
- Interpreting and Communicating Results — Videos Topics and Embedded Video · Web page
A closing lesson on correctly interpreting and writing up results, emphasizing what a p-value is not (e.g., the probability the null is true) and how to communicate conclusions in engineering reports, design reviews, and regulatory submissions.
← Back to the module page
Student Learning Outcomes
By the end of this module, students will be able to:
MO1
Construct correctly structured null and alternative hypotheses — including directional and non-directional forms — from a given real-world engineering claim
Level: ApplyType: CognitiveCourse mapping: —
MO2
Select the appropriate hypothesis test (one-sample t-test, two-sample t-test, paired t-test, chi-square, or ANOVA) for a given data scenario based on data type, number of groups, and sample characteristics
Level: AnalyzeType: CognitiveCourse mapping: —
MO3
Calculate a test statistic and corresponding p-value for a given sample dataset using the correct sampling distribution
Level: ApplyType: CognitiveCourse mapping: —
MO4
Evaluate a hypothesis test conclusion by comparing the p-value to a pre-specified significance level and distinguishing statistical significance from practical significance
Level: EvaluateType: CognitiveCourse mapping: —
MO5
Differentiate between Type I and Type II errors in a hypothesis testing scenario and identify the consequences of each error type for a given engineering context
Level: AnalyzeType: CognitiveCourse mapping: —
Course Outcomes (reference)
No course outcomes have been defined.