Week 9/Module 8 - Hypothesis Testing — Module Topics

Foundations of Hypothesis Testing

• What Is Hypothesis Testing? — Hypothesis testing is a formal statistical procedure used to evaluate claims or assumptions about a population based on sample data.
• The Role of Statistical Hypotheses — A statistical hypothesis is a formal statement about a population parameter that can be tested using sample data.
• The Null Hypothesis (H₀) — The null hypothesis is the default assumption that there is no effect, no difference, or no relationship in the population being studied.
• The Alternative Hypothesis (H₁ or Hₐ) — The alternative hypothesis represents the claim or effect the researcher believes may be true if the null hypothesis is rejected.
• The Logic of Evidence and Decision-Making — Hypothesis testing operates on the principle of indirect proof: we assume the null hypothesis is true and then assess how compatible the sample data are with that assumption.
• Connecting Real-World Claims to Statistical Hypotheses — One of the most important skills in hypothesis testing is translating a practical question or claim into a properly structured pair of statistical hypotheses.

Formulating Null and Alternative Hypotheses

• The Purpose of Hypothesis Formulation — Hypothesis formulation is the foundational step in hypothesis testing, providing a clear framework for evaluating statistical claims about a population.
• The Null Hypothesis (H₀) — The null hypothesis represents the default assumption — typically a statement of no effect, no difference, or no relationship between variables.
• The Alternative Hypothesis (H₁ or Hₐ) — The alternative hypothesis is the claim a researcher seeks to support, representing a deviation from the null hypothesis in a specified or unspecified direction.
• Non-Directional (Two-Tailed) Hypotheses — A non-directional hypothesis tests for any difference from the null value, regardless of direction, and is used when the researcher has no specific prediction about the direction of the effect.
• Directional (One-Tailed) Hypotheses — A directional hypothesis specifies the expected direction of the effect — either greater than or less than the null value — and corresponds to a one-tailed test.
• Choosing Between Directional and Non-Directional Forms — Selecting the correct hypothesis form requires careful consideration of the research question, theoretical background, and the consequences of testing in the wrong direction.
• Common Errors in Hypothesis Formulation — Incorrectly stated hypotheses can invalidate an entire study, making it essential to verify that both H₀ and H₁ are mutually exclusive, exhaustive, and properly structured.

Test Statistics and Sampling Distributions

• What Is a Test Statistic? — A test statistic is a numerical value calculated from sample data that is used to decide whether to reject the null hypothesis.
• Sampling Distributions and Their Role — A sampling distribution describes how a test statistic would behave across all possible random samples of the same size if the null hypothesis were true.
• The Z-Test Statistic — The Z-test statistic is used when testing a population mean and the population standard deviation is known, or when the sample size is sufficiently large.
• The t-Test Statistic — The t-test statistic is used when the population standard deviation is unknown and must be estimated from the sample, which is the more common real-world scenario.
• Choosing the Right Test Statistic — Selecting an appropriate test statistic depends on the type of data, the parameter being tested, the number of groups, and the assumptions that can be met.
• Degrees of Freedom — Degrees of freedom (df) are a parameter that determines the exact shape of several sampling distributions, including the t, chi-square, and F distributions.
• Connecting the Test Statistic to a P-Value — Once a test statistic is calculated, its position within the sampling distribution determines the p-value, which quantifies the probability of observing a result as extreme as the sample under the null hypothesis.

P-Values and Significance Levels

• What Is a P-Value? — A p-value is the probability of observing a test statistic as extreme as, or more extreme than, the one calculated from sample data, assuming the null hypothesis is true.
• Setting the Significance Level (α) — The significance level, denoted α, is a pre-defined threshold that researchers set before conducting a test to determine when results will be considered statistically significant.
• Comparing the P-Value to α — The core decision rule in hypothesis testing is to compare the calculated p-value to the chosen significance level α to determine whether to reject the null hypothesis.
• Interpreting P-Values in Context — Statistical significance does not automatically imply practical significance; p-values must always be interpreted within the real-world context of the research question.
• Common Misinterpretations of P-Values — P-values are among the most frequently misunderstood statistics; recognizing common errors in interpretation is essential for drawing sound conclusions.
• Using P-Values to Make Statistically Grounded Decisions — Hypothesis testing with p-values provides a structured framework for making data-driven decisions while acknowledging the role of chance and uncertainty.

Applying Common Hypothesis Tests

• One-Sample t-Test in Practice — The one-sample t-test is used to determine whether a sample mean differs significantly from a known or hypothesized population mean.
• Two-Sample t-Test for Comparing Groups — The two-sample t-test evaluates whether the means of two independent groups differ significantly from one another.
• Paired t-Test for Before-and-After Data — The paired t-test is applied when the same subjects are measured twice, such as before and after an intervention, to control for individual variability.
• Chi-Square Test for Categorical Data — The chi-square test assesses whether observed frequencies in categorical data differ significantly from expected frequencies, or whether two categorical variables are independent.
• ANOVA for Comparing Multiple Group Means — Analysis of Variance (ANOVA) extends hypothesis testing to situations where three or more group means must be compared simultaneously.
• Selecting the Right Test for Real-World Data — Choosing the appropriate hypothesis test depends on the data type, number of groups, sample size, and whether observations are independent or paired.
• Interpreting and Communicating Test Results — Correctly interpreting test outcomes and communicating findings clearly is as important as performing the calculations themselves.

Type I and Type II Errors

• Defining Type I Error (False Positive) — A Type I error occurs when the null hypothesis is true but is incorrectly rejected, producing a false positive conclusion.
• Defining Type II Error (False Negative) — A Type II error occurs when the null hypothesis is false but fails to be rejected, resulting in a missed detection or false negative.
• The Trade-off Between Type I and Type II Errors — There is an inherent inverse relationship between Type I and Type II errors; reducing one typically increases the other.
• Statistical Power and Its Role in Minimizing Type II Errors — Statistical power is the probability of correctly rejecting a false null hypothesis, and it directly reflects the ability to avoid Type II errors.
• Consequences of Each Error Type in Practice — The real-world consequences of Type I and Type II errors vary widely depending on the field and the decision being made.
• Communicating Error Risk in Statistical Findings — Clearly reporting the risk of both error types is essential for transparent and trustworthy communication of hypothesis test results.

Interpreting and Communicating Results

• Drawing Statistically Sound Conclusions — After conducting a hypothesis test, the conclusion must be grounded in the statistical evidence rather than assumptions or desired outcomes.
• Interpreting P-Values in Context — The p-value represents the probability of obtaining results at least as extreme as the observed data, assuming the null hypothesis is true.
• Communicating Findings Clearly and Accurately — Presenting hypothesis testing results requires precise language that accurately reflects the statistical process and its limitations.
• Distinguishing Statistical Significance from Practical Significance — A result can be statistically significant without being meaningful in practice, and this distinction is critical when communicating findings.
• Acknowledging and Communicating Potential Errors — Every hypothesis test carries the risk of Type I and Type II errors, and honest communication of results acknowledges these limitations.
• Presenting Results with Appropriate Confidence — Confidence intervals complement hypothesis test results by providing a range of plausible values for the parameter of interest, adding depth to the conclusion.