Week 15/Module 14 - ANOVA — Module Topics

Introduction to ANOVA

• What is ANOVA? — Analysis of Variance (ANOVA) is a statistical method used to compare means across three or more groups simultaneously.
• Why Not Just Use Multiple t-Tests? — When comparing more than two groups, repeatedly applying t-tests inflates the overall Type I error rate, making ANOVA the preferred approach.
• When is ANOVA the Appropriate Choice? — ANOVA is appropriate when a researcher needs to compare means from three or more independent groups on a continuous outcome variable.
• Core Assumptions of ANOVA — ANOVA relies on several key assumptions that must be met for the results to be valid and interpretable.
• One-Way vs. Multi-Factor ANOVA Designs — ANOVA can be extended beyond a single grouping variable to accommodate more complex study designs involving multiple factors.

Assumptions Underlying ANOVA

• Normality of the Dependent Variable — ANOVA assumes that the dependent variable is approximately normally distributed within each group being compared.
• Homogeneity of Variance (Homoscedasticity) — ANOVA requires that the variances of the dependent variable be approximately equal across all groups being compared, a property known as homogeneity of variance.
• Independence of Observations — ANOVA assumes that each observation is independent of all others, meaning the value of one data point does not influence or predict the value of another.
• Verifying Assumptions Before Conducting ANOVA — Before running an ANOVA, researchers should systematically check each assumption using a combination of visual methods and formal statistical tests.
• Consequences of Violating ANOVA Assumptions — When one or more ANOVA assumptions are violated, the resulting F-statistic and p-value may be inaccurate, leading to inflated Type I or Type II error rates.
• Alternatives and Remedies When Assumptions Are Violated — When ANOVA assumptions cannot be met, researchers have several alternative approaches to analyze group differences validly.

The F-Statistic and ANOVA Logic

• Why ANOVA Instead of Multiple T-Tests — ANOVA addresses the problem of comparing means across three or more groups simultaneously, avoiding the inflation of Type I error that occurs when running multiple t-tests.
• Partitioning Total Variance — The core logic of ANOVA involves decomposing the total variability in a dataset into two distinct sources: variance attributable to group differences and variance attributable to random error within groups.
• Between-Group Variance (SS_Between) — Between-group variance captures the variability among the group means themselves, representing the effect of the grouping factor or treatment.
• Within-Group Variance (SS_Within) — Within-group variance measures the variability of individual scores around their respective group means, representing random or unexplained error.
• Constructing the F-Statistic — The F-statistic is the ratio of Mean Square Between to Mean Square Within, quantifying how much the group differences exceed the random variability within groups.
• Interpreting the F-Statistic for Significance — To determine statistical significance, the calculated F-statistic is compared to a critical F-value from the F-distribution table, or a p-value is derived from it.
• The ANOVA Summary Table — Results of an ANOVA are typically presented in a structured summary table that organizes the sources of variance, degrees of freedom, mean squares, F-statistic, and p-value.

One-Way ANOVA

• Definition and Purpose of One-Way ANOVA — One-way ANOVA is a statistical procedure used to compare the means of three or more groups based on a single independent variable.
• Structure of the One-Way ANOVA Design — In a one-way ANOVA, participants are assigned to distinct groups (levels) of a single independent variable, and a continuous dependent variable is measured for each participant.
• Assumptions of One-Way ANOVA — Before conducting a one-way ANOVA, researchers must verify that the data meet several key statistical assumptions to ensure valid results.
• The F-Statistic in One-Way ANOVA — One-way ANOVA tests group differences by computing an F-statistic, which is the ratio of variance between groups to variance within groups.
• Conducting a One-Way ANOVA — Performing a one-way ANOVA involves a series of systematic steps from organizing the data to calculating and evaluating the F-statistic.
• Interpreting One-Way ANOVA Results — Interpreting the results of a one-way ANOVA requires evaluating both statistical significance and practical meaning of any observed group differences.
• Post-Hoc Testing Following a Significant One-Way ANOVA — When a one-way ANOVA yields a significant result, post-hoc tests are conducted to identify exactly which pairs of group means are significantly different from one another.

Multi-Factor ANOVA Designs

• From One-Way to Multi-Factor ANOVA — Multi-factor ANOVA extends the one-way framework by incorporating two or more independent variables, called factors, into a single analysis.
• Main Effects — A main effect is the independent influence of a single factor on the dependent variable, averaged across all levels of the other factors in the design.
• Interaction Effects — An interaction effect occurs when the influence of one factor on the dependent variable changes depending on the level of another factor.
• When to Use Multi-Factor ANOVA — Multi-factor ANOVA is appropriate when a researcher is interested in the effects of two or more independent variables on a single continuous dependent variable.
• Structure of a Two-Way ANOVA — The two-way ANOVA is the most common multi-factor design, partitioning total variance into components attributable to Factor A, Factor B, their interaction, and error.
• Interpreting Results in Multi-Factor Designs — Interpreting a multi-factor ANOVA requires evaluating each F-statistic in sequence, typically examining the interaction effect before the main effects.

Post-Hoc Testing and Interpreting Results

• Why Post-Hoc Tests Are Necessary — A significant ANOVA result tells us that at least one group mean differs from the others, but it does not identify which specific groups are different. Post-hoc tests are follow-up analyses conducted after a significant F-statistic to pinpoint exactly where those differences lie.
• Common Post-Hoc Testing Procedures — Several post-hoc tests exist, each balancing the trade-off between statistical power and control of Type I error. Choosing the right procedure depends on sample sizes, group equality, and the research context.
• Interpreting Post-Hoc Output — Post-hoc test output typically presents pairwise comparisons between all group combinations, along with p-values and sometimes confidence intervals. Correct interpretation requires evaluating each comparison against the adjusted significance threshold.
• Controlling Familywise Error Rate — When multiple comparisons are made simultaneously, the probability of making at least one Type I error increases beyond the nominal alpha level — a phenomenon known as familywise error rate inflation. Post-hoc procedures are specifically designed to keep this cumulative error rate in check.
• Drawing Meaningful Conclusions from ANOVA Results — Interpreting ANOVA results accurately requires integrating the F-statistic, significance level, effect size, and post-hoc findings into a coherent narrative. Conclusions must reflect both statistical significance and practical importance.
• Reporting ANOVA and Post-Hoc Results in APA Style — Clear, standardized reporting of ANOVA results ensures transparency and replicability. APA style guidelines provide a consistent format for presenting F-statistics, significance, effect sizes, and post-hoc findings.