Atlas
statminds
GLM (Single-Factor Omnibus Model)The underlying model family class (e.g. GLM, linear model, categorical matrix, log-linear).Parametric ReferenceStatistical methods that assume a specific probability distribution family (typically normal).12-stage workflow

One-Way ANOVA

The fundamental engine for cross-sectional group discovery. One-Way ANOVA identifies whether at least one categorical group deviates from the global average.

Model familyGLM (Single-Factor Omnibus Model)
Hypothesisomnibus
AliasesOne-Factor ANOVA · Between-Subjects Omnibus Test · Single-Predictor GLM
G1
Omnibus Variance Audit
Determine if ANY group mean significantly departs from the grand mean.
G2
Categorical Isolation
Isolate the influence of a single nominal factor on a continuous outcome.
G3
Cross-Group Strike
Identify specific pairs of divergence using adjusted post-hoc comparisons.
Visual Overview Dashboard
1

What is it?

One-Way ANOVA compares the population means of three or more independent groups to see if at least one group differs significantly from the others.

2

When to use it

  • 1 Factor: Independent categorical grouping (3+ levels).
  • 1 Outcome: Continuous scale variable.
  • Independent Subjects: Different participants in each group.
3

Core Idea

It partitions total variance into variance between groups (how far group means are spread) vs variance within groups (noise):

Group AGroup BGroup C
Variance between groups
Variance within groups
F = ratio of variance
4

Hypotheses

H₀: μ₁ = μ₂ = μ₃ (All means equal)
Hₐ: At least one group mean differs
5

How it works

  1. Calculate Group Means & Grand Mean.
  2. Measure Variation between group averages.
  3. Measure Variation within subjects.
  4. F-Statistic = MS(Between) / MS(Within).
6

Assumptions

👥 Independence: Subjects belong to one cell.
📈 Normality: Data is normal within groups.
⚖️ Homogeneity: Equal variance across cells.
7

Important Note

ANOVA is an omnibus test. A significant F-statistic tells you a difference exists, but not where. Post-hoc testing (e.g., Tukey HSD) is required.

8

Quick Example

GroupMean
Treatment A72.1
Treatment B81.4
Control Group59.8
Interactive Sandbox

One-Way ANOVA Live Laboratory

Adjust group means and noise levels to see how variance partition drives the F-statistic and p-value.

Presets
Mean Group A (μ_A)40
Mean Group B (μ_B)45
Mean Group C (μ_C)65
Within-Group Noise (σ)10
Sample Size per Group (N)12
Group Distributions & Grand Mean LineY-Axis: Outcome Score
20406080GM = 50.0Group A (M=40.0)Group B (M=45.0)Group C (M=65.0)
Live Calculations
SourceSSdfMS
Between4200.022100.0
Within3300.033100.0
F-Statistic
21.000
p-value
< 0.001
Statistical Verdict
✅ Reject H₀ (Significant)
At least one group mean differs significantly from the others (p = < 0.001). Tukey HSD post-hoc test recommended.
The 12-Stage Precision Workflow
01Omnibus Logic
Hypotheses
We test the global assumption of equality (H₀) against the possibility of at least one group 'breaking the mold'.
02Normality Thresholds
Assumptions
Verification of group-level distributions. We ensure that the 'shape' of our data justifies the use of parametric means.
03Variance Audit
Diagnostics
Utilizing Levene's test to ensure that group variability is consistent—without which the F-statistic loses its integrity.
04Clinical Scenarios
Examples
Comparing FlowMotion vs. PT vs. Tai Chi in a three-arm trial where each intervention competes for the highest recovery signal.
05Robust Pivots
Alternatives
Knowing when to switch to Kruskal-Wallis if data skewness makes the mean an unreliable indicator of the typical experience.
06Pairwise Strikes
Post-hoc
Executing Tukey HSD to find the 'True Winner' while maintaining a strict 5% shield against False Positive inflation.
07Eta-Squared
Effect Size
Quantifying the total percentage of outcome variance 'stolen' by your categorical groupings.
08Group Balancing
Sample Size
Calculating the minimum N required to ensure that even subtle differences don't hide in the noise of random variation.
09The F-Statement
Reporting
Crafting the definitive ANOVA sentence: F(df_between, df_within) = X.XX, p = .XXX, η² = .XX.
10Sum of Squares
Software
Deciphering Type I, II, and III Sum of Squares to ensure the model respects the distribution of your real-world sample.
11The t-test Trap
Common Mistakes
Avoiding the fatal error of running multiple t-tests, which rapidly degrades the scientific credibility of your alpha level.
12Fisherian Legacy
References
Tracing the model's lineage back to R.A. Fisher and the foundational principles of experimental design.
01Hypothesis test logic

Hypotheses

Pragmatic null and alternative hypotheses defined in mathematical notation.

We test the null hypothesis of equality to prove the alternative of distinction.
Logic Core
Null · H₀

H₀: μ₁ = μ₂ = μ₃ = ... = μₖ (all group population means are equal)

Alternative · Hₐ

Hₐ: At least one group mean differs from the others (∃ i,j: μᵢ ≠ μⱼ)

Why it matters omnibus

ANOVA tests the omnibus null hypothesis simultaneously. If H₀ is rejected (p < α), post-hoc tests determine which specific pairwise comparisons are significant.

02Model diagnostics

Assumptions

The core mathematical criteria needed to ensure that statistical testing remains unbiased and valid.

Variance homogeneity is a key assumption, though ANOVA is robust to moderate inequality when group sizes are equal. Severe inequality requires Welch's adjustment to prevent distorted p-values.
Integrity Shield
6
Assumptions
4
Critical / High Severity
How to check
Quick
Inspect variable type; create histograms & summary statistics to verify scale is not discrete/categorical
Rigorous
Formal measurement scale validation; verify measurement units are truly interval/ratio (equal distances between values)
If violated
If ordinal with limited categories (e.g., Likert 1-5) → Kruskal-Wallis is a common non-parametric alternative, though ANOVA is often robust to Likert data with 5+ levels and symmetric distributions. If many ordinal levels (7+) and approximately interval-like → may proceed cautiously (robust to mild violations). If truly categorical → use chi-square test of independence
kruskal wallis
How to check
Quick
Confirm study design; count unique values of grouping variable (should be ≥3); verify each subject belongs to exactly one group. NOTE: If IV is continuous, linear regression is generally preferred to avoid artificial binning, though binning can be theoretically justified if clinical thresholds or predefined subgroups exist.
Rigorous
Check randomization scheme if experimental; verify no subject appears in multiple groups; ensure group assignment is mutually exclusive. Confirm IV represents discrete categories, not a continuous measurement that was artificially binned.
If violated
If only 2 groups → use independent-samples t-test (or Welch's t-test if unequal variances). If repeated measures on same subjects → use one-way RM-ANOVA. If groups are nested/clustered → use LMM with random effects. If IV is continuous, linear regression is preferred unless binning is theoretically justified (e.g., clinical categories).
independent t testone way repeated measures anovalinear mixed modelols regression
How to check
Quick
Design review; check if any subject IDs appear multiple times; look for clustering indicators (school, site, family IDs)
Rigorous
Durbin-Watson test for autocorrelation (if sequential data); check intraclass correlation (ICC) to detect clustering; verify sampling design ensures independence
If violated
If repeated measures (same subjects across conditions) → use RM-ANOVA or LMM. If clustered data (e.g., students nested in schools) → use LMM with random intercepts for clusters, or use cluster-robust SEs. If time series/autocorrelation → use GLS with AR(1) structure or time series methods. If contamination/carryover between subjects → redesign study or model correlation structure explicitly
one way repeated measures anovalinear mixed modelgee
How to check
Quick
Boxplots by group; identify values >1.5 IQR beyond quartiles; check for extreme z-scores (|z| > 3) within each group
Rigorous
Standardized/studentized residuals (|r| > 3 suspect); Cook's distance (D > 1 or D > 4/n); leverage values (hat > 2p/n); DFFITS; influence plots
If violated
First: verify data entry errors and correct if found. If legitimate outliers: (1) Report results with and without outliers to assess sensitivity; (2) Use robust ANOVA with trimmed means (20% trimming) or M-estimators; (3) Winsorize extreme values to 5th/95th percentiles (pre-specify threshold); (4) Bootstrap confidence intervals; (5) Transform data if outliers due to skewness (log, sqrt). Never remove outliers without justification and transparency
kruskal wallis
How to check
Quick
Q-Q plots for each group (points should fall on diagonal line); histograms of residuals per group; visual inspection for severe skewness or heavy tails
Rigorous
Shapiro-Wilk test per group (n < 50; p > .05 indicates normality); Anderson-Darling test; check skewness (<|2|) and kurtosis (<|7|) values
If violated
If mild violation with balanced sample sizes (n ≥ 30 per group): ANOVA is robust via Central Limit Theorem, proceed. If notable violation: (1) Transform DV (log for right-skewed, sqrt for count-like, inverse for severe right-skew); (2) Use Welch's ANOVA (more robust to non-normality when combined with unequal variances); (3) Use Kruskal-Wallis (rank-based, distribution-free); (4) Permutation ANOVA (resampling-based); (5) Consider GAM if nonlinearity suspected with covariates
kruskal wallisgeneralized additive models
How to check
Quick
Levene's test (p > .05 indicates equal variances); Brown-Forsythe test (median-based, more robust); visual inspection: boxplots should show similar IQR across groups; residual vs fitted values plot should show even spread
Rigorous
Bartlett's test (sensitive to normality); F-max test (ratio of largest to smallest variance < 3); visual: residual plots, spread-location plots
If violated
If Levene's p < .05 (unequal variances detected): (1) BEST: Switch to Welch's ANOVA (does not assume equal variances) + Games-Howell post-hoc; (2) Use HC3 heteroscedasticity-consistent robust SEs for planned contrasts; (3) Transform DV (log if variance increases with mean); (4) Use robust ANOVA with trimmed means (handles both outliers and unequal variances). Note: Regular Tukey HSD is invalid with unequal variances
03Residual Forensics

Diagnostics

Checking residual plots and indices to examine model deviations and ensure standard error integrity.

Trust, but verify. Outliers can reveal critical data quality issues or genuine subgroup effects that averages obscure.
System Health
Essential checks
  1. Levene's test or Brown-Forsythe test for homogeneity of variance
  2. Q-Q plots of residuals by group to assess normality
  3. Boxplots by group to identify outliers
Recommended checks
  1. Shapiro-Wilk test per group (if n < 50)
  2. Residual vs fitted values plot
  3. Cook's distance to identify influential cases
  4. Descriptive statistics (M, SD, n) per group
  5. Histogram of residuals
  6. Homogeneity of variance plot (spread vs level)
04Live Instances

Applied Minds

Review concrete study examples, data layout guidelines, and copy executable syntax scripts.

Theory is the map. Practice is the terrain. Simulation bridges the gap.
Applied Wisdom
Example 01

Yoga Type and Anxiety Reduction (Classic 3-group Design)

Research question: Does type of yoga intervention affect anxiety reduction in adults with generalized anxiety disorder? Design: 8-week RCT with 3 groups (Hatha Yoga n=35, Vinyasa Yoga n=35, Waitlist Control n=35). Outcome: Change in State-Trait Anxiety Inventory (STAI) score from baseline to post-intervention (continuous, range 20-80, higher = more anxiety reduction).

DesignBetween-subjects RCT
GroupsHatha Yoga · Vinyasa Yoga · Waitlist Control
Total n105
Outcome ScaleSTAI anxiety reduction score (continuous)
# One-way ANOVA: Yoga type effect on anxiety reduction
# Based on realistic effect sizes from yoga-anxiety meta-analyses

# Install/load packages
library(car)         # For Levene's test
library(effectsize)  # For omega squared
library(tidyverse)   # For data manipulation and ggplot2

# Simulate realistic data (or load your own: data <- read.csv("yoga_anxiety.csv"))
set.seed(2025)
data <- data.frame(
  yoga_type = rep(c("Hatha", "Vinyasa", "Control"), each=35),
  anxiety_reduction = c(
    rnorm(35, mean=12.3, sd=5.2),   # Hatha: M=12.3, SD=5.2
    rnorm(35, mean=11.8, sd=4.9),   # Vinyasa: M=11.8, SD=4.9  
    rnorm(35, mean=3.1, sd=4.8)      # Control: M=3.1, SD=4.8
  )
)

# === STEP 1: Check Assumptions ===

# 1. Normality per group (Shapiro-Wilk)
by(data$anxiety_reduction, data$yoga_type, shapiro.test)
# Result: All p > .05, normality OK

# Q-Q plots
par(mfrow=c(1,3))
for (group in c("Hatha", "Vinyasa", "Control")) {
  qqnorm(data$anxiety_reduction[data$yoga_type == group], main=group)
  qqline(data$anxiety_reduction[data$yoga_type == group])
}

# 2. Homogeneity of variance (Levene's test)
leveneTest(anxiety_reduction ~ yoga_type, data = data)
# Result: p > .05, equal variances OK

# 3. Outliers (Boxplots)
ggplot(data, aes(x=yoga_type, y=anxiety_reduction, fill=yoga_type)) +
  geom_boxplot() +
  labs(title="Anxiety Reduction by Yoga Type",
       x="Intervention", y="STAI Anxiety Reduction(points)") +
  theme_classic()
# Result: No extreme outliers detected

# === STEP 2: Descriptive Statistics ===
data %>%
  group_by(yoga_type) %>%
  summarise(n = n(),
            M = mean(anxiety_reduction),
            SD = sd(anxiety_reduction),
            SE = SD/sqrt(n))

# === STEP 3: Run One-way ANOVA ===
anova_model <- aov(anxiety_reduction ~ yoga_type, data = data)
summary(anova_model)

# Output:
# Df Sum Sq Mean Sq F value   Pr(>F)    
# yoga_type     2   1523   761.5   32.18 1.52e-11 ***
# Residuals   102   2414    23.7                     

# === STEP 4: Effect Size ===
omega_squared(anova_model)
# ω² = .38 (large effect)

eta_squared(anova_model, partial=TRUE)
# partial η² = .39

# === STEP 5: Post-hoc Tests (if p < .05) ===
TukeyHSD(anova_model, conf.level=0.95)

# Output:
#                           diff       lwr       upr     p adj
# Hatha-Control         9.2     5.8    12.6    <.001
# Vinyasa-Control       8.7     5.3    12.1    <.001  
# Vinyasa-Hatha        -0.5    -3.9     2.9     0.89

# === STEP 6: Visualize Results ===
# Bar plot with error bars
data_summary <- data %>%
  group_by(yoga_type) %>%
  summarise(M = mean(anxiety_reduction),
            SE = sd(anxiety_reduction)/sqrt(n()))

ggplot(data_summary, aes(x=yoga_type, y=M, fill=yoga_type)) +
  geom_bar(stat="identity", width=0.6) +
  geom_errorbar(aes(ymin=M-1.96*SE, ymax=M+1.96*SE), width=0.2) +
  labs(title="Anxiety Reduction by Yoga Type(8-week intervention)",
       x="Intervention Group", y="Mean Anxiety Reduction ± 95% CI") +
  scale_fill_brewer(palette="Set2") +
  theme_classic() +
  theme(legend.position="none")

# === APA-Style Reporting ===
# A one-way ANOVA was conducted to compare the effect of yoga intervention 
# type on anxiety reduction in adults with generalized anxiety disorder. 
# There was a significant effect of intervention type, F(2, 102) = 32.18, 
# p < .001, ω² = .38 (large effect). Post-hoc comparisons using Tukey HSD 
# indicated that both Hatha yoga (M = 12.3, SD = 5.2) and Vinyasa yoga 
# (M = 11.8, SD = 4.9) produced significantly greater anxiety reduction than 
# the waitlist control group (M = 3.1, SD = 4.8), both p < .001. There was 
# no significant difference between Hatha and Vinyasa yoga, p = .89.
Interpretation Blueprint

F(2, 102) = 32.18, p < .001, ω² = .38 (large effect). Both yoga interventions (Hatha and Vinyasa) significantly reduced anxiety compared to waitlist control (p < .001), with no difference between yoga types (p = .89). This supports yoga as an effective anxiety intervention regardless of specific style, consistent with meta-analytic findings (Cramer et al., 2018).

05Tactical Pivots

Alternatives

Structured fallback pathways for choosing alternative tests when normality or slopes requirements fail.

When the path is blocked, pivot. Rigor is not rigidity; it is the intelligent adaptation to reality.
Adaptive Strategy
Measurement Precision Ladder Ideal · Ratio / Interval
Ratio
Maintain ANOVA logic. Provides maximum efficiency for detecting group divergence.
Peak Signal
Interval
Ideal for Primary Outcomes. Ensure the 'Distance' between units is clinically meaningful.
Standard Precision
Ordinal
Pivot to Kruskal-Wallis H Test or Ordinal Logistic Regression to protect against non-linear mean distortion.
Rank Compression
Nominal
Abandon ANOVA. Use Chi-Square Independence or Multinomial Logistic to model group frequencies.
Information Suicide
Temporal Trajectory Audit Static Cross-Sectional
Static
Single snapshot of groups.
Stay with One-Way ANOVA. The gold standard for independent sample discovery.
Repeated
Same subjects over time.
Pivot to One-Way Repeated Measures ANOVA to account for internal subject correlation.
Paired
Exactly 2 timepoints.
Simplify to Paired T-Test for maximum parsimony and clinical clarity.
Adaptive Technical Safeguards · adaptive safeguards
Non-normal residuals
  • Kruskal-Wallis — The robust rank-based alternative for non-normal distributions.
  • Bootstrap ANOVA — Generate significance using resampled error distributions.
Unequal variances
  • Welch's ANOVA — Adjusts the F-statistic to account for unequal group spreads.
  • Brown-Forsythe Test — Median-based audit for variance equality.
independence
  • Linear Mixed Models — Incorporate random effects for clustered or nested data.
06Adjusted Comparisons

Post-hoc

Group mean comparisons and correction controls (e.g. Tukey HSD, Bonferroni) to protect against Family-Wise Error Rates.

The omnibus test opens the door; post-hoc analysis explores the room.
Forensic Detail
Adjusted Comparisons
  • Tukey HSD
  • Bonferroni
  • Scheffé
  • Holm-Bonferroni
Interpretation Guidelines

No specific guidelines provided.

07Standardized scale impact

Effect Size

Understanding effect sizes (e.g., Cohen's d, Partial Eta-Squared) and clinical impact benchmarks.

Eta-squared tells you the 'ownership' of variance. How much of the outcome does your treatment actually control?
Impact Magnitude

Biased upward. Small: .01, Medium: .06, Large: .14 (Cohen, 1988)

Less biased for population. Small: .01, Medium: .06, Large: .14. RECOMMENDED

Used when covariates present. Small: .01, Medium: .06, Large: .14

Recommended Metric: omega_squared (ω²) - less biased estimate of population effect, especially for small samples
Small
0.2
Medium
0.5
Large
0.8
0.50
omega_squared (ω²) - less biased estimate of population effect, especially for small samples
Recommended Measure
4
Available Metrics
ReportUse omega_squared (ω²) - less biased estimate of population effect, especially for small samples to represent clinical impact magnitude.
08Statistical Power

Sample Size

Guidelines for minimum sample requirements and power analysis parameters.

An underpowered study is an ethical failure. Respect the data by collecting enough of it.
Power Protocol
Floor Requirements

An n ≈ 15–20 per group is recommended for robust variance estimation and Central Limit Theorem protections, though ANOVA can run with smaller groups (e.g., n = 5–10) if normality holds.

Effect SizeParametersRequired n
Small Effectf=.10 (Small)n ≈ 969 total
Medium Effectf=.25 (Medium)n ≈ 159 total
Large Effectf=.40 (Large)n ≈ 66 total
Key considerations

Power is a function of group separation vs. internal noise. Balanced designs (equal N) are elite as they maximize the robustness of the Levene's homogeneity audit. Always account for a 15-20% attrition buffer in longitudinal extensions.

G*Power StrategyBenchmark: F-tests → ANOVA: Fixed effects, omnibus, one-way. Parameters: Effect size f, α = .05, Power = .80, Groups = 3. The Power Multiplier: Increasing group count without a proportional increase in N dramatically dilutes the signal-to-noise ratio.
09APA narrative blueprint

Reporting

How to compile statistical results into publication prose matching APA and journal style guides.

Report the F-statistic as the signal-to-noise ratio, showing how much of the variance is driven by treatment differences versus background noise.
Narrative Arc
Worked APA paragraph example
A one-way ANOVA was conducted to compare the effect of yoga intervention type (Hatha, Vinyasa, Waitlist Control) on anxiety reduction in adults with generalized anxiety disorder. Data met assumptions of normality (Shapiro-Wilk p > .05 for all groups) and homogeneity of variance (Levene's test, p = .18). There was a significant effect of intervention type on anxiety reduction, F(2, 102) = 32.18, p < .001, ω² = .38, indicating a large effect. Post-hoc comparisons using Tukey HSD indicated that both Hatha yoga (M = 12.3, SD = 5.2) and Vinyasa yoga (M = 11.8, SD = 4.9) produced significantly greater anxiety reduction than the waitlist control group (M = 3.1, SD = 4.8), both p < .001. There was no significant difference between Hatha and Vinyasa yoga, p = .89. These findings support yoga as an effective intervention for anxiety reduction, regardless of specific yoga style.
Reusable template

A one-way ANOVA was conducted to compare brief description of purpose, e.g., 'the effect of yoga intervention type on anxiety reduction'. If assumptions checked, state briefly: 'Data met assumptions of normality (Shapiro-Wilk p > .05 for all groups) and homogeneity of variance (Levene's test, p = .XX)'. There was a significant/non-significant effect of IV on DV, F(df_between, df_within) = X.XX, p = .XXX, ω² = .XX interpret: small/medium/large effect. If significant: Post-hoc comparisons using Tukey HSD/Games-Howell indicated that describe key pairwise differences with means, SDs, and p-values. Conclude with interpretation in context of research question.

Essential statistics to report
  • F-statistic
  • degrees of freedom (between and within)
  • p-value
  • effect size (ω² or η²)
  • descriptive statistics per group (M, SD, n)
  • post-hoc results if significant (pairwise p-values)
  • statement about assumption checks
10Exhibit Builder

Manuscript Lab

Copy standard summary tables and forensic reporting grids to outline analysis details.

Table 1: One-Way ANOVA Summary for Recovery Yield across Groups
SourceSSdfMSFpη²
Between Groups42.5221.2512.45< .001.14
Within Groups (Error)251.21471.71
Total293.7149
Note. Based on a balanced design with N = 150 (n = 50 per group).
F (12.45)This strong ratio confirms the treatment signal is over 12 times more powerful than sampling noise.
η² (.14)Identifies a Large Effect. 14% of the total variation in recovery is uniquely driven by the treatment type.
Header glossary

Identifies if the variance is coming from the Group Differences or the Residual Error.

Total squared deviation. Represents the raw volume of variation explained by the model vs error.

Statistical Currency. The number of independent data points used to calculate the estimate.

Purified Variance. Calculated as SS divided by df—standardizing the variance for comparison.

Signal-to-Noise Ratio. Measures how many times larger the group effect is compared to random variation.

The Accident Probability. The likelihood that these group differences occurred by random chance. Target < .05.

Total Variance Accounted For. The percentage of the total outcome that is directly 'owned' by the group factor.

11Algorithmic Logic

Command Center

Syntax libraries and function parameters for executing calculations in stats packages.

Code is the modern laboratory. Clean execution ensures reproducible discovery.
Execution Engine
# 1. Execute Omnibus ANOVA
model <- aov(yield ~ group, data = df)
summary(model)

# 2. Visualize Mean Separation
ggplot(df, aes(x=group, y=yield, fill=group)) + 
  geom_boxplot() + 
  theme_minimal()
Library stack
R
carggplot2stats
Python
pingouinscipy
Elite Forensic Strike

Automate the transition from raw data to APA narrative while safeguarding against variance violations.

# Execute 12-point Diagnostic Audit
performance::check_model(model)

# Generate Instant APA Paragraph
report::report(model)
12The Over-adjustment Trap

Common Mistakes

Analytical caveats and corrections to maintain modeling integrity.

Wisdom is learning from the failures of others. Anticipate the error before it occurs.
Defensive Logic
Why it's wrong
Inflates familywise Type I error rate. With 3 groups and α=.05 per test, familywise error ≈ 1-(1-.05)³ = .14 (14% false positive rate, not 5%). With 4 groups (6 pairwise tests), error rate reaches 26%.
The correction
Always run omnibus ANOVA first. Only proceed to post-hoc pairwise tests if ANOVA p < .05. Post-hoc tests control familywise error.
Why it's wrong
Violates homoscedasticity assumption. Can inflate Type I error (if smaller group has larger variance) or reduce power (if larger group has larger variance).
The correction
If Levene's p < .05: (1) Use Welch's ANOVA + Games-Howell post-hoc, or (2) Transform DV (log if variance ∝ mean), or (3) Use robust methods. Never use Tukey HSD with unequal variances.
Why it's wrong
p-value indicates statistical significance but not practical/clinical significance. With large n, tiny trivial effects can be 'significant'. Effect size answers: 'How large is the difference?' (crucial for meta-analysis and replication).
The correction
Always report ω² or η² with interpretation (small .01, medium .06, large .14). Include confidence intervals when possible.
Why it's wrong
Tukey HSD assumes equal variances. Using it with heteroscedasticity invalidates p-values and inflates Type I error for comparisons involving groups with large variances.
The correction
If Levene's p < .05, use Games-Howell post-hoc (robust to unequal variances) or Dunnett's T3. Always check Levene's test before choosing post-hoc method.
Why it's wrong
Selective outlier removal is a 'researcher degree of freedom' that inflates false positives. If outliers are legitimate data (not errors), removing them biases results and limits generalizability.
The correction
First verify outliers are not data entry errors. If legitimate: (1) Report results with and without outliers (sensitivity analysis), (2) Use robust methods (Kruskal-Wallis, trimmed means), or (3) Winsorize (pre-specified threshold). Never remove outliers just to achieve p < .05.
Why it's wrong
ANOVA tests group differences, not causation. In observational studies (e.g., comparing yoga practitioners vs non-practitioners), confounds and self-selection bias prevent causal inference.
The correction
For causal claims, use RCT designs with random assignment. If observational, use causal inference methods (PSM, IPTW) and report 'association' not 'effect'. Be explicit about limitations.
Why it's wrong
Kruskal-Wallis tests whether distributions differ (specifically, stochastic dominance via rank sums), not whether means differ. Only when groups have similar distribution shapes (same variance and skewness) can K-W be interpreted as a test of medians. ANOVA explicitly tests means; Kruskal-Wallis does not. The null hypothesis for K-W is 'all k distributions are identical', not 'all k means are equal'.
The correction
State Kruskal-Wallis results as 'distributions differ' not 'means differ'. If you specifically need to test medians when distributions differ, use Mood's median test or quantile regression at the 50th percentile. If you need to test means with non-normal data, use permutation ANOVA (resamples to test mean differences directly) or robust ANOVA with trimmed means. Only interpret K-W as a median test when groups have similar-shaped distributions.
13Academic Lineage

References

Scholarly lineage and citation keys grounding the statistical framework.

We stand on the shoulders of giants. Honor the source of the method.
Academic Lineage
[1]
Cramer, H., Lauche, R., Anheyer, D., Pilkington, K., de Manincor, M., Dobos, G., & Ward, L. (2018). Yoga for anxiety: A systematic review and meta-analysis of randomized controlled trials. Depression and Anxiety, 35(9), 830-843.
Meta-analysis of 8 RCTs (n=414): yoga reduces anxiety (SMD = -0.48, 95% CI [-0.81, -0.14], p=.005). Basis for Example 1.
doi: 10.1002/da.22762
[2]
Cramer, H., Lauche, R., Langhorst, J., & Dobos, G. (2017). Yoga for depression: A systematic review and meta-analysis. Psychological Medicine, 47(14), 2515-2533.
Meta-analysis of 12 RCTs: yoga reduces depression (SMD = -0.59, 95% CI [-0.99, -0.18]). Comparable to aerobic exercise. Basis for Example 3.
doi: 10.1017/S0033291717001665
[3]
Gothe, N. P., & McAuley, E. (2015). Yoga and cognition: A meta-analysis of chronic and acute effects. Journal of Physical Activity & Health, 12(3), 468-479.
Meta-analysis showing yoga improves executive function, processing speed, and attention. Effect sizes range d=0.3-0.5. Basis for Example 2.
doi: 10.1123/jpah.2014-0008
[4]
Field, A. (2024). Discovering Statistics Using R (6th ed.). Sage Publications.
Comprehensive textbook on ANOVA assumptions, diagnostics, effect sizes, and robust alternatives. Chapter 12: Comparing Several Means.
[5]
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Lawrence Erlbaum Associates.
Classic reference for effect size interpretation guidelines (small/medium/large) and power analysis for ANOVA (f = .10/.25/.40).
[6]
Welch, B. L. (1951). On the comparison of several mean values: An alternative approach. Biometrika, 38(3/4), 330-336.
Original paper on Welch's ANOVA for unequal variances. Extension of Welch's t-test to k groups.
doi: 10.2307/2332579
[7]
Games, P. A., & Howell, J. F. (1976). Pairwise multiple comparison procedures with unequal N's and/or variances: A Monte Carlo study. Journal of Educational Statistics, 1(2), 113-125.
Games-Howell post-hoc test for pairwise comparisons with heteroscedasticity. Superior to Tukey when variances unequal.
doi: 10.3102/10769986001002113
An omnibus significance is only an invitation to a deeper forensic investigation. Never claim a winner until the post-hoc evidence is conclusive.
The Interpretive Rigor Directive
statminds · One-WayMind reference · v2.2 · updated 2026-01-1715 of 15 sections