Friedman Test
The engine for Robust Temporal Discovery. This model audits the rank-based change across three or more repeated measurements, providing a Powerful Within-Subjects shield when ANOVA assumptions collapse.
What is it?
Friedman Test is a nonparametric test for repeated measurements or linked blocks, evaluating rank distributions across conditions within each unit.
When to use it
- Three or More Conditions: Repeated measures designs.
- Ordinal outcome: Skewed scores or rankings violating parametric repeated ANOVA constraints.
Core Idea
Ranks conditions (1, 2, 3) *within* each subject. Under H0, the rank sums across subjects balance out. If one condition dominates, its rank sum will be extremely high:
Hypotheses
How it works
- Rank outcomes (1 to k) within each subject block separately.
- Sum ranks for each condition across all N blocks.
- Compute Q statistic based on condition rank sums.
- Evaluate Q using Chi-Square distribution with df = k - 1.
Assumptions
Effect Size
Measured using **Kendall's W** (Coefficient of Concordance) ranging from 0 (no agreement) to 1 (perfect agreement of condition ranks).
Quick Example
| Subject | C1 Rank | C2 Rank | C3 Rank |
|---|---|---|---|
| S1 | 1 | 2 | 3 |
| S2 | 1 | 3 | 2 |
Friedman Repeated Measures Live Laboratory
Increase condition shift to align individual subject rank slope pathways.
| Condition | Rank Sum |
|---|---|
| Condition 1 | 25 |
| Condition 2 | 18 |
| Condition 3 | 17 |
| Q-statistic (df = 2) | 3.8000 |
| p-value | 0.1527 |
Hypotheses
Pragmatic null and alternative hypotheses defined in mathematical notation.
H₀: All k distributions are identical (no treatment/time effect)
Hₐ: At least one distribution differs from the others
Nonparametric alternative to one-way repeated measures ANOVA. Ranks observations within each subject across k conditions, then compares rank sums. Tests whether repeated measurements on same subjects differ across conditions.
Assumptions
The core mathematical criteria needed to ensure that statistical testing remains unbiased and valid.
Diagnostics
Checking residual plots and indices to examine model deviations and ensure standard error integrity.
- Friedman chi-square statistic (χ²) and p-value
- Degrees of freedom (k - 1, where k = number of conditions)
- Kendall's W (coefficient of concordance): 0-1 scale, effect size
- Post-hoc pairwise tests (Nemenyi or Wilcoxon with Bonferroni adjustment) if significant
- Median ranks per condition (for interpretation)
- Visual comparison of distributions (boxplots or violin plots by condition)
- Individual subject trajectory plots (spaghetti plots) to check for consistent patterns
- Rank sums per condition (for understanding which conditions differ)
- Bootstrap confidence intervals for Kendall's W
- Check for ties (percentage of tied ranks - software should apply tie correction)
- Effect size for pairwise comparisons (r = z / sqrt(N))
- Profile plot with medians and IQRs by condition
- Test for monotonic trend if conditions are ordered (compare with Page's trend test)
- Sensitivity analysis: Compare with repeated measures ANOVA if borderline normality
- Within-subject consistency: Calculate individual Kendall's tau-b
Applied Minds
Review concrete study examples, data layout guidelines, and copy executable syntax scripts.
Pain Ratings Across 4 Time Points (Post-Surgery Recovery)
Research question: Does pain decrease over time after surgery? Design: Within-subjects (n=20 patients), measured at 4 time points (Day 1, Day 3, Day 7, Day 14 post-surgery). Outcome: Pain rating (0-10 Numeric Rating Scale, ordinal). Friedman used because pain data are ordinal and typically right-skewed (floor effects as pain decreases). Natural clinical research scenario.
# ==============================================================================
# Friedman Test: Pain Ratings Across 4 Time Points Post-Surgery
# Research Question: Does pain decrease over time after surgery?
# ==============================================================================
library(tidyverse) # Data manipulation and visualization
library(rstatix) # Statistical tests and effect sizes
library(PMCMRplus) # Post-hoc Nemenyi test
library(coin) # Exact tests
library(ggpubr) # Publication-ready plots
# Generate simulated data (n = 20 patients, 4 time points)
set.seed(2025)
data <- expand.grid(
subject = factor(1:20),
time = factor(c("Day_1", "Day_3", "Day_7", "Day_14"),
levels = c("Day_1", "Day_3", "Day_7", "Day_14"))
) %>%
arrange(subject, time) %>%
mutate(
# Pain decreases over time with individual variation
time_numeric = as.numeric(time),
pain_base = rep(sample(7:10, 20, replace=TRUE), each=4), # Initial pain
pain_rating = pmax(0, pmin(10, round(
pain_base - time_numeric * 1.5 + rnorm(80, 0, 0.8)
)))
) %>%
select(subject, time, pain_rating)
# ==============================================================================
# STEP 1: DESCRIPTIVE STATISTICS
# ==============================================================================
cat("\n=== DESCRIPTIVE STATISTICS BY TIME POINT ===\n")
descriptives <- data %>%
group_by(time) %>%
summarise(
n = n(),
median = median(pain_rating),
mean = mean(pain_rating),
sd = sd(pain_rating),
IQR = IQR(pain_rating),
min = min(pain_rating),
max = max(pain_rating)
)
print(descriptives)
# Check data completeness
cat("\n=== DATA COMPLETENESS CHECK ===\n")
completeness <- data %>%
group_by(subject) %>%
summarise(n_obs = n())
cat(sprintf("All subjects have %d observations: %s\n",
4, all(completeness$n_obs == 4)))
cat(sprintf("Total subjects: %d\n", n_distinct(data$subject)))
cat(sprintf("Complete cases: %d\n", sum(completeness$n_obs == 4)))
# ==============================================================================
# STEP 2: ASSUMPTION CHECKING
# ==============================================================================
cat("\n=== ASSUMPTION CHECKS ===\n")
# Assumption 1: Within-subjects design with k ≥ 3
cat("\n1. Design Check:\n")
cat(sprintf(" Design: Within-subjects(repeated measures)\n"))
cat(sprintf(" Number of conditions: %d(≥ 3 required)\n", n_distinct(data$time)))
cat(sprintf(" Conditions: %s\n", paste(levels(data$time), collapse=", ")))
# Assumption 2: Ordinal or continuous DV
cat("\n2. Measurement Scale:\n")
cat(" Pain rating: 0-10 Numeric Rating Scale(ordinal)\n")
cat(" Can be meaningfully ranked: YES\n")
# Assumption 3: Similar distribution shapes across conditions
cat("\n3. Distribution Shape Similarity:\n")
# Compare IQRs
iqrs <- data %>%
group_by(time) %>%
summarise(IQR = IQR(pain_rating))
cat(" IQRs by time point:\n")
print(iqrs)
cat(" Interpretation: Similar IQRs suggest similar spreads\n")
# Compare coefficient of variation
cv <- data %>%
group_by(time) %>%
summarise(
mean = mean(pain_rating),
sd = sd(pain_rating),
CV = sd / mean * 100
)
cat("\n Coefficient of Variation(CV) by time point:\n")
print(cv)
# Assumption 4: Check for extreme outliers
cat("\n4. Outlier Detection:\n")
outliers <- data %>%
group_by(time) %>%
identify_outliers(pain_rating)
if(nrow(outliers) > 0) {
cat(" Extreme outliers detected:\n")
print(outliers)
} else {
cat(" No extreme outliers detected.\n")
}
# Assumption 5: Check for carryover/order effects
cat("\n5. Order Effects Check:\n")
cat(" Design: Repeated measures over time(not counterbalanced)\n")
cat(" Expected trend: Monotonic decrease in pain(clinical expectation)\n")
cat(" Interpretation: If trend exists, this is the research question, not a confound\n")
# ==============================================================================
# STEP 3: VISUALIZATIONS
# ==============================================================================
cat("\n=== GENERATING VISUALIZATIONS ===\n")
# Visualization 1: Boxplots by time point
p1 <- ggplot(data, aes(x=time, y=pain_rating, fill=time)) +
geom_boxplot(alpha=0.7, outlier.shape=NA) +
geom_jitter(width=0.2, alpha=0.3, size=2) +
stat_summary(fun=median, geom="point", shape=23, size=4, fill="red") +
labs(title="Pain Ratings Over Time Post-Surgery",
subtitle="Red diamond = median(n=20 patients)",
x="Time Point", y="Pain Rating(0-10 NRS)") +
theme_minimal() +
theme(legend.position="none") +
scale_y_continuous(breaks=0:10)
print(p1)
# Visualization 2: Individual trajectories (spaghetti plot)
p2 <- ggplot(data, aes(x=time, y=pain_rating, group=subject)) +
geom_line(alpha=0.3) +
geom_point(alpha=0.3) +
stat_summary(aes(group=1), fun=median, geom="line",
color="red", size=1.5, linetype="solid") +
stat_summary(aes(group=1), fun=median, geom="point",
color="red", size=3) +
labs(title="Individual Pain Trajectories",
subtitle="Red line = median trajectory",
x="Time Point", y="Pain Rating(0-10)") +
theme_minimal() +
scale_y_continuous(breaks=0:10)
print(p2)
# Visualization 3: Violin plots (distribution shapes)
p3 <- ggplot(data, aes(x=time, y=pain_rating, fill=time)) +
geom_violin(alpha=0.6, trim=FALSE) +
geom_boxplot(width=0.1, fill="white", alpha=0.8) +
labs(title="Distribution Shapes: Pain Ratings Over Time",
subtitle="Check for similar shapes across time points",
x="Time Point", y="Pain Rating(0-10)") +
theme_minimal() +
theme(legend.position="none") +
scale_y_continuous(breaks=0:10)
print(p3)
# Visualization 4: Mean ranks by time point (from Friedman test)
# Will create after running Friedman test
# ==============================================================================
# STEP 4: FRIEDMAN TEST
# ==============================================================================
cat("\n=== FRIEDMAN TEST ===\n")
# Main Friedman test
friedman_result <- friedman.test(pain_rating ~ time | subject, data=data)
print(friedman_result)
# Extract test statistics
chi_sq <- friedman_result$statistic
df <- friedman_result$parameter
p_value <- friedman_result$p.value
cat(sprintf("\nχ²(%d) = %.2f, p = %.4f\n", df, chi_sq, p_value))
cat(sprintf("Result: %s at α = .05\n",
ifelse(p_value < 0.05, "Significant", "Not significant")))
# ==============================================================================
# STEP 5: EFFECT SIZE (KENDALL'S W)
# ==============================================================================
cat("\n=== EFFECT SIZE: KENDALL'S W ===\n")
# Calculate Kendall's W using rstatix
effect_size_result <- friedman_effsize(data, pain_rating ~ time | subject)
print(effect_size_result)
kendalls_w <- effect_size_result$effsize
magnitude <- effect_size_result$magnitude
cat(sprintf("\nKendall's W = %.3f (%s effect)\n", kendalls_w, magnitude))
cat("Interpretation: W = .1 (small), .3 (medium), .5 (large)\n")
cat("W represents degree of agreement in rankings across subjects\n")
cat("W = 0: no agreement, W = 1: perfect agreement\n")
# Alternative: Epsilon squared
n_subjects <- n_distinct(data$subject)
k_conditions <- n_distinct(data$time)
epsilon_sq <- chi_sq / (n_subjects * k_conditions - 1)
cat(sprintf("\nAlternative effect size: ε² = %.3f\n", epsilon_sq))
# ==============================================================================
# STEP 6: POST-HOC TESTS (If significant)
# ==============================================================================
if(p_value < 0.05) {
cat("\n=== POST-HOC TESTS: NEMENYI TEST ===\n")
cat("Friedman test was significant. Conducting pairwise comparisons...\n\n")
# Nemenyi post-hoc test (preferred for Friedman)
nemenyi_result <- frdAllPairsNemenyiTest(
pain_rating ~ time | subject,
data = data
)
print(nemenyi_result)
cat("\nInterpretation: p-values show pairwise comparisons between time points\n")
cat("Values < .05 indicate significant differences between pairs\n")
# Calculate median ranks for interpretation
cat("\n=== MEDIAN RANKS BY TIME POINT ===\n")
# Manually calculate ranks within each subject
data_with_ranks <- data %>%
group_by(subject) %>%
mutate(rank = rank(pain_rating, ties.method = "average")) %>%
ungroup()
mean_ranks <- data_with_ranks %>%
group_by(time) %>%
summarise(
mean_rank = mean(rank),
median_value = median(pain_rating)
) %>%
arrange(mean_rank)
print(mean_ranks)
cat("\nNote: Lower mean ranks indicate lower pain ratings\n")
# Visualization 4: Mean ranks plot
p4 <- ggplot(mean_ranks, aes(x=time, y=mean_rank, fill=time)) +
geom_col(alpha=0.7) +
geom_text(aes(label=sprintf("%.2f", mean_rank)), vjust=-0.5) +
labs(title="Mean Ranks by Time Point",
subtitle="Lower ranks = lower pain ratings",
x="Time Point", y="Mean Rank") +
theme_minimal() +
theme(legend.position="none") +
ylim(0, 4)
print(p4)
# Alternative: Pairwise Wilcoxon signed-rank tests with Bonferroni
cat("\n=== ALTERNATIVE POST-HOC: PAIRWISE WILCOXON SIGNED-RANK ===\n")
pairwise_wilcox <- data %>%
pairwise_wilcox_test(
pain_rating ~ time,
paired = TRUE,
p.adjust.method = "bonferroni"
)
print(pairwise_wilcox)
} else {
cat("\nFriedman test not significant(p > .05). Post-hoc tests not needed.\n")
}
# ==============================================================================
# STEP 7: BOOTSTRAP CONFIDENCE INTERVAL FOR KENDALL'S W
# ==============================================================================
cat("\n=== BOOTSTRAP 95% CI FOR KENDALL'S W ===\n")
set.seed(2025)
n_boot <- 2000
boot_w <- numeric(n_boot)
for(i in 1:n_boot) {
# Resample subjects with replacement
boot_subjects <- sample(unique(data$subject), replace=TRUE)
boot_data <- data %>%
filter(subject %in% boot_subjects)
# Calculate Friedman test for bootstrap sample
boot_friedman <- tryCatch({
friedman.test(pain_rating ~ time | subject, data=boot_data)
}, error = function(e) NULL)
if(!is.null(boot_friedman)) {
boot_chi <- boot_friedman$statistic
boot_n <- n_distinct(boot_data$subject)
boot_k <- n_distinct(boot_data$time)
boot_w[i] <- boot_chi / (boot_n * (boot_k - 1))
} else {
boot_w[i] <- NA
}
}
boot_w <- boot_w[!is.na(boot_w)]
boot_ci <- quantile(boot_w, c(0.025, 0.975))
cat(sprintf("Kendall's W = %.3f, 95%% CI [%.3f, %.3f]\n",
kendalls_w, boot_ci[1], boot_ci[2]))
cat(sprintf("Bootstrap iterations: %d\n", length(boot_w)))
# ==============================================================================
# STEP 8: SENSITIVITY ANALYSIS (Compare with RM-ANOVA if appropriate)
# ==============================================================================
cat("\n=== SENSITIVITY ANALYSIS: COMPARE WITH RM-ANOVA ===\n")
# Check normality of differences (for RM-ANOVA assumption)
data_wide <- data %>%
pivot_wider(names_from = time, values_from = pain_rating)
cat("Note: RM-ANOVA assumes normality of differences between conditions.\n")
cat("For ordinal pain data(0-10 NRS), Friedman is generally preferred.\n")
cat("However, comparing results can provide robustness check:\n\n")
# Run RM-ANOVA for comparison
rm_anova <- aov(pain_rating ~ time + Error(subject/time), data=data)
cat("Repeated Measures ANOVA Results:\n")
print(summary(rm_anova))
cat("\nComparison:\n")
cat(sprintf(" Friedman: χ²(%d) = %.2f, p = %.4f, W = %.3f\n",
df, chi_sq, p_value, kendalls_w))
cat(" RM-ANOVA: See F-statistic above\n")
cat("\nInterpretation: If both significant, results are robust.\n")
cat("For ordinal data(pain NRS), report Friedman as primary analysis.\n")
# ==============================================================================
# STEP 9: FINAL INTERPRETATION
# ==============================================================================
cat("\n========================================\n")
cat("FINAL INTERPRETATION\n")
cat("========================================\n\n")
if(p_value < 0.05) {
cat(sprintf(
"A Friedman test showed a statistically significant difference in pain ratings\n"))
cat(sprintf(
"across the four post-surgery time points, χ²(%d) = %.2f, p < .001,\n",
df, chi_sq))
cat(sprintf(
"Kendall's W = %.2f (%s effect).\n\n", kendalls_w, magnitude))
cat("Post-hoc pairwise comparisons using Nemenyi tests revealed:\n")
cat(" - Pain at Day 1 was significantly higher than Day 7 and Day 14\n")
cat(" - Pain at Day 3 was significantly higher than Day 14\n")
cat(" - No significant difference between Day 7 and Day 14\n\n")
cat("Median pain ratings decreased from Day 1 (Mdn = X.X) to Day 14 (Mdn = X.X),\n")
cat("demonstrating expected post-surgical pain recovery trajectory.\n\n")
cat("Conclusion: Pain ratings significantly decrease over the first 14 days\n")
cat("post-surgery, with most recovery occurring in the first week.\n")
} else {
cat("A Friedman test showed no significant difference in pain ratings\n")
cat(sprintf("across time points, χ²(%d) = %.2f, p = %.3f.\n", df, chi_sq, p_value))
}
# ==============================================================================
# STEP 10: APA-STYLE REPORTING TEMPLATE
# ==============================================================================
cat("\n========================================\n")
cat("APA-STYLE REPORTING TEMPLATE\n")
cat("========================================\n\n")
cat("A Friedman test was conducted to evaluate differences in pain ratings\n")
cat("across four post-surgery time points(Day 1, Day 3, Day 7, Day 14) in\n")
cat("20 patients. Pain was measured using a 0-10 Numeric Rating Scale(ordinal).\n")
cat("Visual inspection of boxplots and violin plots indicated similar distribution\n")
cat("shapes across time points, supporting interpretation as a test of medians.\n\n")
cat(sprintf(
"The Friedman test revealed a statistically significant effect of time on\n"))
cat(sprintf(
"pain ratings, χ²(%d) = %.2f, p < .001, Kendall's W = %.2f (%s effect).\n\n",
df, chi_sq, kendalls_w, magnitude))
cat("Post-hoc pairwise comparisons were conducted using Nemenyi tests.\n")
cat("Pain ratings at Day 1 (Mdn = X.X) were significantly higher than at\n")
cat("Day 7 (Mdn = X.X, p = .001) and Day 14 (Mdn = X.X, p < .001). Pain at\n")
cat("Day 3 (Mdn = X.X) was significantly higher than Day 14 (p = .012), but\n")
cat("no significant differences were found between Day 7 and Day 14 (p = .156)\n")
cat("or between Day 1 and Day 3 (p = .089).\n\n")
cat("These findings demonstrate a significant decrease in pain over the first\n")
cat("14 days post-surgery, with the largest reductions occurring between\n")
cat("Day 1 and Day 7, consistent with typical post-surgical recovery patterns.\n")
cat("========================================\n")χ²(3) = 45.2, p < .001, W = .75 (large effect). Post-hoc Nemenyi tests: Day 1 > Day 7 (p = .001), Day 1 > Day 14 (p < .001), Day 3 > Day 14 (p = .012). Pain significantly decreased over 14 days post-surgery, with largest reductions in first week. Ordinal pain data (0-10 NRS) justified nonparametric approach.
Alternatives
Structured fallback pathways for choosing alternative tests when normality or slopes requirements fail.
- One-Way Repeated Measures ANOVA — Return to the mean-based path to increase statistical efficiency.
- Quade Test — A more powerful rank-based alternative for small designs (k < 5) with many identical values.
- Chi-Square Homogeneity — If rank order is irrelevant, treat as multi-stage categorical profiles.
Post-hoc
Group mean comparisons and correction controls (e.g. Tukey HSD, Bonferroni) to protect against Family-Wise Error Rates.
- Compare with repeated measures ANOVA (if assumptions met)
- Examine Kendall's W for effect size (concordance)
- Bootstrap confidence intervals for mean ranks
Friedman test compares 3+ related groups. Post-hoc tests ARE applicable to identify which groups differ.
Effect Size
Understanding effect sizes (e.g., Cohen's d, Partial Eta-Squared) and clinical impact benchmarks.
negligible effect - no agreement across subjects
small effect - weak agreement
medium effect - moderate agreement
large effect - strong agreement
Sample Size
Guidelines for minimum sample requirements and power analysis parameters.
Temporal Rank Minimum: A minimum of 15 participants is required for a pure within-subjects rank discovery, assuming at least 3 longitudinal measurements.
| Effect Size | Parameters | Required n |
|---|---|---|
| Small Effect | f=.10 (Small) | n ≈ 132 total |
| Medium Effect | f=.25 (Medium) | n ≈ 31 total |
| Large Effect | f=.40 (Large) | n ≈ 17 total |
Complete Block Mandate: Friedman requires 'Complete Cases' (no missing timepoints). Account for a 20-30% attrition buffer in long-term studies to ensure the final 'Balanced Grid' meets your power target.
Reporting
How to compile statistical results into publication prose matching APA and journal style guides.
Manuscript Lab
Copy standard summary tables and forensic reporting grids to outline analysis details.
| Timepoint | Median | Mean Rank | χ² (Friedman) | df | p | Kendall's W |
|---|---|---|---|---|---|---|
| Baseline | 42.0 | 1.45 | 18.42 | 2 | < .001 | .23 |
| Month 6 | 55.0 | 2.12 | — | — | — | — |
| Month 12 | 62.0 | 2.43 | — | — | — | — |
The Intra-Subject Rank Variance. Tests if the rank of a subject consistently changes across the repeated measurements.
The Agreement Metric. .23 represents the strength of the trend. Measures how consistent subjects are in their ranking across time.
Command Center
Syntax libraries and function parameters for executing calculations in stats packages.
# 1. Execute Friedman Test
friedman.test(as.matrix(df_wide))
# 2. Extract Effect Size (Kendall's W)
rstatix::friedman_effsize(df_long, score ~ time | id)
# 3. Post-hoc Wilcoxon Nemenyi Test
PMCMRplus::frdAllPairsNemenyiTest(as.matrix(df_wide))Friedman is the 'Rank-Based Repeated Measures ANOVA'. It assumes that the relative ranking of subjects across time is what matters, not the raw differences.
# Execute Conover's Post-hoc (More powerful than Nemenyi for pairwise trends)
PMCMRplus::frdAllPairsConoverTest(as.matrix(df_wide))Common Mistakes
Analytical caveats and corrections to maintain modeling integrity.
References
Scholarly lineage and citation keys grounding the statistical framework.