Cohen's d
The definitive engine for Magnitude Discovery. Cohen's d audits the standardized distance between two group means, revealing the 'Clinical Weight' of a difference without being hostage to the p-value.
What is it?
Cohen's d is designed to mathematically isolate and quantify the magnitude of an observed outcome or model factor, independently of sample size.
The definitive engine for Magnitude Discovery. Cohen's d audits the standardized distance between two group means, revealing the 'Clinical Weight' of a difference without being hostage to the p-value.
Goals & Indications
- Magnitude Audit: Determine the real-world significance of a group difference in standard deviation units.
- Scale Neutralization: Compare outcomes across different studies and instruments by converting them into a common metric.
- Clinical Weight discovery: Isolate the practical impact of an intervention, identifying if a 'Significant' result is actually 'Large'.
Core Idea Diagram
Claims tested
How it works
- Calculate the difference between the two group sample means.
- Determine the pooled standard deviation to establish standard units.
- Divide raw mean difference by the pooled standard error SD.
- Assess standardized index value against benchmark thresholds (0.2, 0.5, 0.8).
Assumptions
Important Note
Cohen's d is a descriptive statistic, not an inferential test. It quantifies effect magnitude. Use confidence intervals to assess precision and statistical significance of the effect size.
Worked Example
| d Value | Overlap | Grade |
|---|---|---|
| 0.20 | 92.0% | Small |
| 0.50 | 80.2% | Medium |
| 0.80 | 68.9% | Large |
Standardized Mean Difference Laboratory
Slide the group means and the pooled standard deviation. Watch how variance changes dilute the standardized difference metric.
Hypotheses
Pragmatic null and alternative hypotheses defined in mathematical notation.
H₀: δ = 0 (no effect; population standardized mean difference is zero)
Hₐ: δ ≠ 0 (non-zero effect; groups differ in standardized terms)
Cohen's d is a descriptive statistic, not an inferential test. It quantifies effect magnitude. Use confidence intervals to assess precision and statistical significance of the effect size.
Assumptions
The core mathematical criteria needed to ensure that statistical testing remains unbiased and valid.
Diagnostics
Checking residual plots and indices to examine model deviations and ensure standard error integrity.
- Descriptive statistics (M, SD, n) per group
- Visual comparison of distributions (histograms, density plots)
- Variance ratio or Levene's test for homogeneity
- Confidence interval for Cohen's d
- Q-Q plots to assess normality
- Boxplots to identify outliers
- Effect size interpretation with Cohen's benchmarks (0.2, 0.5, 0.8)
- Sensitivity analysis (d with/without outliers)
- Comparison with Hedges' g (if small sample)
- Unstandardized mean difference in original units for interpretability
Applied Minds
Review concrete study examples, data layout guidelines, and copy executable syntax scripts.
Cognitive Behavioral Therapy vs. Control for Depression (Classic RCT)
Research question: What is the magnitude of CBT effect on depression compared to waitlist control? Design: RCT (CBT n=50, Control n=50). Outcome: Beck Depression Inventory-II (BDI-II) at post-treatment (continuous, 0-63, higher = more depression). Calculate Cohen's d to quantify treatment effect size.
# Cohen's d: CBT vs. Control for Depression
library(effsize) # cohen.d()
library(ggplot2)
library(dplyr)
# Simulate data (or load: data <- read.csv("depression_rct.csv"))
set.seed(2025)
data <- data.frame(
group = rep(c("CBT", "Control"), each=50),
depression = c(
rnorm(50, mean=12.3, sd=8.2), # CBT: M=12.3, SD=8.2
rnorm(50, mean=23.7, sd=9.5) # Control: M=23.7, SD=9.5
)
)
# === STEP 1: Descriptive Statistics ===
data %>%
group_by(group) %>%
summarise(
n = n(),
M = mean(depression),
SD = sd(depression),
Min = min(depression),
Max = max(depression)
)
# === STEP 2: Visual Comparison ===
# Overlaid density plots
ggplot(data, aes(x=depression, fill=group)) +
geom_density(alpha=0.5) +
geom_vline(data = data %>% group_by(group) %>%
summarise(M=mean(depression)),
aes(xintercept=M, color=group), linetype="dashed", size=1) +
labs(title="Distribution of Depression Scores by Group",
subtitle="Dashed lines = group means",
x="BDI-II Score(0-63)", y="Density") +
scale_fill_brewer(palette="Set1") +
scale_color_brewer(palette="Set1") +
theme_classic()
# === STEP 3: Calculate Cohen's d ===
# Method 1: Using effsize package
cohen_result <- cohen.d(depression ~ group, data=data)
print(cohen_result)
# Output: Cohen's d estimate: -1.29 (large)
# 95% CI: [-1.71, -0.87]
# Method 2: Manual calculation (for understanding)
M_cbt <- mean(data$depression[data$group == "CBT"])
M_control <- mean(data$depression[data$group == "Control"])
SD_cbt <- sd(data$depression[data$group == "CBT"])
SD_control <- sd(data$depression[data$group == "Control"])
n_cbt <- sum(data$group == "CBT")
n_control <- sum(data$group == "Control")
# Pooled standard deviation
SD_pooled <- sqrt(((n_cbt-1)*SD_cbt^2 + (n_control-1)*SD_control^2) /
(n_cbt + n_control - 2))
cohen_d <- (M_cbt - M_control) / SD_pooled
cat("\n=== Manual Calculation ===")
cat("\nMean CBT:", round(M_cbt, 2))
cat("\nMean Control:", round(M_control, 2))
cat("\nMean Difference:", round(M_cbt - M_control, 2))
cat("\nPooled SD:", round(SD_pooled, 2))
cat("\nCohen's d:", round(cohen_d, 2), "\n")
# Interpretation
if (abs(cohen_d) < 0.2) {
interpretation <- "negligible"
} else if (abs(cohen_d) < 0.5) {
interpretation <- "small"
} else if (abs(cohen_d) < 0.8) {
interpretation <- "medium"
} else {
interpretation <- "large"
}
cat("Effect size:", interpretation, "(Cohen, 1988)\n")
# === STEP 4: Hedges' g (bias-corrected for small samples) ===
# Correction factor J
df <- n_cbt + n_control - 2
J <- 1 - (3 / (4*df - 1))
hedges_g <- cohen_d * J
cat("\nHedges' g(bias-corrected):", round(hedges_g, 2))
cat("\nCorrection factor J:", round(J, 4), "\n")
# === STEP 5: Alternative Effect Sizes ===
# Glass's delta (standardize by control SD only)
glass_delta <- (M_cbt - M_control) / SD_control
cat("\nGlass's Δ (control SD):", round(glass_delta, 2), "\n")
# Unstandardized difference with 95% CI
library(rstatix)
t_result <- t.test(depression ~ group, data=data)
cat("\nUnstandardized difference:", round(M_cbt - M_control, 2),
"points on BDI-II")
cat("\n95% CI:", round(t_result$conf.int, 2), "\n")
# === STEP 6: Visualization with Effect Size ===
data_summary <- data %>%
group_by(group) %>%
summarise(
M = mean(depression),
SD = sd(depression),
SE = SD/sqrt(n())
)
ggplot(data_summary, aes(x=group, y=M, fill=group)) +
geom_bar(stat="identity", width=0.6, alpha=0.8) +
geom_errorbar(aes(ymin=M-1.96*SE, ymax=M+1.96*SE), width=0.2) +
annotate("text", x=1.5, y=max(data_summary$M)-5,
label=paste0("Cohen's d = ", round(abs(cohen_d), 2), " (large)"),
size=5, fontface="bold") +
labs(title="CBT Effect on Depression",
subtitle=paste0("Mean difference = ", round(M_control - M_cbt, 1),
" points on BDI-II"),
x="Group", y="Mean BDI-II Score ± 95% CI") +
scale_fill_brewer(palette="Set1") +
theme_classic() +
theme(legend.position="none")
# === APA-Style Reporting ===
cat("\n=== APA-Style Report ===\n")
cat("Cohen's d was calculated to quantify the magnitude of CBT effect on\n")
cat("depression. The CBT group(M =", round(M_cbt, 1), ", SD =", round(SD_cbt, 1),
") showed\n")
cat("substantially lower depression than the control group(M =", round(M_control, 1),
", SD =", round(SD_control, 1), "),\n")
cat("d =", round(cohen_d, 2), "(95% CI [", round(cohen_result$conf.int[1], 2),
",", round(cohen_result$conf.int[2], 2), "]).\n")
cat("This represents a large effect(Cohen, 1988), indicating CBT produced\n")
cat("a clinically meaningful reduction in depression symptoms.\n")Cohen's d = -1.29, 95% CI [-1.71, -0.87]. Large effect size indicating CBT group had depression scores 1.29 standard deviations lower than control. Unstandardized: 11.4-point reduction on BDI-II. Consistent with Cuijpers et al. (2013) meta-analysis showing CBT for depression has large effects (d=0.71).
Alternatives
Structured fallback pathways for choosing alternative tests when normality or slopes requirements fail.
- Hedges' g Strike — Mandatory pivot when N < 20 per group to correct for upward bias.
- Glass's Delta — Use the control group SD as the anchor if treatment alters the spread.
- Robust d — Utilize trimmed means and Winsorized SDs to protect the magnitude estimate.
- Rank-Biserial r — Conversion of U-statistic into an effect size index.
- Log-Magnitude — Calculate d on log-transformed data.
Post-hoc
Group mean comparisons and correction controls (e.g. Tukey HSD, Bonferroni) to protect against Family-Wise Error Rates.
- Compare with Hedges' g (bias-corrected for small samples)
- Bootstrap confidence intervals for d
- Assess sensitivity to outliers (robust effect sizes)
- Compare pooled vs control group SD in denominator
- Convert to other metrics: r, odds ratio, NNT for interpretation
Cohen's d is an effect size measure, not a hypothesis test. Post-hoc tests are not applicable.
Effect Size
Understanding effect sizes (e.g., Cohen's d, Partial Eta-Squared) and clinical impact benchmarks.
Standardized mean difference. Cohen's benchmarks (1988): |d| = 0.2 small, 0.5 medium, 0.8 large. Field-specific norms may differ. Always interpret in context.
Bias-corrected d for small samples (n < 50). Reduces upward bias of Cohen's d. Preferred for meta-analysis and small-sample studies.
Uses control/comparison group SD only. Appropriate when treatment may affect variability, or when control SD is more stable/reliable.
Within-subjects effect size. dz = M_diff / SD_diff. Not directly comparable to between-subjects d (dz typically larger due to correlation).
0.2
0.5
0.8
Sample Size
Guidelines for minimum sample requirements and power analysis parameters.
The 'Stability Threshold': A minimum of 20 participants per group is required. Effect size point estimates are dangerously unstable in lean samples, leading to 'Magnification Bias' (overestimating the true effect).
| Effect Size | Parameters | Required n |
|---|---|---|
| Small Effect | d=0.20 (Small) | n ≈ 788 total |
| Medium Effect | d=0.50 (Medium) | n ≈ 128 total |
| Large Effect | d=0.80 (Large) | n ≈ 52 total |
The 'Precision Strike': Reporting the point estimate of d is descriptive; reporting the 95% CI is elite. If the CI is wider than 0.5 units, your magnitude discovery lacks statistical authority. Increase N to tighten the interval.
Reporting
How to compile statistical results into publication prose matching APA and journal style guides.
Cohen's d or Hedges' g for small samples was calculated to quantify the magnitude of describe comparison. The Group 1 group (M = XX.X, SD = X.X, n = XX) showed higher/lower outcome compared to the Group 2 group (M = XX.X, SD = X.X, n = XX), d = X.XX or g = X.XX, 95% CI X.XX, X.XX. This represents a small/medium/large effect (Cohen, 1988), indicating interpret practical significance in context.
- Effect size value (d or g)
- 95% confidence interval
- Descriptive statistics per group (M, SD, n)
- Effect size interpretation (small/medium/large with Cohen's benchmarks)
- Contextual interpretation (practical/clinical significance)
- Note if Hedges' g used for small sample correction
Manuscript Lab
Copy standard summary tables and forensic reporting grids to outline analysis details.
| Metric | Value | Interpretation | 95% CI |
|---|---|---|---|
| Cohen's d | 0.82 | Large Effect | [0.41, 1.22] |
| Non-overlap | 47.4% | Active scores differ | — |
| Probability of Superiority | 71.4% | Active > Control | — |
The Distance Multiplier. d = 0.82 means the group means differ by 0.82 standard deviations.
The 'Common Language' Effect. The chance that a random person from the Treatment group scores higher than a random person from Control.
Command Center
Syntax libraries and function parameters for executing calculations in stats packages.
# 1. Execute Cohen's d with Confidence Intervals
effsize::cohen.d(score ~ group, data = df)
# 2. Extract Hedges' g correction (for small samples)
effsize::cohen.d(score ~ group, data = df, hedg.correction = TRUE)If group variances are unequal, Cohen's d is biased. You must use 'Hedges' g' or 'Glass's delta' to maintain accuracy.
# Automated Robust Effect Size Selection
effectsize::effectsize(t.test(x, y))Common Mistakes
Analytical caveats and corrections to maintain modeling integrity.
References
Scholarly lineage and citation keys grounding the statistical framework.