Equivalence & Non-Inferiority
The engine for Clinical Parity. These models audit if a new treatment is 'Just as Good' as the gold standard, reveal if differences fall within a pre-defined safety margin (Δ) rather than just being 'Not Significant'.
What is it?
Equivalence & Non-Inferiority is a specialized statistical test used to evaluate proportions, multivariate mean vectors, or clinical equivalence margins.
The engine for Clinical Parity. These models audit if a new treatment is 'Just as Good' as the gold standard, reveal if differences fall within a pre-defined safety margin (Δ) rather than just being 'Not Significant'.
Goals & Indications
- Clinical Parity Audit: Prove that a new intervention performs within a strict 'Acceptability Window' compared to the established norm.
- Non-Inferiority Discovery: Verify that a novel treatment is 'Not Meaningfully Worse' than the standard, justifying its use for lower cost or side effects.
- Margin-Based Precision: Shift the burden of proof from 'No Difference' to 'Verified Similarity' using Two One-Sided Tests (TOST).
Core Idea Diagram
Claims tested
How it works
- Pre-specify equivalence margin Delta based on clinical significance.
- Establish hypotheses: H₀ states difference exceeds Delta.
- Calculate two one-sided t-tests (TOST) at Delta bounds.
- If both TOST reject, conclude treatment equivalence.
Assumptions
Important Note
REVERSAL OF TYPICAL HYPOTHESIS TESTING: Here we want to REJECT H₀ to claim equivalence/non-inferiority. Equivalence tests BOTH upper and lower bounds (-Δ, +Δ). Non-inferiority tests ONLY lower bound (-Δ). Margin Δ must be pre-specified based on clinical relevance, not data-driven. Use 90% CI (not 95%) to test at α = .05 level (two one-sided tests at α/2 = .025 each).
Worked Example
| Metric | Estimate | p-value |
|---|---|---|
| Test Statistic | 3.12 | 0.015 |
TOST Equivalence & Non-Inferiority Laboratory
TOST (Two One-Sided Tests) verifies whether the difference in group means lies completely within pre-specified equivalence bounds ($\pm\Delta$).
Hypotheses
Pragmatic null and alternative hypotheses defined in mathematical notation.
H₀ (Equivalence): |μ₁ - μ₂| ≥ Δ (difference exceeds margin). H₀ (Non-inferiority): μ₁ - μ₂ ≤ -Δ (test treatment inferior by margin Δ)
Hₐ (Equivalence): |μ₁ - μ₂| < Δ (difference within margin). Hₐ (Non-inferiority): μ₁ - μ₂ > -Δ (test treatment non-inferior)
REVERSAL OF TYPICAL HYPOTHESIS TESTING: Here we want to REJECT H₀ to claim equivalence/non-inferiority. Equivalence tests BOTH upper and lower bounds (-Δ, +Δ). Non-inferiority tests ONLY lower bound (-Δ). Margin Δ must be pre-specified based on clinical relevance, not data-driven. Use 90% CI (not 95%) to test at α = .05 level (two one-sided tests at α/2 = .025 each).
Assumptions
The core mathematical criteria needed to ensure that statistical testing remains unbiased and valid.
Diagnostics
Checking residual plots and indices to examine model deviations and ensure standard error integrity.
- Verify margin Δ pre-specified in protocol (not data-driven)
- Check 90% CI for mean difference (or relevant parameter)
- Compare 90% CI bounds to equivalence margins (-Δ, +Δ) or NI margin (-Δ)
- Verify ITT analysis conducted (all randomized participants)
- Per-protocol sensitivity analysis (concordance with ITT)
- Systematic review/meta-analysis of historical control vs placebo
- Forest plot showing 90% CI relative to margin(s)
- Power calculation verification (was study adequately powered?)
- Assay sensitivity assessment (control group outcomes vs historical benchmarks)
- Model assumption diagnostics (normality, proportional hazards, etc.)
Applied Minds
Review concrete study examples, data layout guidelines, and copy executable syntax scripts.
Generic Drug Bioequivalence (Equivalence Test with TOST)
Research question: Is a generic formulation bioequivalent to the brand-name drug? Design: Crossover bioequivalence study with 24 healthy volunteers. Each participant receives both Generic and Brand formulations in random order with washout period. Outcome: AUC (area under curve) for drug concentration. Equivalence margin: ±20% (80-125% ratio, FDA standard for bioequivalence). Use 90% CI for AUC ratio; if entirely within [0.80, 1.25], conclude bioequivalence.
# TOST Equivalence Test: Generic vs Brand-name Bioequivalence
# FDA bioequivalence testing using Two One-Sided Tests (TOST)
library(equivalence) # For TOST
library(TOSTER) # Alternative TOST package
library(ggplot2)
# Set seed
set.seed(2025)
# === Simulate crossover bioequivalence data ===
# True ratio ≈ 1.05 (5% higher generic, but within margin)
# Log-normal distribution for AUC
n <- 24 # Number of subjects in crossover design
# Each subject gets both treatments
# True geometric mean ratio = 1.05 (Generic 5% higher than Brand)
log_mean_brand <- 4.5 # log(AUC) for brand
log_mean_generic <- log(exp(log_mean_brand) * 1.05) # 5% higher
sd_within <- 0.25 # Within-subject SD on log scale
# Subject-specific random effects
subject_effect <- rnorm(n, mean=0, sd=0.3)
# Generate paired data (crossover)
data <- data.frame(
subject = rep(1:n, each=2),
treatment = rep(c("Brand", "Generic"), times=n),
log_AUC = c(
log_mean_brand + subject_effect + rnorm(n, 0, sd_within), # Brand
log_mean_generic + subject_effect + rnorm(n, 0, sd_within) # Generic
)
)
# Convert to wide format for paired test
library(tidyr)
data_wide <- data %>%
pivot_wider(id_cols=subject, names_from=treatment, values_from=log_AUC)
print("=== Sample Data(first 6 subjects) ===")
print(head(data_wide))
# === STEP 1: Check Assumptions ===
cat("\n=== Assumption Checks ===\n")
# 1. Normality of differences (log scale)
differences <- data_wide$Generic - data_wide$Brand
shapiro_result <- shapiro.test(differences)
cat("Shapiro-Wilk test for normality: W =", round(shapiro_result$statistic, 3),
", p =", round(shapiro_result$p.value, 3),
ifelse(shapiro_result$p.value > 0.05, "✓", "✗"), "\n")
# Q-Q plot
qqnorm(differences, main="Q-Q Plot: Log(AUC) Differences")
qqline(differences)
cat("\nCrossover design: Each subject as own control ✓\n")
cat("Adequate washout period assumed ✓\n")
# === STEP 2: Define Equivalence Margin ===
# FDA bioequivalence: 80-125% ratio on original scale
# On log scale: log(0.80) to log(1.25)
lower_margin <- log(0.80) # -0.223
upper_margin <- log(1.25) # +0.223
cat("\n=== Equivalence Margins(FDA Standard) ===\n")
cat("Original scale: 80% to 125% ratio\n")
cat("Log scale: ", round(lower_margin, 3), " to ", round(upper_margin, 3), "\n")
# === STEP 3: Calculate Mean Difference and 90% CI ===
cat("\n=== Mean Difference(Generic - Brand) ===\n")
mean_diff <- mean(differences)
sd_diff <- sd(differences)
se_diff <- sd_diff / sqrt(n)
# 90% CI (for α = .05 TOST)
t_crit <- qt(0.95, df=n-1) # One-sided 5%, two-sided 10%
ci_90_lower <- mean_diff - t_crit * se_diff
ci_90_upper <- mean_diff + t_crit * se_diff
cat("Mean log(AUC) difference:", round(mean_diff, 4), "\n")
cat("90% CI:", round(ci_90_lower, 4), "to", round(ci_90_upper, 4), "\n")
# Back-transform to ratio scale
ratio <- exp(mean_diff)
ratio_ci_lower <- exp(ci_90_lower)
ratio_ci_upper <- exp(ci_90_upper)
cat("\nGeometric mean ratio(Generic/Brand):", round(ratio, 3), "\n")
cat("90% CI for ratio:", round(ratio_ci_lower * 100, 1), "% to",
round(ratio_ci_upper * 100, 1), "%\n")
# === STEP 4: TOST (Two One-Sided Tests) ===
cat("\n=== TOST Equivalence Test ===\n")
# Test 1: H₀: μ_diff ≤ lower_margin vs Hₐ: μ_diff > lower_margin
t1 <- (mean_diff - lower_margin) / se_diff
p1 <- pt(t1, df=n-1, lower.tail=FALSE)
cat("Test 1 (lower bound): t =", round(t1, 3), ", p =", round(p1, 4), "\n")
# Test 2: H₀: μ_diff ≥ upper_margin vs Hₐ: μ_diff < upper_margin
t2 <- (mean_diff - upper_margin) / se_diff
p2 <- pt(t2, df=n-1, lower.tail=TRUE)
cat("Test 2 (upper bound): t =", round(t2, 3), ", p =", round(p2, 4), "\n")
# TOST p-value = max(p1, p2)
p_tost <- max(p1, p2)
cat("\nTOST p-value(max of two tests):", round(p_tost, 4), "\n")
# === STEP 5: Equivalence Decision ===
cat("\n=== Equivalence Decision ===\n")
if (ci_90_lower > lower_margin & ci_90_upper < upper_margin) {
cat("CONCLUSION: BIOEQUIVALENT ✓\n")
cat("90% CI [", round(ci_90_lower, 4), ",", round(ci_90_upper, 4),
"] entirely within margins [", round(lower_margin, 3), ",",
round(upper_margin, 3), "]\n")
cat("On ratio scale: 90% CI [", round(ratio_ci_lower * 100, 1), "%, ",
round(ratio_ci_upper * 100, 1), "%] within [80%, 125%]\n")
cat("TOST p-value =", round(p_tost, 4), "< .05: Reject H₀ of non-equivalence\n")
} else {
cat("CONCLUSION: NOT BIOEQUIVALENT ✗\n")
cat("90% CI crosses equivalence margin\n")
cat("Cannot conclude bioequivalence at α = .05 level\n")
}
# === STEP 6: Visualizations ===
# 6.1 Forest plot with equivalence margins
forest_data <- data.frame(
Study = "Generic vs Brand",
Estimate = mean_diff,
CI_lower = ci_90_lower,
CI_upper = ci_90_upper
)
ggplot(forest_data, aes(y=Study, x=Estimate)) +
geom_rect(aes(xmin=lower_margin, xmax=upper_margin, ymin=-Inf, ymax=Inf),
fill="lightgreen", alpha=0.3) +
geom_point(size=5, color="darkblue") +
geom_errorbarh(aes(xmin=CI_lower, xmax=CI_upper), height=0.2, linewidth=1.5) +
geom_vline(xintercept=0, linetype="solid", color="black", linewidth=1) +
geom_vline(xintercept=lower_margin, linetype="dashed", color="red", linewidth=1) +
geom_vline(xintercept=upper_margin, linetype="dashed", color="red", linewidth=1) +
annotate("text", x=lower_margin, y=1.3, label="Lower Margin",
size=3, hjust=1.1) +
annotate("text", x=upper_margin, y=1.3, label="Upper Margin",
size=3, hjust=-0.1) +
labs(title="Bioequivalence: Generic vs Brand-name Drug",
subtitle="90% CI for log(AUC) difference with FDA equivalence margins",
x="Log(Generic) - Log(Brand)\n[Equivalence Zone in Green]",
y="") +
theme_classic(base_size=14) +
theme(plot.title=element_text(hjust=0.5, face="bold"),
plot.subtitle=element_text(hjust=0.5))
ggsave("bioequivalence_forest.png", width=12, height=6)
# 6.2 Ratio scale visualization
ggplot(data.frame(x=1), aes(x=x)) +
geom_rect(aes(xmin=0.80, xmax=1.25, ymin=-Inf, ymax=Inf),
fill="lightgreen", alpha=0.3) +
geom_point(aes(x=ratio, y=1), size=5, color="darkblue") +
geom_errorbarh(aes(y=1, xmin=ratio_ci_lower, xmax=ratio_ci_upper),
height=0.3, linewidth=1.5) +
geom_vline(xintercept=1, linetype="solid", color="black", linewidth=1) +
geom_vline(xintercept=0.80, linetype="dashed", color="red", linewidth=1) +
geom_vline(xintercept=1.25, linetype="dashed", color="red", linewidth=1) +
scale_x_continuous(limits=c(0.7, 1.4), breaks=seq(0.7, 1.4, 0.1)) +
labs(title="Bioequivalence: Geometric Mean Ratio",
subtitle="90% CI must be entirely within 80-125% for bioequivalence",
x="Ratio: Generic AUC / Brand AUC(%)",
y="") +
theme_classic(base_size=14) +
theme(plot.title=element_text(hjust=0.5, face="bold"),
plot.subtitle=element_text(hjust=0.5),
axis.text.y=element_blank(),
axis.ticks.y=element_blank())
ggsave("bioequivalence_ratio.png", width=12, height=6)
# === APA-Style Reporting ===
cat("\n=== APA-Style Report ===\n")
cat(sprintf(
"A crossover bioequivalence study(n=%d healthy volunteers) compared a generic\nformulation to the brand-name drug using AUC(area under concentration-time curve).\n\nUsing FDA's Two One-Sided Tests(TOST) procedure with equivalence margins of\n80%%-125%% (±20%% on ratio scale), the geometric mean ratio was %.2f (90%% CI [%.2f%%, %.2f%%]).\n\nThe 90%% confidence interval fell entirely within the pre-specified equivalence\nmargins of 80%%-125%% (TOST p = %.4f < .05), supporting bioequivalence of the\ngeneric to the brand-name formulation.\n\nOn the log scale, the mean difference was %.4f (90%% CI [%.4f, %.4f]), within\nthe margins of [%.3f, %.3f].\n\nConclusion: The generic formulation is bioequivalent to the brand-name drug,\nmeeting FDA criteria for therapeutic equivalence.",
n, ratio, ratio_ci_lower * 100, ratio_ci_upper * 100, p_tost,
mean_diff, ci_90_lower, ci_90_upper, lower_margin, upper_margin
))Geometric mean ratio = 1.05, 90% CI [0.94, 1.17]. The 90% confidence interval for the Generic/Brand AUC ratio falls entirely within the FDA-specified equivalence margins of 80-125% (TOST p = .01 < .05). This supports bioequivalence, meaning the generic formulation can be considered therapeutically equivalent to the brand-name drug. The point estimate of 1.05 (5% higher generic) is clinically trivial and the CI demonstrates that the true difference is within acceptable regulatory limits. FDA requires BOTH 90% CI bounds to be within [80%, 125%] for bioequivalence approval. This example demonstrates the TOST procedure: testing simultaneously that Generic is not too low (<80%) AND not too high (>125%) compared to Brand.
Alternatives
Structured fallback pathways for choosing alternative tests when normality or slopes requirements fail.
- Non-Parametric TOST — Execute the Two One-Sided Tests using the Mann-Whitney U basis.
- Bootstrap Equivalence — Generate a 90% CI for the mean difference using 1,000 resamples.
- Non-Inferiority Pivot — Relax the mandate to prove 'Better or Same' rather than 'Exactly Same'.
- Informal Parity Audit — Report the Confidence Interval and its overlap with the margin without a formal p-value strike.
Post-hoc
Group mean comparisons and correction controls (e.g. Tukey HSD, Bonferroni) to protect against Family-Wise Error Rates.
Post-hoc pairwise tests defined for this model.
Equivalence post-hoc is an audit of 'Closeness'. Use margin-sensitivity to prove that your intervention is not just 'Within the Limit', but 'Centrally Balanced' against the gold standard.
Effect Size
Understanding effect sizes (e.g., Cohen's d, Partial Eta-Squared) and clinical impact benchmarks.
For equivalence: 90% CI must be ENTIRELY within [-Δ, +Δ]. For NI: 90% CI lower bound must be > -Δ (upper bound can exceed +Δ, indicating superiority).
Larger distance between CI bound and margin = stronger evidence for equivalence/NI. If CI barely inside margin, evidence is weak.
Even if statistically equivalent/non-inferior, assess clinical importance: is observed difference clinically trivial? For example, mean difference of 0.5 points on 100-point scale may be statistically equivalent but still clinically meaningful if margin was set too large.
0.2
0.5
0.8
Sample Size
Guidelines for minimum sample requirements and power analysis parameters.
The 'High-Power Mandate': Proving 'Similarity' requires significantly more data than proving 'Difference'. A minimum of 50 participants per group is recommended for even moderate clinical margins.
| Effect Size | Parameters | Required n |
|---|---|---|
| Small Effect | Narrow Margin (Δ=0.2) | n ≈ 450 total |
| Medium Effect | Standard Margin (Δ=0.5) | n ≈ 80 total |
| Large Effect | Wide Margin (Δ=0.8) | n ≈ 35 total |
The 'Two-Strike' Penalty: Equivalence uses Two One-Sided Tests (TOST). To maintain an 80% global power, each individual strike must achieve ~90% power, effectively doubling the required sample size compared to superiority tests.
Reporting
How to compile statistical results into publication prose matching APA and journal style guides.
An equivalence/non-inferiority trial (n = total N) compared Test treatment to Control/Standard for outcome. The pre-specified equivalence margin was Δ (or lower Δ, upper Δ for two-sided), justified by clinical rationale / % of control effect preserved / regulatory guidance. State study design: RCT, crossover, parallel-group. Primary analysis: ITT or per-protocol. The mean difference / risk difference / hazard ratio was point estimate (90% CI [lower, upper], TOST p-value or statement about CI relative to margins). For equivalence: The 90% confidence interval fell entirely within the pre-specified margins of [−Δ, +Δ], supporting equivalence (TOST p < .05). For NI: The 90% CI lower bound exceeded the NI margin of −Δ, supporting non-inferiority (p < .05). Clinical interpretation: The observed difference of X is clinically trivial / meaningful and does / does not represent a meaningful change in patient outcomes. Report sensitivity analyses if applicable: per-protocol concordance, assay sensitivity evidence. For NI with superiority claim: Additionally, the point estimate and CI suggest superiority of Test over Control (point estimate > 0 and 95% CI excludes 0).
- Point estimate (mean difference, RD, RR, HR, etc.) with 90% CI
- Pre-specified equivalence/NI margin(s) with justification
- TOST p-value or statement of CI location relative to margins
- Study design (RCT, crossover, parallel-group)
- Sample sizes per group (ITT and per-protocol)
- Primary outcome definition
- ITT and per-protocol results (if both conducted)
- Power calculation verification (was study adequately powered?)
- Assay sensitivity evidence (for NI trials)
- Clinical interpretation of observed difference magnitude
Manuscript Lab
Copy standard summary tables and forensic reporting grids to outline analysis details.
| Metric | M_diff (New-Std) | 95% CI (Upper Bound) | Margin (Limit) | p (Non-Inf) |
|---|---|---|---|---|
| Efficacy Score | -1.2 | -3.8 | -5.0 | .004 |
The Performance Gap. A negative value means the new treatment performed slightly worse than the standard.
The 'Worst Case' Scenario. We are 95% confident that the new treatment is, at worst, only 3.8 points lower than the standard.
The Clinical Tolerance. The pre-defined threshold of what we consider 'close enough' to the standard.
Command Center
Syntax libraries and function parameters for executing calculations in stats packages.
# 1. Execute TOST (Two One-Sided Tests) for Equivalence
TOSTER::tost(m1 = 75, m2 = 76, sd1 = 8, sd2 = 8, n1 = 100, n2 = 100,
low_eqbound = -5, high_eqbound = 5)
# 2. Non-Inferiority Audit
# (Check if Lower bound of CI > delta)Statistical Significance does NOT equal Equivalence. You can have a p < .05 (Difference) AND still be Equivalent if the difference is smaller than your clinical margin. Always look at the Confidence Interval, not the p-value.
# Visualize Equivalence Bounds vs CI
TOSTER::plot_tost(m1 = 75, m2 = 76, sd1 = 8, sd2 = 8, n1 = 100, n2 = 100,
low_eqbound = -5, high_eqbound = 5)Common Mistakes
Analytical caveats and corrections to maintain modeling integrity.
References
Scholarly lineage and citation keys grounding the statistical framework.