Atlas
statminds
NonparametricThe underlying model family class (e.g. GLM, linear model, categorical matrix, log-linear).Parametric ReferenceStatistical methods that assume a specific probability distribution family (typically normal).12-stage workflow

Sign Test

Simple nonparametric test for paired data; tests if median difference is zero by counting positive vs. negative differences.

Model familyNonparametric
Hypothesistwo-tailed
AliasesBinomial Sign Test · Median Sign Test
G1
association
G2
description
Visual Overview Dashboard
1

What is it?

Sign Test evaluates the directional differences between paired scores based purely on their positive (+) or negative (-) signs, ignoring magnitude.

2

When to use it

  • Highly Skewed Metrics: Outliers render differences uninterpretable.
  • Pure Direction: Only the direction of change (growth vs decline) is verified.
3

Core Idea

Under the null hypothesis, positive and negative signs should occur with equal probability (p = 0.50). An excess of either sign rejects H0:

++++----
4

Hypotheses

H0: Median difference is 0 (prob of positive sign is 0.5)
Ha: Median difference is non-zero (probability differs from 0.5)
5

How it works

  1. Calculate differences between paired measurements.
  2. Assign positive (+) or negative (-) signs, discarding ties.
  3. Count occurrences of the less frequent sign (k).
  4. Evaluate cumulative binomial probability with p = 0.5.
6

Assumptions

📊 Ordinal differences: Outperforming vs underperforming can be assessed.
👤 Independence: Subject pairs are independent.
7

Effect Size

Represented using **Cohen's g**: g = (positives / total) - 0.5. Ranges from -0.5 to +0.5, where 0 represents the null profile.

8

Quick Example

Positive (+)Negative (-)Binomial p
1550.041 (Significant)
Interactive Sandbox

Binomial Sign Live Laboratory

Adjust the positive success probability rate to see binomial separation.

Presets
Positive Rate (H1)0.50
Pair Count (N)20
Observed directional signs (Green: +, Amber: -)Sorted combined signs grid
-
-
+
-
-
-
-
-
-
-
+
+
+
-
-
+
-
+
+
+
Calculations Output
MetricValue
Positive Signs (+)8
Negative Signs (-)12
Cumulative Binomial p0.5034
Statistical Verdict
❌ Balanced Outcome
No significant directional dominance detected (p = 0.503). Fail to reject H0.
01Hypothesis test logic

Hypotheses

Pragmatic null and alternative hypotheses defined in mathematical notation.

A hypothesis is a question sharpened to a point. Ambiguity is the enemy of inference.
Logic Core
Null · H₀

H₀: P(X > Y) = 0.5 (probability of positive difference equals probability of negative difference)

Alternative · Hₐ

Hₐ: P(X > Y) ≠ 0.5 (more positive than negative differences, or vice versa)

Why it matters two-tailed

Tests whether positive and negative differences are equally likely. Unlike Wilcoxon, uses only direction (+/-) not magnitude. Based on binomial distribution under H₀: p = 0.5.

02Model diagnostics

Assumptions

The core mathematical criteria needed to ensure that statistical testing remains unbiased and valid.

Build your analysis on rock, not sand. Verify the mathematical foundation before building the model.
Integrity Shield
5
Assumptions
3
Critical / High Severity
How to check
Quick
Verify study design: pre-post, matched pairs, or within-subjects. Confirm each subject/unit has exactly 2 measurements to compare
Rigorous
Check data structure for pairing. For matched designs, verify matching criteria. Ensure no unpaired observations
If violated
If independent groups → use chi-square goodness-of-fit or binomial test for proportions. If multiple time points → use Cochran's Q test
How to check
Quick
Calculate differences (Time2 - Time1). Verify each can be classified as: positive (improvement/increase), negative (worsening/decrease), or zero (no change)
Rigorous
Ensure measurement scale allows directional comparison. Even ordinal data works if direction is meaningful (e.g., 'better' vs 'worse')
If violated
If differences cannot be ordered (nominal outcome) → use McNemar's test for binary paired data or Bowker's test for multi-category symmetry
mcnemarbowker
How to check
Quick
Design review: verify pairs/subjects are independent (no clustering, family grouping). Within-pair dependence is expected; between-pair independence required
Rigorous
Check for clustering (family, site, therapist). Verify no subject appears in multiple pairs. For longitudinal data, check for autocorrelation
If violated
If clustered pairs → use GEE with binomial family or mixed-effects logistic regression with random effects for clusters
gee
How to check
Quick
Count zero differences (Time2 - Time1 = 0). High proportion (>25%) suggests measurement insensitivity or ceiling/floor effects
Rigorous
Calculate: (# zeros / total n) × 100%. If >25%, investigate whether scale is sensitive enough to detect changes
If violated
Zeros automatically excluded, reducing effective n. If many zeros: (1) Report effective n after exclusion; (2) Consider more sensitive outcome measure; (3) Investigate ceiling/floor effects. For extreme ties, consider exact binomial test
How to check
Quick
No check needed! Sign test works regardless of skewness, outliers, or distribution shape. This is its key advantage
Rigorous
Sign test is valid even with severely skewed differences or extreme outliers. Only requires ability to determine sign (+/-) of differences
If violated
This assumption cannot be violated—Sign test is truly distribution-free
03Residual Forensics

Diagnostics

Checking residual plots and indices to examine model deviations and ensure standard error integrity.

Trust, but verify. The outliers often hold more truth than the averages.
System Health
Essential checks
  1. Count of positive differences (n+)
  2. Count of negative differences (n-)
  3. Count of zero differences (excluded)
  4. Effective sample size (n = n+ + n-)
Recommended checks
  1. Histogram of differences (to show why Sign test chosen over Wilcoxon)
  2. Proportion of positive differences
  3. Binomial probability under H₀ (p = 0.5)
  4. 95% CI for proportion of positive differences
04Live Instances

Applied Minds

Review concrete study examples, data layout guidelines, and copy executable syntax scripts.

Theory is the map. Practice is the terrain. Simulation bridges the gap.
Applied Wisdom
Example 01

Preference for Meditation vs. Exercise (Severely Skewed Differences)

Research question: Do participants rate meditation higher than exercise for stress reduction? Design: Within-subjects (n=50), each rates both on 0-100 scale. Outcome: Difference scores (Meditation - Exercise) severely right-skewed with outliers. Sign test chosen because Wilcoxon symmetry assumption violated.

DesignWithin-subjects preference ratings
Outcome ScaleRating difference (continuous, severely skewed)
# Sign Test: Meditation vs. Exercise preference (skewed data)
library(DescTools)
library(tidyverse)

set.seed(2025)
n <- 50
data <- data.frame(
  id = 1:n,
  meditation = round(rnorm(n, 65, 18)),
  exercise = round(rnorm(n, 58, 15))
)
data$difference <- data$meditation - data$exercise

# Check skewness (justifies Sign test over Wilcoxon)
skew <- moments::skewness(data$difference)
cat("Skewness =", round(skew, 2), "\n")
cat("Severely skewed → Sign test appropriate\n\n")

# Count signs
n_pos <- sum(data$difference > 0)  # Meditation > Exercise
n_neg <- sum(data$difference < 0)  # Exercise > Meditation
n_zero <- sum(data$difference == 0)  # Ties (excluded)
n_effective <- n_pos + n_neg

cat("Positive differences(Meditation > Exercise):", n_pos, "\n")
cat("Negative differences(Exercise > Meditation):", n_neg, "\n")
cat("Zero differences(ties, excluded):", n_zero, "\n")
cat("Effective n =", n_effective, "\n\n")

# Sign Test
sign_result <- SignTest(data$meditation, data$exercise, alternative="two.sided")
print(sign_result)

# Manual calculation: binomial test
binom_result <- binom.test(n_pos, n=n_effective, p=0.5, alternative="two.sided")
print(binom_result)

# Effect size: proportion of positive differences
prop_pos <- n_pos / n_effective
cat("\nProportion favoring meditation:", round(prop_pos, 2), "\n")
cat("95% CI:", round(binom_result$conf.int, 2), "\n")

# Visualize
ggplot(data, aes(x=difference)) +
  geom_histogram(bins=20, fill="steelblue", alpha=0.7) +
  geom_vline(xintercept=0, color="red", linetype="dashed", size=1) +
  labs(title="Severely Skewed Differences → Sign Test",
       subtitle=paste("Skewness =", round(skew, 2)),
       x="Difference(Meditation - Exercise)") +
  theme_minimal()

# APA Report
cat("\n=== APA Report ===\n")
cat(paste0(
  "A Sign test was conducted to compare preference ratings for meditation vs. exercise. ",
  "The test was chosen because difference scores were severely skewed(skew = ", round(skew, 2), "), ",
  "violating Wilcoxon's symmetry assumption. Of ", n_effective, " participants with non-zero differences, ",
  n_pos, " rated meditation higher and ", n_neg, " rated exercise higher(Sign test p ", 
  ifelse(binom_result$p.value < 0.001, "< .001", paste("=", round(binom_result$p.value, 3))), "). ",
  "Proportion favoring meditation: ", round(prop_pos, 2), ", 95% CI [",
  round(binom_result$conf.int[1], 2), ", ", round(binom_result$conf.int[2], 2), "]."
))
Interpretation Blueprint

Sign test p = .012. Of 48 participants with non-zero differences, 32 (67%) preferred meditation over exercise, 95% CI [52%, 79%]. Severely skewed differences (skew = 1.85) made Wilcoxon inappropriate; Sign test valid regardless of shape.

05Tactical Pivots

Alternatives

Structured fallback pathways for choosing alternative tests when normality or slopes requirements fail.

When the path is blocked, pivot. Rigor is not rigidity; it is the intelligent adaptation to reality.
Adaptive Strategy
Measurement Precision Ladder Ideal · Directional Binary (+/-)
Ratio / Interval
Consider Paired T-Test or Wilcoxon. Sign Test ignores both magnitude and rank-magnitude, sacrificing significant power.
Extreme Data Loss
Ordinal Pairs
Pivot to Wilcoxon Signed-Rank Test to exploit the relative sizes of the internal shifts.
Threshold Loss
Binary Direction
Maintain Sign Test logic. The absolute standard for auditing 'Up-Flips' vs 'Down-Flips' without distributional assumptions.
Peak Signal
Temporal Trajectory Audit Static Sign Audit
Paired Shift
Before vs After.
Stay with Sign Test. Find if the overall clinical pulse is directional.
Multi-Temporal
3+ timepoints.
Pivot to Cochran's Q Test to audit the consistency of directional success.
Adaptive Technical Safeguards · adaptive safeguards
symmetric deltas detected
  • Wilcoxon Signed-Rank — Return to the rank-weighted strike to increase power by 30%.
too many zeros
  • Zero-Augmented Sign Test — Account for the clinical meaning of 'No Change' participants.
  • McNemar Strike — Treat 'Zero' as a stable state in a 2x2 grid.
06Adjusted Comparisons

Post-hoc

Group mean comparisons and correction controls (e.g. Tukey HSD, Bonferroni) to protect against Family-Wise Error Rates.

The omnibus test opens the door; post-hoc analysis explores the room.
Forensic Detail
Adjusted Comparisons

Post-hoc pairwise tests defined for this model.

Interpretation Guidelines

The Sign Test is the most robust audit of 'Momentum'. If your participants don't even agree on the *direction* of change, your intervention lacks a unified clinical story.

07Standardized scale impact

Effect Size

Understanding effect sizes (e.g., Cohen's d, Partial Eta-Squared) and clinical impact benchmarks.

Significance is noise. Magnitude is the signal. Measure the impact, not just the probability.
Impact Magnitude

Range: 0 to 1. Under H₀: p = 0.5. Report with 95% CI. Example: 'Proportion improved: 0.72, 95% CI [0.58, 0.83]'

p_positive - p_negative. Range: -1 to +1. Example: 0.72 - 0.28 = 0.44 (44% more positive than negative)

Recommended Metric: proportion of positive differences with 95% CI (from binomial distribution)
Small
0.2
Medium
0.5
Large
0.8
0.50
proportion of positive differences with 95% CI (from binomial distribution)
Recommended Measure
3
Available Metrics
ReportUse proportion of positive differences with 95% CI (from binomial distribution) to represent clinical impact magnitude.
08Statistical Power

Sample Size

Guidelines for minimum sample requirements and power analysis parameters.

An underpowered study is an ethical failure. Respect the data by collecting enough of it.
Power Protocol
Floor Requirements

At least 10 pairs for reasonable power. For n < 25, use exact binomial test (not normal approximation)

Effect SizeParametersRequired n
Small Effectp = 0.60 (60% positive)n ≈ 130 pairs
Medium Effectp = 0.70 (70% positive)n ≈ 23 pairs
Large Effectp = 0.80 (80% positive)n ≈ 9 pairs
G*Power StrategyUse binomial power analysis: power = P(reject H₀ | true proportion). For p = 0.7 (70% positive), α = .05, power = .80: need n ≈ 23 pairs
09APA narrative blueprint

Reporting

How to compile statistical results into publication prose matching APA and journal style guides.

Data does not speak for itself. It requires a translator. Be clear, be precise, be honest.
Narrative Arc
Reusable template

A Sign test was conducted to compare condition 1 and condition 2. State why Sign test chosen: 'The Sign test was used due to severely skewed differences (skewness = X.XX) violating Wilcoxon's symmetry assumption' OR 'to ensure robustness to extreme outliers'. Of n_effective participants with non-zero differences, n_positive showed improvement and n_negative showed worsening (Sign test p = .XXX). The proportion showing improvement was proportion, 95% CI XX%, XX%. Interpret in context.

Essential statistics to report
  • Number of positive differences
  • Number of negative differences
  • Number of zeros (excluded)
  • Effective sample size
  • p-value (from binomial test)
  • Proportion of positive differences with 95% CI
  • Justification for using Sign test over Wilcoxon
10Exhibit Builder

Manuscript Lab

Copy standard summary tables and forensic reporting grids to outline analysis details.

Table 1: Sign Test for Paired Comparison (Directional Change)
ComparisonPositive (+) ShiftsNegative (-) ShiftsTiesp-value
Post - Pre3884< .001
Note. N = 50. Only counts the direction of change, ignoring the magnitude.
38 vs 8Identifies the 'Majority Rule'. 76% of participants improved, providing clear evidence of a treatment effect even without normality.
Header glossary

The 'Directional Winners'. The number of subjects who improved, regardless of by how much.

Binomial Probability. The chance of getting 38/50 'heads' if the intervention was a random coin flip.

11Algorithmic Logic

Command Center

Syntax libraries and function parameters for executing calculations in stats packages.

Code is the modern laboratory. Clean execution ensures reproducible discovery.
Execution Engine
# 1. Execute Sign Test
DescTools::SignTest(x, y)
Library stack
R
DescTools
Python
scipy.stats
Elite Forensic Strike

The Sign test only considers the direction of change, completely ignoring magnitude. If differences are symmetric, the Wilcoxon signed-rank test is much more powerful.

# Compare directly to Wilcoxon Signed-Rank Test
wilcox.test(x, y, paired = TRUE)
12The Over-adjustment Trap

Common Mistakes

Analytical caveats and corrections to maintain modeling integrity.

Wisdom is learning from the failures of others. Anticipate the error before it occurs.
Defensive Logic
Why it's wrong
Sign test does NOT test medians, means, or any location parameter. It tests the PROPORTION of positive vs. negative differences (H₀: P(+) = P(-) = 0.5). Even if median difference is zero, Sign test can be significant if proportions differ from 50-50. Unlike Wilcoxon which tests stochastic dominance or medians (when symmetric), Sign test is purely about direction proportions. This is analogous to Divine et al. (2018) clarification for Mann-Whitney, but Sign test is even more limited—it only tests sign proportions.
The correction
Report Sign test results as 'proportion of cases showing positive/negative change' NOT 'median/mean difference'. Example: 'Sign test showed 72% of participants improved (p = .003)' NOT 'median improvement was significant'. If you need to test medians with skewed data, use quantile regression at 50th percentile or Mood's median test. Sign test is only about direction, not magnitude.
Why it's wrong
When differences are approximately symmetric, Wilcoxon has ~30-50% more power than Sign test. Wilcoxon uses magnitude information (ranks); Sign test only uses direction. Using Sign test unnecessarily when Wilcoxon is valid wastes statistical power—you're throwing away magnitude information.
The correction
Check symmetry of differences (histogram, skewness). If symmetric (|skew| < 0.5): use Wilcoxon (more powerful). If asymmetric (|skew| > 1): use Sign test (robust). Report: 'Sign test was used because differences were severely right-skewed (skew = 1.35), violating Wilcoxon's symmetry assumption'.
Why it's wrong
Zero differences are excluded from Sign test, reducing effective n. Failing to report: (1) inflates apparent sample size, (2) hides measurement issues (many zeros suggest insensitive scale or ceiling/floor effects). Readers need effective n for power interpretation.
The correction
Always report: 'Of N participants, X had zero difference (excluded), yielding effective n = Y for Sign test'. If >25% zeros, investigate measurement sensitivity. Example: 'Sign test was based on 42 participants; 8 showed no change and were excluded from analysis'.
Why it's wrong
p-value indicates significance but not magnitude. CI shows range of plausible proportions and practical importance. For Sign test, proportion with CI is the key effect size. Without CI, readers can't assess whether effect is large (e.g., 80% improved) or small (e.g., 52% improved).
The correction
ALWAYS report proportion with CI: 'Proportion improved: 0.68, 95% CI [0.53, 0.81]'. Calculate exact binomial CI: binom.test()$conf.int in R or use Wilson score interval. Interpret: does CI include 0.5 (no effect)? How far from 0.5?
Why it's wrong
Sign test requires PAIRED observations (pre-post, matched pairs). For independent groups comparing proportions, need different test. Using Sign test on independent groups violates pairing assumption and gives invalid results.
The correction
For independent groups: use chi-square test of proportions, Fisher's exact test, or binomial test for single group vs. hypothesized proportion. For paired binary data: use McNemar's test. Sign test is ONLY for paired continuous/ordinal data comparing direction of differences.
Why it's wrong
Non-significant Sign test (p > .05) means 'insufficient evidence that proportion differs from 50-50', NOT 'no difference exists'. Could be: (1) true null (50-50 split), (2) underpowered study, (3) small effect size undetectable with current n. Absence of evidence ≠ evidence of absence.
The correction
For p > .05, report: 'Sign test did not detect significant difference in direction of changes (p = .XX), with [proportion]% showing positive change, 95% CI [XX%, XX%]'. Check if CI includes 0.5 and is wide (suggests low power). Consider post-hoc power analysis or equivalence testing if claiming 'no effect'.
Why it's wrong
Sign test treats a 1-point improvement same as 100-point improvement—both count as +1. Patient who improved 2 points = patient who improved 50 points in Sign test. This can miss clinically meaningful differences in magnitude. For example, all patients might improve, but improvements could be trivial (all +1 point) vs. substantial (all +20 points)—Sign test can't distinguish.
The correction
When magnitude matters (clinical significance), supplement Sign test with descriptive statistics: 'Sign test: 85% improved (p < .001). Among improvers, median improvement = 12 points (IQR: 8-18); among those who worsened, median worsening = 3 points (IQR: 2-5)'. Or use Wilcoxon if assumptions met (incorporates magnitude via ranks).
Why it's wrong
Some software uses normal approximation to binomial for Sign test. With small n (< 25), this approximation is inaccurate—can inflate Type I error. Exact binomial test is needed for valid p-values with small samples.
The correction
For n < 25: use exact binomial test (binom.test() in R, binom_test() in Python). For n ≥ 25: normal approximation is acceptable. Always verify software uses exact test for small samples. Report: 'Exact Sign test was used due to small sample size (n = 18)'.
13Academic Lineage

References

Scholarly lineage and citation keys grounding the statistical framework.

We stand on the shoulders of giants. Honor the source of the method.
Academic Lineage
[1]
Dixon, W. J., & Mood, A. M. (1946). The statistical sign test. Journal of the American Statistical Association, 41(236), 557-566.
Original paper on the Sign test. Foundation for distribution-free paired comparisons based on signs.
doi: 10.1080/01621459.1946.10501898
[2]
Divine, G. W., Norton, H. J., Barón, A. E., & Juarez-Colunga, E. (2018). The Wilcoxon–Mann–Whitney procedure fails as a test of medians. The American Statistician, 72(3), 278-286.
While focused on Mann-Whitney, this paper's logic applies to Sign test interpretation: nonparametric tests don't necessarily test location parameters (medians/means). Sign test specifically tests proportion of positive differences, not medians. See one_way_anova.json common mistake #7.
doi: 10.1080/00031305.2017.1305291
[3]
Conover, W. J. (1999). Practical Nonparametric Statistics (3rd ed.). Wiley.
Comprehensive coverage of Sign test, including exact and asymptotic methods, power comparisons with Wilcoxon, and appropriate use cases.
[4]
Gibbons, J. D., & Chakraborti, S. (2011). Nonparametric Statistical Inference (5th ed.). Chapman & Hall/CRC.
Detailed treatment of Sign test theory, relationship to binomial distribution, and efficiency relative to parametric tests. Chapter 6.
statminds · SignMind reference · v2.2 · updated 2026-01-1715 of 15 sections