Learn AI Create the Future — with Pranay
Main site

Hypothesis Testing and Inference

Pearson, Spearman and Kendall

Three ways to score how two quantities move together — straight-line strength, rank agreement, and pairwise ordering — and when each one misleads.

Read these first

On this page 5
  1. Why it exists
  2. How the three differ
  3. A real example you have seen
  4. Remember this
  5. What to learn next

One lesson, three depths. Pick the one that fits you today — you can switch any time.

Beginner — No maths. Plain English.

A correlation coefficient is a single number, from -1 to +1, scoring how two quantities move together.

Watch a market on a rainy day. Umbrella sales go up, ice-cream sales go down, and onion prices ignore the weather completely. Each pairing has a direction and a strength of moving together.

Correlation packs that into one number. Near +1: they rise together. Near -1: one rises as the other falls. Near 0: no steady connection.

Why it exists

"Taller parents have taller children — but how strongly?" Francis Galton asked that in the 1880s, and his student Karl Pearson turned the answer into a formula.

One number lets you compare relationships. Is study time more tied to marks than sleep is? Does a feature predict the label strongly enough to keep? You need a common scale to argue with.

How the three differ

There are three famous versions because "move together" hides an ambiguity.

Pearson   asks: do the points sit near a STRAIGHT LINE?
Spearman  asks: when one goes up, does the other go up too —
                by any amount, straight or curved?
Kendall   asks: pick any two data points. Does the pair agree
                on the ordering? Score all pairs.

Study hours and marks rise together, but each extra hour adds less than the last. That curve weakens Pearson — the points bend away from a line. Spearman and Kendall stay at full score, because the direction never reverses.

And one typo-sized outlier can wreck Pearson, while the rank-based two shrug it off. You will watch that happen in the developer tab.

A real example you have seen

Weather apps correlating humidity with rain chance. Fitness bands correlating steps with calories. Cricket analysts correlating dot-ball percentage with match wins.

And a warning that made history: ice-cream sales correlate with drowning deaths. Summer causes both. Correlation is not causation — moving together is not the same as one causing the other.

Remember this

  • One number, -1 to +1: direction and strength of moving together.
  • Pearson = straight lines; Spearman/Kendall = consistent direction, any curve.
  • Outliers wreck Pearson; ranks protect the other two.

What to learn next

Developer — Code and libraries.

Setup

bash
pip install numpy scipy

Outputs verified with numpy 1.26 and scipy 1.14, CPU.

Three coefficients, one curve, one typo

Study hours versus exam score. The gains flatten out — realistic, not straight.

three_correlations.py
import numpy as np
from scipy import stats

hours = np.array([1, 2, 3, 4, 5, 6, 7, 8])
score = np.array([35, 52, 61, 68, 72, 75, 77, 78])

r, p = stats.pearsonr(hours, score)
rho, p_s = stats.spearmanr(hours, score)
tau, p_k = stats.kendalltau(hours, score)
print(f"Pearson  r   = {r:.3f}")
print(f"Spearman rho = {rho:.3f}")
print(f"Kendall  tau = {tau:.3f}")

# One data-entry mistake: someone typed 780 instead of 78.
score_typo = score.copy(); score_typo[-1] = 780
print(f"with the typo -> Pearson {stats.pearsonr(hours, score_typo)[0]:.3f}, "
      f"Spearman {stats.spearmanr(hours, score_typo)[0]:.3f}")
Output
Pearson  r   = 0.923
Spearman rho = 1.000
Kendall  tau = 1.000
with the typo -> Pearson 0.619, Spearman 1.000

The walkthrough

The clean data tells the shape story. Scores rise with every hour, without exception, so Spearman and Kendall report a perfect 1.000. Pearson reports 0.923 — high, but docked for the curve. The gap between Pearson and Spearman is itself a diagnostic: monotonic but not linear.

The typo line tells the robustness story. One wrong digit drags Pearson from 0.923 to 0.619. Spearman does not move at all: 780 occupies the same rank as 78 did — still the largest.

Kendall reads as a probability. tau = 1.0 means every pair of students agrees: more hours, higher score. In general, tau is (agreeing pairs − disagreeing pairs) as a fraction of all pairs — a cleaner interpretation than Spearman's, at similar robustness.

All three come with p-values (the second return value) testing "true correlation is zero". With 8 points, even strong-looking correlations can be flukes — check the p-value before celebrating.

Common mistakes

Reading correlation as cause. A model feature can correlate with the label through a hidden third factor, and break in production when that factor drifts. The correlation was real; the arrow was imaginary.

Trusting r = 0 to mean "no relationship". A perfect U-shape scores near zero on all three — direction reverses halfway, and everything cancels. Plot before you summarise; see matplotlib.

Comparing correlations computed on different ranges. Restricting the range shrinks correlation. Marks versus hours among only toppers looks weak because everyone studies a lot. This is called range restriction, and it silently rigs comparisons.

Feature selection by Pearson alone. It misses non-linear, non-monotonic dependence entirely. For that job, consider mutual information (sklearn.feature_selection.mutual_info_regression, scikit-learn 1.7).

Try it yourself

Build a perfect parabola: y = (hours - 4.5) ** 2. Predict all three coefficients before running. Then check what each reports, and reconcile it with the "consistent direction" idea.

What to learn next

Researcher — Mathematics and papers.

Definitions

Pearson measures linear association:

$$ r = \frac{\sum_{i=1}^{n}(x_i - \bar{x})(y_i - \bar{y})}{\sqrt{\sum_i (x_i - \bar{x})^2}\sqrt{\sum_i (y_i - \bar{y})^2}} $$

Where:

  • $x_i, y_i$ — paired observations; $\bar{x}, \bar{y}$ — sample means.
  • $r$ — the sample estimate of $\rho = \operatorname{Cov}(X,Y)/(\sigma_X \sigma_Y)$.

Spearman's $\rho_s$ is Pearson computed on ranks $R(x_i), R(y_i)$; with no ties it reduces to

$$ \rho_s = 1 - \frac{6 \sum_i d_i^2}{n(n^2 - 1)}, \qquad d_i = R(x_i) - R(y_i) $$

Kendall's $\tau$ counts pair agreements. A pair $(i, j)$ is concordant when $(x_i - x_j)(y_i - y_j) > 0$, discordant when negative:

$$ \tau = \frac{n_c - n_d}{\binom{n}{2}} $$

with $n_c, n_d$ the concordant and discordant counts (tie-corrected variants $\tau_b$, $\tau_c$ adjust the denominator; scipy computes $\tau_b$).

Inference

For Pearson under bivariate normality, $t = r\sqrt{(n-2)/(1-r^2)} \sim t_{n-2}$ under $\rho = 0$. Confidence intervals use Fisher's z-transformation:

$$ z = \operatorname{arctanh}(r), \qquad z \sim \mathcal{N}!\left(\operatorname{arctanh}(\rho),\ \tfrac{1}{n-3}\right) $$

which normalises the skewed sampling distribution of $r$ near $\pm 1$. For Kendall, the null variance is $\frac{2(2n+5)}{9n(n-1)}$ and normality kicks in quickly. All three have permutation-exact versions for small $n$.

Relationships and properties

  • $\tau$ and $\rho_s$ are both functionals of the copula alone — invariant to any monotone transformation of the margins. Pearson is not.
  • Under bivariate normality: $\tau = \frac{2}{\pi}\arcsin(\rho)$ and $\rho_s = \frac{6}{\pi}\arcsin(\rho/2)$ (Greiner's relations) — so the three are mutually convertible only under that assumption.
  • $|\tau| \leq |\rho_s|$ in typical data; $\tau$'s interpretation as $2\Pr(\text{concordance}) - 1$ makes it the preferred report in modern practice.
  • Zero correlation does not imply independence except under joint normality. Distance correlation (Székely, Rizzo and Bakirov, 2007, Measuring and testing dependence by correlation of distances) is zero iff independent, at $O(n^2)$ cost; scipy.stats exposes the related energy-distance tools, and dcor implements it directly.

Complexity

Pearson and Spearman: $O(n \log n)$ (sorting dominates). Naive Kendall is $O(n^2)$; Knight's algorithm (1966) achieves $O(n \log n)$ via merge-sort inversion counting — scipy uses it.

History

Galton introduced regression toward the mean (1886); Pearson formalised $r$ (1896); Spearman proposed rank correlation in The proof and measurement of association between two things (1904); Kendall's $\tau$ appeared in A new measure of rank correlation (1938).

What to learn next