The Law of Small Numbers

Kahneman examines our flawed intuitions about statistics, beginning with a genuine mathematical fact: small samples produce extreme results far more often than large samples do. A survey of a handful of people will yield wildly variable proportions, while a large survey converges on the true value. The error he diagnoses is that we expect small samples to be as reliable and representative as large ones — a belief he and Tversky wryly named the 'law of small numbers,' a faith that any sample resembles the population it is drawn from.

The mechanism is that System 1 is a pattern-seeker that suppresses doubt about the adequacy of evidence. It does not have an intuitive sense that sample size matters; it treats the result of three observations with nearly the same confidence as the result of three thousand. WYSIATI compounds this: the mind works with the data in front of it and never registers that a small sample is thin evidence. We are therefore primed to over-infer, to see a real effect where there is only the noise of a small number.

Kahneman's signature example concerns the incidence of kidney cancer across U.S. counties. The counties with the lowest rates are predominantly rural, sparsely populated, and Republican — inviting a tidy causal story about clean country living. But the counties with the highest rates are also predominantly rural, sparsely populated, and Republican. The truth is that small populations produce extreme rates in both directions purely as a sampling artifact; there is no causal story, only the mathematics of small numbers — yet the mind insists on inventing one.

The broader consequence is our chronic misreading of randomness. We see clusters, streaks, and patterns in sequences that are in fact random — the supposed 'hot hand' in basketball, the apparent clustering of wartime bombs over London — because random processes routinely produce groupings that look meaningful. System 1 cannot accept that a sequence is patternless; it manufactures causes for what chance alone explains, and it is especially fooled when the sample is small enough to be volatile.

The applied takeaway is to ask how much evidence a striking result actually rests on. When a small school, a small clinic, a short track record, or a brief trial shows an extreme outcome, the extremity is more likely a product of small-sample volatility than of a real underlying cause. Before believing a pattern, ask whether the sample is large enough to support it; before crediting a streak, ask whether chance alone would produce it about as often.

Kahneman's deeper observation is that we are far too willing to believe in causes and far too reluctant to credit chance, and the law of small numbers is one face of that bias. The instinct to explain — to find the reason behind every pattern — is so strong that we routinely overstate the reliability of small samples and understate the role of luck. Statistical thinking does not come naturally; it must be deliberately summoned by System 2, against the grain of a System 1 built to find meaning everywhere.