Regression to the Mean

Kahneman explains one of the most underappreciated statistical phenomena and the source of countless causal illusions: regression to the mean. Whenever an outcome combines skill and luck, an extreme result is unlikely to repeat, because the luck that produced the extreme will not recur; the next observation drifts back toward the average. This is a pure consequence of imperfect correlation, requiring no cause at all — yet the mind insists on finding one.

The mechanism is that any performance reflects both a stable component and chance. An exceptionally good result usually combines genuine ability with good luck; since luck does not persist, the following performance tends to be less extreme — not because anything caused a decline, but because the lucky boost is gone. The same holds in reverse for an exceptionally poor result, which tends to be followed by improvement. Whenever the correlation between two measures is less than perfect, regression toward the mean is mathematically guaranteed.

Kahneman's unforgettable illustration comes from instructors in the Israeli Air Force. They observed that cadets praised after an excellent maneuver tended to do worse next time, while those screamed at after a poor one tended to do better, and concluded that punishment works and reward backfires. But the pattern was pure regression: an unusually good or bad flight is naturally followed by a more average one regardless of feedback. The instructors had constructed a confident causal theory out of a statistical inevitability — and, worse, it taught them to favor punishment.

The consequence is that regression is everywhere misread as cause and effect. The 'sophomore slump,' the apparent failure of treatments tried at a crisis point, the seeming jinx on magazine-cover athletes — all are substantially regression effects dressed up as causal stories. Because System 1 demands a cause for every change and has no intuitive grasp of regression, we routinely credit or blame interventions for movements that chance alone would produce.

The applied takeaway is to suspect regression whenever an extreme is followed by a move toward the average. Before concluding that a coaching change, a new policy, a medical treatment, or a motivational tactic caused an improvement or decline, ask whether the starting point was itself extreme — because extremes regress on their own. The only way to know if an intervention truly worked is a proper comparison, not the natural drift back toward the mean that would have happened anyway.

Kahneman's deeper observation is that regression is profoundly counterintuitive because it has no agent and tells no story, and our minds are built for agents and stories. Even trained scientists slip into causal language about it, and it took the genius of Francis Galton to identify the phenomenon at all. Grasping regression is a hallmark of statistical thinking precisely because it requires resisting the automatic causal interpretation that System 1 supplies — accepting that sometimes the explanation for a change is simply that the previous result was unusual.