The Illusion of Validity

He and his fellow assessors formed vivid, confident impressions of who would make good officers — and follow-up data showed their predictions were barely better than chance. Yet the striking thing was that knowing their assessments had failed did nothing to dent the confidence with which they made the next batch; the coherent impression of leadership felt valid regardless of the statistical record that said it was not.

Kahneman extends the analysis to financial markets, where the illusion of validity has enormous stakes. The performance of mutual-fund managers from year to year shows almost no persistence — results consistent with luck rather than skill — yet the industry is built on the premise of skill, and managers are paid and celebrated as if their good years proved expertise. He recounts presenting a firm with evidence that its traders' rankings were essentially random, and watching the news bounce off; the culture of skill was immune to the data.

The mechanism sustaining the illusion is both cognitive and social. Cognitively, a confident, coherent story feels like knowledge, and the feeling does not wait for confirmation. Socially, professions and institutions are organized around the assumption of expertise, providing mutual reinforcement, prestige, and incentive to maintain confidence; admitting that one's judgments lack validity would undermine careers and self-image. The illusion is therefore protected by feeling, by community, and by money all at once.

The applied takeaway is to treat your own confidence — and others' — as a weak signal of accuracy. The strength of a conviction reflects how good the story feels, not how well it predicts, so high confidence in a forecast deserves scrutiny rather than trust, especially in domains where outcomes are uncertain and feedback is poor. Be particularly wary of confident experts in markets, long-range forecasting, and selection, where the data on validity is far less flattering than the confidence on display.

Kahneman's deeper point is that the illusion of validity is not a failing of unintelligent or dishonest people but a structural feature of how the mind generates conviction, reinforced by institutions that depend on it. Confidence and competence are different things, and they can diverge wildly; the most confident forecaster may be no more accurate than chance. Recognizing this does not abolish the feeling of certainty — Kahneman admits he still feels it — but it warns us not to mistake that feeling for evidence, in ourselves or in the experts we are tempted to trust.