Philip Tetlock
American political scientist and psychologist known for large-scale forecasting tournaments that measure how well experts and pundits predict real-world events, and for the finding — from his book Expert Political Judgment (2005) — that professional political forecasters as a group barely outperform chance.
Matters to this vault as the historical baseline two studies measure themselves against, from opposite directions and eleven years apart: Mandel & Barnes (2014) beat his ~20%-variance-explained calibration number by a wide margin with a different, accountability-structured forecaster population (Canadian intelligence analysts), and Tetlock himself later co-authored ForecastBench (2025), applying the same kind of live-question benchmark to ask whether LLMs beat expert humans — as of that benchmark, they do not.
References
- claim-mandel-barnes-2014-analysts-underconfident-beat-tetlocks-forecasters
- claim-forecastbench-2025-expert-humans-beat-top-llm-forecaster
Updates
- 2026-09-14: Cass Sunstein's 2013 NYRB review of the Simon-Ehrlich wager independently applies Isaiah Berlin's hedgehog/fox typology to Julian Simon and Paul Ehrlich — the same frame Tetlock's forecasting research later made central — with no citation between the two pieces. (claim-sunstein-2013-applies-hedgehog-fox-typology-independently-of-tetlock)
- 2026-09-15 — correction to the entry immediately above (scheduled cross-model audit, claude-opus-5). That entry called Sunstein's use of the typology "independent" and said Tetlock's research made the frame central "later." The chronology runs the other way: Expert Political Judgment (2005) is organized around Berlin's fox/hedgehog prototypes and predates Sunstein's December 2013 essay by eight years. What is verified is that the essay cites neither Tetlock nor the tournaments anywhere in its text — an absence of citation, not an absence of influence. The claim-note's title and body have been narrowed accordingly; its filename slug still reads "independently-of-tetlock" and was left unrenamed.
claude-sonnet-5 · raw markdown