The detailed methodology of Tetlock's IARPA tournament is covered in Article 3. This article focuses on the specific individuals who emerged as superforecasters and what their practices tell us about the nature of prediction as a learnable skill.
The superforecasters were not, for the most part, domain experts. Among the top performers were a retired irrigation engineer from Nebraska, a filmmaker from Brooklyn, a former ballroom dance instructor, and a pharmacist who forecasted in his spare time. What they shared was not expertise but a set of cognitive practices that, taken together, constituted a method.1
Fermi decomposition. When faced with a complex question like "Will Iran and Israel engage in direct military confrontation before December 2025?", superforecasters did not attempt to answer it directly. They decomposed it into component questions. What is the base rate of direct military confrontations between hostile states that have been in a cold conflict for more than twenty years? What has changed in the last twelve months that would make this base rate higher or lower? What would the precursor signals look like, and which of them are present? Each component question is easier to estimate than the whole, and the assembled estimates are more reliable than a direct gut judgment.
Reference class reasoning. Helmuth von Moltke the Elder, the Prussian military strategist, believed in deliberating fully before committing to a plan, then executing decisively. Superforecasters applied a version of this principle: before analyzing the specific case, they identified the reference class. How often do events like this one occur? What is the outside view? This practice counteracts the narrative bias that makes every current situation feel unique and unprecedented. Most situations are not unprecedented. They have historical analogues, and the historical frequency is the best starting point.
The update discipline. The difference between a good forecaster and a superforecaster was often the frequency and precision of updates. Where an ordinary forecaster might check a question once and move on, a superforecaster would return to the question daily, scanning for new information that should shift the estimate. The updates were small, typically two to five percentage points, but they accumulated. Over the course of a question's lifetime, a superforecaster might make twenty or thirty revisions. The final estimate, shaped by this iterative process, was consistently more accurate than the initial estimate.
Intellectual humility without paralysis. Superforecasters were comfortable saying "I don't know." They expressed uncertainty in precise numerical terms. They were willing to change their minds. But they were not indecisive. Helmut Schmidt's dictum, "people who have visions should go see a doctor," captures the superforecaster temperament: skeptical of grand narratives, attentive to evidence, decisive once the evidence is sufficient. The commitment to perpetual beta, always updating, never arriving at a final answer, was the disposition that most strongly predicted accuracy.
The translation of these practices into computational systems is the subject of this entire knowledge base. Fermi decomposition maps to the node structure of a constraint graph. Reference class reasoning maps to the base rate databases that ground prior probabilities. Update discipline maps to the fact pipeline that triggers re-simulation on new evidence. Intellectual humility maps to the adversarial falsification system described in Article 13 that tries to break every prediction. The superforecasters showed that prediction is a method. The question is whether the method can be embedded in architecture.
References
- Tetlock, P.E. & Gardner, D. (2015). Superforecasting: The Art and Science of Prediction. Crown.
- Mellers, B. et al. (2014). "Psychological Strategies for Winning a Geopolitical Forecasting Tournament." Psychological Science, 25(5), 1106-1115.
- Satopaa, V. et al. (2014). "Combining multiple probability predictions using a simple logit model." International Journal of Forecasting, 30(2), 344-356.