Book Review Superforecasting
Book Review Superforecasting
Everyone is a forecaster. Based on anticipated events, we all make decisions regarding our future for leisure, business, work, and retirement. But, who makes the most accurate forecasts? How would ordinary citizens compare against newspaper columnists, television pundits, and other so-called experts? And what determines the forecaster's accuracy?
In Superforecasting: The Art and Science of Prediction (Crown, 2015), Philip E. Tetlock and Dan Gardner present the results of the Good Judgment Project, a ground-breaking study of forecasting habits and results. Prof. Tetlock, of the University of Pennsylvania, holds positions in both the Wharton School of Business and the School of Arts and Sciences; Gardner is an award-winning journalist. From 1984 through 2004, the Good Judgment Project compared the forecasting accuracy of tens of thousands of ordinary volunteers with that of global experts.
For the study, participants worked alone or in teams; some were traders in prediction markets. Annually, the results were consistent. The teams of ordinary forecasters beat the wisdom of the crowd, the combined forecasts of those working alone, by roughly 10 percent. Prediction markets beat ordinary teams by about 20 percent. Superteams, teams composed of superforecasters
â?? those in the top 2 percentâ??beat prediction markets by 15 percent to 30 percent.
From the study, the researchers found that several patterns emerged. Just because you know a lot about something doesn't mean you'll be a good forecaster. Compared with the experts, the top amateur forecasters were on average 30 percent more accurate.
What you think turned out to be much less important than how you think. Superforecasters were not quants, but were better than the experts at gathering evidence, analyzing it, and drawing conclusions. They were more willing to change their estimates in response to feedback or when new information came in. They also used probabilistic thinking. Recall the raid on Bin Laden where the intelligence community rated the success of the operation in terms of a 50, 60, or 90-percent chance that he was hiding out in the compound
Tetlock divided forecasters into two groups based on their analytical style. "Hedgehogs have one grand theory (Marxist, Libertarian, whatever) which they are happy to extend into many domains. Foxes adjust their ideas based on actual events," according to Tetlock. Superforecasters were overwhelmingly foxes rather than hedgehogs, especially for long-term forecasts.
Superforecasters "Fermi-ize" complex problems; i.e., they break them down into smaller, more manageable problems. Enrico Fermi used this technique in planning the development of the atomic bomb: He divided each major step in the project into many smaller tasks, asking which must be accomplished before later ones could be tackled.
Before the Internet and without a phone book, Fermi had his students guess the number of piano tuners in Chicago, first identifying the data they needed to reach a good answer. They by estimat
ied the number of pianos in Chicago, how often pianos are tuned annually, how long it takes to tune a piano, and how many hours each year the average piano tuner works. Even today, engineering schools utilize similar Fermi problems. One popular example asks students to acquire the information needed to estimate how many square inches of pizza the student body consumes during a semester.
With over 20,000 intelligence analysts, the US intelligence community wanted to determine the accuracy of their forecasts. So, the Intelligence Advance Research Projects Activity (IARPA) sponsored a forecasting tournament. The tournament used typical intelligence questions and posed them to university teams from Michigan and MIT, teams of intelligence analysts with access to classified information, and the Good Judgment Project. Tetlock's amateur forecasters easily won the contest.
This book definitely lives up to its title, delivering important insights into the basic skills needed to improve accuracy in predictions. Because superforecasting skills are cultivated, with practice anyone should be able to improve.
All Public Comments
© 2013-2017 TechCast Global Inc Printed By: Dec 12, 2018 For personal use only