The power of the amateur
25 March, 2016
I've written before about my disdain for so-called experts who are given guru or celebrity status when it is often undeserved. Forecasters and commentators, whose views are bold and offered repeatedly, often become revered by the masses.
If someone expresses a confident view and provides a comment on anything and everything, well, he must be right. Unfortunately as The Economist noted recently, today's world is based on eminence rather than evidence.
Experts happily provide predictions and forecasts on all manner of things, influencing the opinions and behaviour of government and business leaders, investors and sometimes society as a whole.
They are frequently wrong, yet there is no post-forecast analysis undertaken to determine their success rate. Rather, the opinions of the same individuals who have been famously wrong are sought over and over, cementing their fame, irrespective of their accuracy.
If only the non-expert masses realised their opinions and forecasts can be just as accurate, and sometimes more so, than the professionals.
In the book Superforecasting: the Art and Science of Prediction, author Philip Tetlock demonstrates amateurs outperform experts in finance, politics and even national intelligence when it comes to prediction. What's more, anyone can improve their forecasting skills so all of us can be as good as the experts who currently command our attention.
Tetlock, a psychologist and professor of management at the University of Pennsylvania, identified common behavioural traits and a successful forecasting methodology after launching a competition in 2011 pitting 2,800 volunteers against some of the world's most prominent and elite forecasters.
Participants were asked questions on diverse topics including future gold prices, expectations of OPEC's global production and the stability or otherwise of political regimes.
The competition is still running today — you can check out your forecasting prowess by visiting The Good Judgment Project (www.gjopen.com) — and the results continue to confirm the ability of amateurs to deliver more confident and accurate forecasts than experts.
On average, the best performing amateur forecasters were 30 per cent more accurate than the experts, even when experts had access to classified and scientific information.
Some members of the top 2 per cent of participants, referred to by Tetlock as the "superforecasters", do not hold the educational or professional qualifications that one might expect. Among this elite group of scientists, professors and engineers were a housewife, a welfare case worker, a Pilates instructor and an underwater hockey coach.
Tetlock summarised the key traits of the superforecasters as being careful, curious, open-minded, persistent and self-critical. He says experts can be limited by their tendency to have closed minds, be quick to arrive at a view and slow to question or change it.
It seems what you think is much less important than how you think. Tetlock says it pays to look at the world not in a binary way ("it must be x or y") but in terms of probabilities (is there a greater than 50:50 chance of it being x and not y?).
He listed the lessons for those wanting to be superforecasters as:
- Begin by gathering as much information as possible.
- Nurture and develop the habit of thinking in terms of probabilities.
- Consult with others to improve your odds of being correct.
- Regularly keep score of your projections.
- Be willing to admit error and quickly change course.
While we won't all achieve superforecaster status, we should take comfort knowing our opinions are as likely to be right as those of the experts.
Three cheers for the amateur.