How to Become a Superforecaster
For every decision, from simple things like judging whether to carry an umbrella, how much food to order, which route to use in the morning to more complex ones like how much inventory to hold, what growth strategy to follow, which team to invest in, what policy decision to rollout or intervention to undertake halfway across the globe, we make forecasts.
Even though I now realize that I make thousands of them every day, I had not really actively thought of the act of forecasting and its importance, what fallacies I might be a victim of or how to build and improve my mental models until I read the book Superforecasting, by Philip Tetlock and Dan Gardner, which I highly recommend.
Since I learn by sharing (and writing) I wanted to jot down my quick takeaways from the book which changed by understanding of how I navigate my everyday life and can grow as an investor and a leader.
- Call Me Maybe: When I say that there is a fair chance it’ll rain next week or when Steve Ballmer, Microsoft’s CEO in 2007, said “There’s no chance that the iPhone is going to get any significant market share” what does that all mean? Well, it depends. Let’s focus on the rain example. If it does rain, fair chance can be stretched out to mean greater than 50%, else it can be shrunk to something smaller than 50%. Either way, I nailed it, right? But did I actually and was Ballmer utterly wrong? This is the wrong-side-of-maybe fallacy. Given the incentives, it is not difficult to understand why people choose to be sort of right at the expense of being vague. But without clear language around time, place and probabilities, we can never be proven wrong and as a result never improve. Or as beautifully put in my favorite sentence from the book “to get better a certain type of forecasting, that is the type of forecasting you must do — over and over again, with good feedback telling you how your training is going, and a cheerful willingness to say, “Wow, I got that one wrong. I’d better think about why.””
- All Businessy Words And No Substance Makes Jack A Dull Forecaster: I don’t know whether I agree with Elon Musk fully when he said the root of all evil (in corporate America) is MBAs in leadership positions but I definitely think we are guilty of jargonizing and mistaking being verbose for being smart. “To build on that/the previous point” (I cringe as I write this), we must recognize that words have weight and meaning. We should take responsibility for the words we use and aim to be precise and intentional to create a shared understanding and make better, more informed decisions.
- Outside In: When trying to assess the probability of something happening (e.g how likely is it that Jane who has 3 kids and lives in the Bronx has a dog?) our knee jerk reaction is to try to squeeze every bit of information from the details. Instead, taking the outside view and asking how common something is within a broader class (e.g what percentage of people own pets in the US or NY?) can be more helpful as a starting point. And then you may be able to see the forest for the trees and not just pinecones and a green blob.
- Friends Who Forecast Together: May also stay together but most probably will be proven wrong together as well. As demonstrated by countless experiments on social conformity, we like to be liked and fit in. We take unanimous agreement as proxies for accuracy and validity. Unless we proactively create spaces and systems for disagreements, we might be barking at the wrong tree together.
- Stop, Gator Brain No: Most of us probably have seen the Müller — Lyer illusion a hundred times but it still takes effort to say “Stop Brain. They are the same. Same, same, same” and yes, that’s what makes illusions great. But even if we can’t turn off our tip-of-the-nose reaction, we can learn to monitor the answers that bubble up and check ourselves when we have more mental capacity.
- Owning the WIP: In software development, a program that is not intended to be released in a final version but will instead be used, analyzed and improved without end is called a perpetual beta. I like this idea, a lot. I am, you are, we all are perpetual betas. Recognizing that I am a work in progress, helps me become more patient with myself, practice self-compassion and humility and gives me strength to try again, and again to learn, to strive to be better.
- Thinker vs. Doer: When we think of leaders, we envision charismatic, decisive, go-getters and not ruminators that dabble in probabilities, but actually the divide between doers and thinkers is a false dichotomy. As Moltke put it, in life, as “in war, everything is uncertain” and leaders should have the determination to overcome challenges but also be willing to throw out their plan when need be. Finding the right balance between reflecting and doing is something I personally struggle with a lot. The impulse to act now, to be on the move, to do something is really strong but I’ve also realized with time that the more I reflect on myself, on how I feel and react to the world around me, the better I navigate murky waters and behave like my true, authentic self. And that makes a difference in how I communicate, take up space and treat others.
To sum up, though seemingly painful at first (just like writing this Medium post was), documenting your predictions in a systematic way, doing objective post-mortems and updating them frequently, will help you understand your mental models (as well as the importance of being humble😊) and push you to learn more about yourself as an employee, manager, investor or regulator but more importantly, as a human being.
Finally, to practice what I have summarized (or preached), I’d like to end this post by claiming that the likelihood of one of the authors of the book reacting to this post is between 60–65% (to be updated with incoming information)
