The Dangers of Putting Our Faith in Statistics

Most people try to avoid thinking about statistics, but they’re part of everything we do in investing. When other people get around to thinking about statistics, they often end up thinking about them the wrong way.

Most people assume the most important numbers of any data set are the average, median, and standard deviation, and they can safely ignore everything else. But it’s not that simple.

Statistics are about trying to figure out the unknown based on what we have seen in the past. It’s all about working with uncertainty. We can’t know what the true center of the stock market’s distribution of returns is, but we can make an estimate.

With investing, we always need to be careful about our level of certainty, even when the statistics seem rock solid. The financial markets are noisy places, and we need to appreciate that.

One of my favorite examples of this is to ask someone which they would expect to be larger next year – the size or value premium for US stocks (admittedly, there’s a pretty small group you can do this with). This seems like it should be really easy – from 1927 to 2016, the average annual size premium was 2.80 percent, and the average annual value premium was 4.98 percent*. Yes, they both have pretty big standard deviations, but look at the difference in the average – it’s pretty clear what the answer is, right?

Not so fast – that variance matters. We actually can’t really tell the difference between the two premiums. There’s no statistically significant difference between these two averages. To be precise, the p-value (a measure of how confident we are in a statistical result) for the difference in the annual average returns was 0.25.

In other words, there was a 1 in 4 chance that this difference is purely the result of random noise. To be considered statistically significant, we want there to be less than a 1 in 20 chance that it’s random.

This matters from an investment standpoint because it will affect how you divvy up your risk budget and what your asset allocation looks like. But the bigger and more important point is that you constantly need to be thinking in terms of uncertainty.

We talk a lot about thinking through risk, but it’s important to recognize that everything – including our descriptive statistics – is uncertain.

So what does this mean for you?

It’s one thing to play with numbers and point out interesting relationships. It’s another to actually do something useful with them.

But there’s a big takeaway from this uncertainty: we should be incredibly wary of overconfidence.

Overconfidence is one of the most destructive behavioral biases investors have. We all think we are better than we are – statistically speaking, almost everyone reading this article thinks they are a better than average driver.

In fact, there’s really no such thing as under-confidence. Even people who self-identify as pessimistic are still overconfident, just by slightly less than the average person.

Even when we are primed to think about the overconfidence of investors – or ourselves, specifically –pretty much everyone still acts overconfident. I am overconfident. You are overconfident. Your boss is really overconfident (and that goes double if you’re your own boss).

My mother-in-law used to call this America’s superpower. We just assume we know how to do everything – or at least we’ll figure it out as we go along. But overconfidence is universal – it may vary a little in degree across cultures, but it doesn’t go away.

So we need to be hypervigilant about minimizing the effects of overconfidence. Simply relying on a passive investment philosophy doesn’t cut it. It’s certainly better than trying to beat the markets, but it doesn’t eliminate the dangers of overconfidence, it just changes what we are overconfident about.

The biggest area where passive investors are overconfident (we’ll ignore just how overconfident this statement probably is) is probably our assessment of our own risk tolerance – especially men. We think we understand the risk and return relationships of the market (and we’ve already seen that’s not the case), so we think we can tough out those periods when the markets turn against us.

This is better than trying to outguess where the markets are going, but it’s still not good. Being overconfident about the level of investment risk that you can tolerate just straps you right back into the fear and greed roller coaster that we’re all trying to get off of.

Investing is about managing risk. And a huge part of doing this well is being humble. Humility allows us to recognize that however smart we are (or think we are), the markets are smarter.

Statistics allow us to understand certain relationships, and we become pretty comfortable making certain statements about why things happen. But we still regularly have to throw up our hands and say the markets are just weird sometimes.

It’s really easy to overextend and think we understand everything that is happening – especially when we have something as seemingly reliable as statistics on our side – but behavioral biases like overconfidence can prevent you from reaching the retirement you’ve always wanted.