Hi Tip-Sheeters,

One thing that led me toward a career in data science was reading popular statistics books.

Books like The Signal and the Noise by Nate Silver and Naked Statistics by Charles Wheelan do a fantastic job of explaining in real-life terms how concepts like the law of large numbers or multivariate linear regression do a lot of real work in our lives.

I wrote a Medium article a few years ago, reviewing some of my favorites, and you can still read it here: A Roundup of My Favorite Stats Books.

Underlying Abilities and Regression Toward the Mean

A few of the ideas from these books have challenged me intellectually and altered my perception of time and reality. That sounds overly dramatic, but it's not far from the truth.

The first was from Gary Smith's book Standard Deviations: Flawed Assumptions, Tortured Data, and Other Ways to Lie with Statistics. He had a chapter on regression toward the mean -- the tendency of one-time occurrences far above or below a group's mean to be followed by an occurrence less extreme.

For systems that included randomness, a really high or low result is on average likely to be followed by one less extreme (closer to the mean).

For example, a web layout that shows much higher click-through in experiments is likely to show more modest benefits when it rolls to production because the extreme performance was likely over-performing based on random noise.

In this chapter, he used the term "ability" to represent the true underlying state of something if you could remove all randomness.

So a student's ability is what they would score on a test that had all possible questions.

A baseball hitter's ability is how they would hit every possible pitch from every possible pitcher.

A web layout's ability is how it would perform if all possible users viewed that page in all possible situations.

He summarizes the concept with the established phrase: results fluctuate around underlying ability.

In situations where we're trying to find out the ability of something, we're left with the frustrating reality that we probably can't find that out -- we're left only with (incomplete) observed results.

The most mind-blowing example Smith uses: a person's height. A person's height is influenced by the genetics they inherited from their parents, but imperfectly. Their real-world height is just one possible result of their genetics.

In these cases, Smith argues that reality itself is filled only with observed instances of a hidden true ability.

Taleb's Invisible Histories

Nassim Nicholas Taleb takes this concept further in his book Fooled By Randomness: The Hidden Role of Chance in Life and in the Markets.

Taleb has spent numerous hours running financial or other events through a Monte Carlo simulation to model potential outcomes of a process that includes randomness (which is most of them). In a Monte Carlo simulation, he can observe that processes produce some outcomes more frequently than others through multiple simulations. These repeated outcomes can be said to have a higher likelihood.

The outcome of this repeated practice is that Taleb views reality itself as an imperfect reflection of the underlying processes driving it.

I want to make a key distinction here: he's not proposing simply that everything in life is random and unknowable. He does believe that underlying processes exist for finances, or a career, or a proposed business venture. But those processes have the potential to generate many different outcomes -- but you'll only observe one of them in real life.

Invisible histories

If events in life are outcomes of Monte Carlo-like processes, then along with the single realized outcome, there exist for Taleb all the other potential outcomes. And many of these were more likely than the one observed.

A concrete example: let's say one person spends a year making risky bets in options trading, and ends up making a fortune at the end of that year. From Taleb's perspective, that fortune was just one possible outcome of the choices made -- and one of the less likely ones. More likely outcomes might have been much smaller returns or even massive losses.

Taleb argues that when we commonly talk about situations like this, we don't put enough weight on the more likely outcomes. He calls these other possible invisible histories to give more substance to them. Crucially, he doesn't view these invisible histories as meaningless. He has come to see them as real -- even if not realized.

Alternative Accounting

To put some substance on his argument, he proposes a system he calls Alternative Accounting. This system would evaluate realized outcomes in view of their likelihood. Going back to the options trader, he argues that an unlikely fortune made through risky options bets should be devalued by its unlikeliness.

He compares it to a fortune made by dentistry. The dentist's fortune should be valued higher, because it was a more likely outcome of the process used to achieve it.

So what does it all mean?

The popular stats books are always fun for me, especially when they alter how I view decision-making.

Although I find Taleb's arguments on this topic pretty convincing, I haven't completely been able view invisible histories equally to real-world events (yet).

But I am just nerdy enough to adjust the emotional side of decision-making at times based on the underlying probabilities.

📚 Have any good books to recommend? Hit reply and let me know -- I'm always looking for my next great book.

Keep coding,

Ryan Day

👉 https://tips.handsonapibook.com/ -- no spam, just a short email every week or so.