companies, in search of the common denominators of corporate
culture and management practices that led some companies to
succeed where others fail. But as Kahneman points out, Collins and Porras did not account for the possibility that at least
part of the (lack of ) success of the companies they selected to
analyze was due to luck. A decade after the book’s appearance,
a Fast Company article reported that half of the high performers had “slipped dramatically in performance and reputation…
[and] none of them today would meet B TL’s criteria for visionary companies.” Even without access to Collins’ data, Harold
Hotelling most likely would have predicted as much.
Kind of Blue
The regression fallacy is but one instance of the genre of cognitive
biases involving failures of probabilistic reasoning. Another instance, base rate neglect, will be familiar to anyone who has worked
in statistical fraud detection. Suppose 1 percent of bodily injury
claims are fraudulent, and you have built a machine learning classifier with a 95 percent accuracy rate. Now suppose that a newly
reported claim has been flagged as “fraudulent” by your classifier.
What is the probability—given your classifier’s estimation—that
the claim actually is fraudulent? Although intuition might suggest an answer close to 95 percent, the actual answer is closer to 16
percent. Essentially the same problem bedevils medical decision-making, in which rare diseases must be diagnosed by procedures
and practitioners that inevitably produce false positives. Not taking
the (low) base rate into account leads to a dramatic overestimate of
the probability that the disease is present. (See “Enhanced Credibility: Actuarial Science and the Renaissance of Bayesian Thinking,”
September/October 2010 Contingencies, for more details.)
The fact that people have difficulty with these facts indicates
that the human mind is poorly wired for coherently using probabilities to make judgments and decisions involving uncertainty.
This has nothing to do with individual intelligence and everything
to do with systematic biases in human cognition. For example,
the legendary Hungarian mathematician Paul Erdős—who published more papers than any other mathematician in history—had
to view a computer simulation model before he was convinced
that the correct move in the Monty Hall problem was to switch
doors. (See “Analyzing Analytics: The Debate Between Intuition
and Institutional Thinking,” July/August 2008 Contingencies.)
It is remarkable enough that our minds did not evolve to
instinctively follow the laws of inductive logic that we need
to make everyday decisions. But more can be said. Consider
Kahneman’s story of the green and blue taxis:
A cab was involved in a hit-and-run accident at night.
Two cab companies, the Green and the Blue, operate in the city.
You are given the following data:
■ ■ Eighty-five percent of the cabs in the city are Green and
15 percent are Blue.
■ ■ A witness identified the cab as Blue. The court tested the
reliability of the witness under the circumstances that
existed on the night of the accident and concluded that the
witness correctly identified each one of the two colors
80 percent of the time and failed 20 percent of the time.
What is the probability that the cab involved in the accident
is Blue rather than Green?
As with the diagnostic problems involving insurance fraud
and rare diseases, this is a garden-variety problem in Bayesian
inference. The common answer is 80 percent, but an application of Bayes’ Theorem reveals the correct answer to be 41
percent.
Kahneman next poses an alternate form of the same story:
You are given the following data:
■ ■ The two companies operate the same number of cabs,
but Green cabs are involved in 85 percent of the accidents.
■ ■ The information about the witness is as in the
previous version.
What is the probability that the cab involved in the accident
as Blue rather than Green?
The two versions are mathematically identical. Yet Kahneman and Tversky found that, in contrast with those who were
presented with the original version of the problem, people who
saw this alternate version did in fact give significant weight to
the base rate ( 15 percent). Indeed, their average judgment was
not too far from the Bayesian solution.
Kahneman’s lesson is that “causes trump statistics” in the
human mind. We reliably go astray when dealing with facts
about abstract statistical ensembles (85 percent of the taxis
