Experienced experts look at exactly the same case and give wildly different answers. A Nobel Prize-winning psychologist and his co-authors name the hidden flaw in human judgment, and show how to fix it.
Ten words. Work through the four activities. Say your answer out loud before you reveal it.
Three words: noise · intuition · consistent. Say the right one, then reveal.
Read each part. After it, there's a quick check (answer from memory first), then one question to discuss.
Imagine you work for a large insurance company. Customers ask you for a price to insure their business. The price is called a premium, and the people who decide it are called underwriters. Underwriters are trained experts. Their job is to look at a case and decide how risky it is.
Now imagine that two of them look at exactly the same case. How different would their prices be? A psychologist called Daniel Kahneman put this question to the managers of one big financial company. Most of them guessed about 10%. A small difference, they thought, is normal.
Then Kahneman's team did a test. They gave the same cases to a large group of underwriters and asked each one to name a price. The real difference was not 10%. It was about 55%. If one underwriter quoted $9,500, another quoted $16,700, for the same case.
The managers were shocked. Nobody had ever measured it before. Every underwriter believed they were being sensible. But for the customers, the price was a lottery. It depended on which person happened to pick up the file. And for the company, it was expensive: prices that are too low lose money, and prices that are too high lose customers.
1. What did the managers expect, and what did the test find?
2. Why was the result a problem for both the customers and the company?
1. They expected a difference of about 10% between two underwriters. The test found about 55%.
2. Customers got a lottery: their price depended on who took the case. The company lost money on prices that were too low and lost customers on prices that were too high.
Daniel Kahneman was born in 1934. He grew up in France and later moved to Israel. As a young man in the 1950s, he served in the army as a psychologist. One of his jobs was to help choose which recruits should become officers. The method was an interview: a short conversation, then a score.
Kahneman noticed something strange. The interviewers felt very sure about their scores. They thought they could "see" who would be a good officer. But when Kahneman compared the scores with how the recruits actually performed later, the predictions were only a little better than guessing. Even so, the interviewers stayed confident. Kahneman later called this the illusion of validity: the feeling of knowing something, when you don't.
So he changed the method. Interviewers had to score six different qualities, one at a time, using the same questions for everyone. Only at the end could they add a general impression. The new scores were much better at predicting who would succeed.
Kahneman went on to become one of the most famous psychologists in the world. With his colleague Amos Tversky, he showed how our thinking is full of mental shortcuts that push us in a particular direction. Those shortcuts are biases. In 2002 he won the Nobel Prize in Economics, even though he was a psychologist. In 2011 he wrote the bestseller Thinking, Fast and Slow. But he never forgot the problem he had seen in the army and in the insurance company. It was not bias. It was something else.
1. What did Kahneman notice about the army interviewers?
2. How did he change the interview method?
1. They felt very sure about their scores, but their predictions were only slightly better than guessing. And they stayed confident anyway.
2. Interviewers scored six separate qualities one by one, with the same questions for every recruit, and gave an overall impression only at the end.
In 2021 Kahneman published Noise with two co-authors: Olivier Sibony, a French professor of business strategy, and Cass Sunstein, an American law professor. The book's main idea is simple. There are two different kinds of error in human judgment.
Think about a bathroom scale. Suppose it always shows two kilos too much. That's bias: the same error, in the same direction, every time. Now suppose a different scale shows 61 kilos, then 58, then 64, for the same person, in one minute. That's noise: random differences that have no good reason. Everyone talks about bias. Very few people notice noise, because you can only see it when you compare several judgments of the same case.
And there is noise almost everywhere. Studies found it in the sentences of judges who read identical case files, and in the diagnoses of doctors who looked at identical scans. It appears in job interviews, in asylum decisions and in forecasts. Even one person can be inconsistent: the same expert can give a different answer to the same question on a different day.
The authors' solution has a strange name: decision hygiene. Like washing your hands, it doesn't fix one particular problem. It stops many problems you can't see. The idea is to design the way we decide, so that it depends less on who is deciding, and on what kind of day they're having.
Five ideas from the book. Read each idea. Can you think of an example from real life? Say it, then reveal one.
1. What's the difference between bias and noise? Use the bathroom scale.
2. Why is noise so hard to notice?
1. A scale that always shows two kilos too much has bias: the same error in the same direction. A scale that gives a different number each time has noise: random differences with no good reason.
2. You only see it when several people judge the same case and you compare their answers. A single judgment always looks reasonable.
The whole book is built on one contrast: bias versus noise. English has many ways to set two ideas against each other. Each one needs a different grammar.
Choose a, b or c. Say the whole sentence aloud, then reveal.
Each sentence has one grammar mistake with a contrast word. Say the correct sentence, then reveal.
Finish each sentence out loud with your own idea. There is no single right answer.
1. Rules and checklists are consistent, whereas …
2. Unlike a machine, a human judge …
3. Despite having years of experience, …
4. Many people trust their intuition. In contrast, …
5. Although an average of many opinions is usually better, …
Take these structures into the Speak tab. You'll compare ideas in almost every task.
The authors use a picture of four teams shooting at a target. Each red dot is one shot. The centre is the truth. Look at the four targets. What is going wrong for each team?
For each situation, decide: bias or noise? Say it, then reveal.
All of it out loud, against the clock. Use the key words, the five ideas, and the language of contrast.
Tell the story: the insurance test → the army interviews → bias and noise → the four targets → decision hygiene.
Then be Kahneman in a meeting with the company's managers. They believe their underwriters are consistent. Explain what your test found, why it matters, and what you would do first. Use at least five key words (bias, audit, sentence, noise, intuition, consistent, guideline, hygiene, judgment, error).
Choose three of these pairs. For each, give a two-sentence comparison using a different contrast word each time (whereas, unlike, in contrast, although, despite, however).
Think of a judgment that people make about the same thing: marking an essay, judging a talent show, choosing a restaurant, refereeing a football match, or deciding who gets a job. How much noise do you think there is? How would you measure it? Which two of the five ideas would you use to reduce it, and why?
Prefer something different? Take one of these:
I can… explain the difference between bias and noise, and compare ideas clearly using contrast language.