The Numbers Game: When Statistics Lie

The Seduction of Numbers

Numbers feel objective. They feel true. When someone says "studies show" or "the data proves," we tend to believe them. After all, numbers don't lie.

Except they do. Constantly.

65% of statistics in news articles contain at least one misleading element - a cherry-picked timeframe, a missing comparison, or a conflated cause.

Source: Royal Statistical Society media analysis, 2023

The problem isn't that people fabricate numbers. It's much more subtle than that. The same true data can tell completely different stories depending on how you present it, what you compare it to, and what you leave out.

Darrell Huff, statistician: "If you torture the data long enough, it will confess to anything." His 1954 book "How to Lie with Statistics" remains one of the best-selling statistics books ever written - because nothing has changed.

90% of doctors got this wrong: "If a disease affects 1 in 1,000 people, and a test is 99% accurate, what's the probability that someone who tests positive actually has the disease?" Most said 99%. The real answer is about 9%.

Source: Gerd Gigerenzer, "Risk Savvy" (2014)

In this lesson, you'll develop immunity to numerical manipulation. You'll learn to spot the tricks, understand the paradoxes, and ask the questions that reveal what numbers are really saying.

When was the last time you saw a statistic and thought "that can't be right"? What made you suspicious?

Spot the Trick: A Gallery of Lies

These are all real techniques used in business presentations, political campaigns, and news reports. For each one, try to identify the manipulation before clicking to reveal.

Market Share Comparison

This graph was used in a corporate presentation to show Brand B's "crushing victory."

Truncated Y-Axis

The Y-axis starts at 40%, not 0%. The actual difference is only 2 percentage points (40% vs 42%), but it looks like Brand B is three times larger. Starting from zero would show nearly identical bars.

Company Stock Performance

The CEO shared this chart at the annual meeting to celebrate "exceptional performance."

Cherry-Picked Timeframe

The chart starts in March - right after a major crash. If it showed the full year, you'd see the stock is actually still below its January high. By choosing when to start measuring, you can make any trend look good.

A Disturbing Connection?

This correlation was shared on social media with the caption: "What are they putting in organic food?"

Spurious Correlation

Two things increasing over time will always correlate - it doesn't mean one causes the other. Internet users also correlates with drownings in swimming pools (r=0.998). The website tylervigen.com has hundreds of these absurd-but-real correlations.

+300%

Increase in shark attacks in coastal region

"Beaches more dangerous than ever"

Danger at the Beach

This headline ran in a local newspaper, causing beach attendance to drop significantly.

Missing Base Rate

A 300% increase sounds terrifying - until you learn it went from 1 attack to 4 attacks. Meanwhile, beach visitors increased from 500,000 to 2 million. The rate per visitor actually decreased. Always ask: "300% of what?"

The Takeaway

None of these examples contain false numbers. Every statistic is technically true. The manipulation is in what's hidden, what's compared, and what's implied. Numbers need context to mean anything.

The Paradox: When Data Contradicts Itself

The Case of the Impossible Hospital

The Situation

Two hospitals in the same city treat patients for the same condition. A health researcher compares their outcomes and finds something strange:

Hospital	Patients Treated	Successful Outcomes	Success Rate
City General	1,000	710	71%
University Medical	1,000	830	83%

The conclusion seems obvious: University Medical is the better hospital. Patients should go there.

But then the researcher breaks down the data by patient condition:

Mild Cases:

Hospital	Patients	Successes	Rate
City General	100	90	90%
University Medical	800	700	87.5%

Severe Cases:

Hospital	Patients	Successes	Rate
City General	900	620	68.9%
University Medical	200	130	65%

The Puzzle

Look at the data carefully. City General has a HIGHER success rate for mild cases AND a HIGHER success rate for severe cases. Yet University Medical has a higher OVERALL success rate.

How is this mathematically possible? Which hospital is actually better?

Simpson's Paradox

This is one of the most counter-intuitive phenomena in statistics. The answer lies in the patient mix.

City General is the regional trauma centre - it receives 90% severe cases. University Medical is a general hospital - it receives 80% mild cases.

When you combine the groups, the hospital that treats more "easy" cases will always look better overall, even if it performs worse in every individual category.

The Real Answer: City General is the better hospital - it outperforms University Medical for both mild AND severe patients. But it looks worse overall because it takes on the harder cases. If you were seriously ill, you'd want City General - the exact opposite of what the aggregate data suggests.

Real-world impact: Simpson's Paradox has led to wrong conclusions in medical trials, education policy, and hiring discrimination cases. The Berkeley gender bias case (1973) is a famous example - the university appeared to discriminate against women in admissions, but department-by-department analysis showed slight bias toward women.

The Courtroom: Numbers That Convict the Innocent

The Case of the DNA Match

The Trial

A woman is murdered in a city of 5 million people. Police have no witnesses and no leads. In desperation, they run DNA from the crime scene through a database of 500,000 profiles.

They get a match: Roberto, a 45-year-old accountant with no criminal record.

The forensic expert testifies: "The probability of a random person matching this DNA profile is 1 in 100,000. There is therefore only a 1 in 100,000 chance - 0.001% - that Roberto is innocent."

Roberto has no alibi. The jury convicts.

Your Investigation

The forensic expert's statistic (1 in 100,000) is correct. But their conclusion is dangerously wrong.

What error did the expert make?
What's the actual probability that Roberto is guilty?
Why might Roberto be completely innocent?

The Prosecutor's Fallacy

The expert confused two very different questions:

What the expert said	What they should have calculated
"Probability of this DNA match if innocent"	"Probability of being guilty given this DNA match"
1 in 100,000	???

The real calculation:

Database searched: 500,000 people
Expected false matches at 1:100,000 rate: approximately 5 people
Roberto is one of those matches
Probability Roberto is the actual killer (with no other evidence): roughly 1 in 5, or 20%

The Truth: There's an 80% chance Roberto is innocent - one of several people who happened to match. The "1 in 100,000" number tells you how rare the match is, not how likely guilt is. When you search large databases, rare events happen frequently.

Real cases: This exact error contributed to wrongful convictions in the UK (the Sally Clark case), the US, and Australia. People have spent years in prison because juries - and sometimes judges - don't understand this distinction.

Test Yourself: The Probability Challenge

This problem defeats most people - including doctors, lawyers, and statisticians. Take your time.

The Scenario

A rare disease affects 1 in 1,000 people in the population.

A test for this disease is 99% accurate:

If you HAVE the disease, the test will correctly say "positive" 99% of the time
If you DON'T have the disease, the test will correctly say "negative" 99% of the time

You take the test. It comes back POSITIVE.

What's the probability you actually have the disease?

The Surprising Answer: About 9%

Let's work through it with 100,000 people:

100 people actually have the disease (1 in 1,000)
Of these 100, the test correctly identifies 99 (99% accuracy)
99,900 people don't have the disease
Of these, the test incorrectly says "positive" for 999 (1% false positive)

Total positive results: 99 + 999 = 1,098 people

True positives: 99 out of 1,098 = 9%

Why it feels wrong: Your intuition focuses on the test's 99% accuracy. But when a disease is rare, even a small false positive rate produces more false alarms than true cases. This is called base rate neglect - ignoring how common the condition is in the first place.

The Language of Statistical Skepticism

These terms will help you discuss, question, and challenge numerical claims with precision.

to cherry-pick (data/evidence)

To selectively choose only the data that supports your argument while ignoring data that contradicts it.

The company cherry-picked a three-month period where sales looked good, ignoring the overall downward trend.

a confounding variable

A hidden factor that affects both things being compared, creating a false appearance of cause and effect.

Ice cream sales and drowning deaths both increase in summer - the confounding variable is hot weather.

correlation vs causation

The difference between two things happening together (correlation) and one thing actually causing the other (causation).

There's a correlation between education and income, but that doesn't prove education causes higher earnings - other factors may be involved.

the base rate

The underlying frequency of something in the general population - essential context for interpreting statistics.

A 200% increase in rare side effects sounds alarming until you consider the base rate was 1 in a million.

to adjust for / to control for

To mathematically account for differences between groups so you can make a fair comparison.

When we adjust for age and income, the apparent health difference between the two regions disappears.

statistically significant

A result unlikely to have occurred by chance - but NOT the same as "important" or "large."

The difference was statistically significant but tiny - blood pressure dropped by 0.5 points, which has no practical medical value.

sample size / margin of error

The number of observations in a study, and the range of uncertainty in the results.

With a sample size of only 50 people and a margin of error of plus or minus 10%, these results are essentially meaningless.

to extrapolate

To extend a trend beyond the available data - often dangerously assuming the pattern will continue.

You can't extrapolate from three months of growth and predict the entire year - too many things could change.

Practice: Using vocabulary from above, explain why the "market share comparison" graph from the gallery was misleading - as if you were explaining it to a colleague.

Your Turn

Choose 2-3 topics for discussion. Try to use the vocabulary naturally.

The News Problem

Most journalists don't have statistical training. Should news organisations hire statisticians to review articles? Would it help, or would it just make reporting slower and more boring?

Lies in Your Industry

What statistical tricks or misleading numbers are common in your professional field? How do companies, competitors, or even your own organisation manipulate data?

The Honest Liar

Is presenting technically-true-but-misleading statistics lying? Where's the line between "presenting your case" and "being deceptive"? Are there times when it's acceptable?

Statistical Education

Should basic statistics be taught in schools as a core subject - as important as reading and writing? What would change in society if everyone understood base rates and margins of error?

The Expert Problem

Even doctors and judges fall for statistical fallacies. Why are humans so bad at intuitive probability? Can we ever overcome it, or should we just use computers?

Personal Decisions

Think of a significant decision you made based on numbers (buying something, choosing a job, health decision). Looking back, did you have enough information? What questions should you have asked?

Deep Dive: Extended Speaking

Choose one challenge. Speak for the full time, developing your argument with examples.

Challenge A: The Consultant

A client shows you a report full of impressive-looking statistics supporting a business decision. Walk them through what questions they should ask before trusting the numbers. Use at least 4 concepts from today's lesson.

Challenge B: The Devil's Advocate

Argue the opposite position: Why statistical literacy might be OVERRATED. When is "trusting the experts" actually the right approach? When does questioning numbers become counterproductive?

Challenge C: The Investigator

Choose a statistic you've heard recently (in news, advertising, or at work). Analyse what information is missing, what comparisons would be needed, and whether the conclusion is justified.

Challenge D: The Policy Maker

Design a simple system or checklist that organisations could use to ensure statistics in their reports aren't misleading. What rules would you include? How would you enforce them?

3:00

Wrap-Up

Today's Key Defences

1 Check the axes - Does the graph start at zero?
2 Ask about the timeframe - Why this period?
3 Demand the base rate - Percentage of what?
4 Look for confounding variables - What else could explain this?
5 Question aggregated data - Does this hide important subgroups?

The Golden Question

"What would make this number go up or down without anything real changing?"

Useful Phrases for Skepticism

Probing the data: "What's the base rate here?" / "Are we comparing like with like?"

Challenging presentation: "Why does the graph start at that point?" / "What's the margin of error?"

Questioning causation: "Couldn't there be a confounding variable?" / "Correlation isn't causation..."

Homework challenge: This week, find one statistic in the news or at work. Before accepting it, ask three questions from today's lesson.