Huff on charts that lie
📚SOURCE
💎PERMANENT NOTES
🧵SYNTHESIS
Paraphrase, not quotation: the whole of 📕 How to Lie with Statistics, ten short chapters.
Huff, a journalist and not a statistician, wrote a field guide to the ways a true number can leave a false impression. Each chapter is one trick. The sample that selected itself. The average chosen from mean, median and mode according to which flatters. The figure given with no range and no sample size. The difference too small to matter, reported as news. The graph with the bottom cut off so that a ripple looks like a cliff. The picture whose height doubles and whose area therefore quadruples. The number that is accurate and about something else. And the two things that rose together, presented as cause and effect.
He ends by turning it round: five questions for talking back to a statistic.
It is seventy years old, breezy, crude in places, and I recognised a slide of my own in chapter 5.
✨KEY IDEAS
Cut the bottom off the vertical axis and stretch what remains, and a ten percent rise fills the page.
An unqualified average could be any of three numbers, and the writer has usually picked.
The interesting facts are often the ones left out: how many were asked, how much the results varied, compared with what.
If you cannot prove what you want, prove something near it and let the reader make the jump.
Before accepting a figure, ask who is saying it, how they know, what is missing, whether the subject has been changed, and whether it makes sense.
📝READING NOTES
Read where, and how
Easter weekend in Averley, finished on Good Friday, written up this afternoon at Mum's kitchen table after helping at her reading morning. It is on the course list as light relief. It took three sittings and one of them was a bath.
The catalogue
| Chapter | The trick | Where I have seen it |
|---|---|---|
| 1 | A sample that chose itself | Survey of customers who answer surveys |
| 2 | The convenient average | Mean resolution time, dragged by six monsters |
| 3 | The missing figures | Up 30 percent, from what, among how many |
| 4 | A difference inside the noise | Two teams ranked on a gap of nothing |
| 5 | The truncated axis | My own slide |
| 6 | Pictures scaled in two dimensions | Any infographic with growing icons |
| 7 | The nearly relevant number | Activity reported as outcome |
| 8 | Correlation presented as cause | Customers who use the feature churn less |
| 9 | All of the above, on purpose | Rare, in my experience |
| 10 | How to talk back | The useful chapter |
What I take from it
A chart is a claim someone is making, and it has an author with a purpose even when the author is me on a Thursday. Once I think of it that way, the right response to a chart is the response to any argument: what is it asking me to believe, and what would I need to see to disbelieve it.
I rewrote his five questions as my own in 🧭 Five questions to ask a chart: what is the axis, what is missing, compared with what, who made it, what would change it. His does it make sense I dropped because it flatters the person asking. His who says so I kept.
What I do not take
Huff assumes the liar is always someone else: an advertiser, a politician. Most misleading charts I meet are made by honest, hurried people using the default settings. He has no chapter on that. And it is worth knowing that he later took the tobacco industry's money to cast doubt on the evidence about smoking, using this same toolkit. The method cuts both ways. Scepticism without a standard of proof is just a way of not believing things you dislike.
It is also dated in the manner of a 1954 American paperback: the jokes, the cartoons, the assumptions about who reads statistics. I winced twice.
What it produced
💎 A chart is an argument and can be cross-examined, written here before lunch.
Added 24 April: 📝 Issue 38 - The chart that won the meeting went out this morning. It is this note with my truncated axis as the opening story.
Related notes
- 🧭 Five questions to ask a chart
- 📝 Issue 38 - The chart that won the meeting
- 📖 Spiegelhalter on uncertainty
🗣️IN SHORT
- In : How to Lie with Statistics (1954), all ten chapters
- Huff argues that : statistics mislead less through false numbers than through selection, presentation and omission, and that a reader can learn to question them
- by stating that : biased samples, chosen averages, absent comparisons, distorted graphs and misplaced causation each have a recognisable signature, and a short set of questions exposes most of them