Spiegelhalter on asking a question of data
📚SOURCE
💎PERMANENT NOTES
🧵SYNTHESIS
Paraphrase, not quotation: the introduction and chapter 1 of 📕 The Art of Statistics.
Spiegelhalter opens with a murder case, not a formula: a family doctor who killed patients for years. The questions are human ones. How many? Could it have been spotted sooner? Neither can be answered by calculation until someone has decided what to count and what to compare it with.
That is his point about the whole subject. Statistics is a cycle of five stages, which he borrows from school teaching in New Zealand: problem, plan, data, analysis, conclusion. The arithmetic everyone fears is the fourth. Most of what goes wrong goes wrong in the first three: a vague question, a careless measure, data collected for another purpose.
Chapter 1 adds that the same finding can be framed to alarm or to soothe. A survival rate and a death rate are one number.
The evidence against the doctor was a comparison: his patients against other doctors' patients.
✨KEY IDEAS
The cycle is problem, plan, data, analysis, conclusion, and then usually back to a better problem.
Turning the world into data means deciding what counts as one of the thing, and that decision is a judgement.
A death rate of 2 percent and a survival rate of 98 percent are the same fact and do not feel like it.
The doctor's guilt showed in data only when his patients' deaths were set beside those of comparable practices, by time of day.
Data are almost always an imperfect measure of what we care about.
📝READING NOTES
Why the first chapter gets a note of its own
I started the book on Tuesday as revision and expected to skim to the chapters on intervals. I have been stuck on the first thirty pages all week, because they describe the part of the work I skip.
The cycle
- Problem. What do we actually want to know, and for what decision?
- Plan. What would we need to measure, on whom, and how?
- Data. Collect it, clean it, find out how it was really gathered.
- Analysis. Tables, charts, models.
- Conclusion. What it means, how sure we are, what we do next.
He uses the cycle to organise case after case. The murders. Children's heart surgery at an English hospital in the 1990s, where the question was this unit worse turned on which operations to count and which hospitals to compare with.
My own record, held against it
| Stage | What I usually do |
|---|---|
| Problem | Can we get the numbers on triage |
| Plan | None |
| Data | Whatever Bao can export by Thursday |
| Analysis | A chart |
| Conclusion | The one I had on Monday |
That is unfair to Bao and fair to me.
The comparison is the evidence
The thread through his opening cases is that a number on its own says nothing. The doctor's death certificates were unremarkable one at a time. The pattern appeared when they were laid against other doctors'. The heart unit's results meant something only next to other units'.
This is what Adaeze keeps saying about 🏛️ Florence Nightingale: the diagrams of army deaths worked because they put disease beside wounds, month by month, and the difference in area did the persuading. Adaeze has told me to read Snow on cholera next for the same reason.
PPDAC is drawn as a neat loop. In my job the problem is handed over already phrased by someone senior, and re-opening it costs political capital the diagram does not show. He is writing as a statistician who gets consulted. I would like the version for someone who is told the conclusion first.
For the exam
📌 Statistics I - Final exam is on the 10th. Hale's long questions all have this shape: a scenario, some data, what would you conclude. I am going to write P, P, D, A, C down the margin before touching a calculation.
What it produced
💎 Evidence persuades when it compares is not written yet. I have the claim and one example. I want a second before I commit. Added 7 June: Snow supplied it.
Added 10 June: wrote the five letters in the margin of the long question before touching the numbers. It took a minute and it was worth it.
Questions it left me with
- What is the problem behind the Carrier-level numbers people keep asking me for? I have never asked.
- If the comparison is the evidence, what is the fair comparison for the Exceptions Inbox: last year, or customers without it?
- How do I re-open a question that arrived as a conclusion without it sounding like a refusal?
People in this note
- Adaeze Nwosu
- 🏛️ Florence Nightingale
🗣️IN SHORT
- In : The Art of Statistics, introduction and chapter 1
- Spiegelhalter argues that : statistical work is a cycle that starts from a problem, and that the stages before the analysis decide whether the analysis can mean anything
- by stating that : definitions, measurement and the choice of comparison are judgements made before any calculation, and that the same result can be framed to opposite effect