Extreme results drift back toward the average on their own
💎CORE ARGUMENT
Any result is part signal and part luck. An extreme result is usually one where the luck ran strongly in one direction, and luck does not repeat on request. So the next measurement tends to sit closer to the average, with no cause required. Nothing happened. The dice were rolled again.
This would be a curiosity if we did not respond to extremes. But extremes are exactly when we act, and whatever we did then gets the credit or the blame for a movement that was coming anyway.
Three cases I keep.
- The carrier with one terrible month. It is called in, warned and put on a plan. The next month is better, and the plan is declared a success. A carrier warned by nobody would probably have improved too, because a worst month is rarely followed by a worse one.
- The athlete on the magazine cover. They get the cover after the best year of their career, and the following year is ordinary. People call it a curse. It is arithmetic.
- My own best week. The last week of August scored 8.4, by a lake, on holiday. The week after scored 7.0, and I spent an evening wondering what I had done wrong. I had come home.
The smaller the sample, the wilder the swings. A carrier with six loads can rank top or bottom on chance alone, which is what makes a naive scorecard unfair to small carriers in particular: they will fill both ends of the table and deserve neither.
The check I use: before explaining a change, ask what I would have expected had I done nothing. If the answer is about this, I have no finding.
Where I might be wrong. It assumes a fixed average to return to. If the carrier's real level is falling, last month's disaster is the new centre and wait and see is negligence.
I also like this idea a little too much at the moment. It arrived in the first autumn lecture, I have applied it to everything since, and a tool that explains every change as no change is not far from one that explains nothing.
🧱BUILT FROM
🌍WHY IT MATTERS
- I do not praise or blame on one extreme reading, of a carrier, a colleague or a week of mine.
- Any before-and-after claim at work now has to say what the after would have been untouched.
- I read my weekly scores in runs of four.
Tested on: Scorecard prototype data.
☑️NEXT STEPS
- Find how many carriers in the prototype data have fewer than ten loads
- Write the fairness note on small carriers
- Chart my weekly scores as four-week averages
The chapter where this first appeared for me, with its flight instructors who concluded that praise harms and shouting helps.
The average being drifted towards is the base rate in 💎 A number without its base rate is a story. Showing how far chance alone could move a score is the job of 💎 An interval is more honest than a point.
An observed value is a stable component plus chance. Selecting cases because they are extreme selects for extreme chance, which is not carried into the next observation. The stable component remains, so the expected next value lies between the extreme and the mean.
Some extremes are real. A carrier can truly have got worse, and a manager who answers every bad month with it will regress will be right most of the time and badly wrong when it matters.
The fairness note I owe on small carriers, the objection that prompted it, and a longer question about what a ranking owes the people in it.