TOOL
Sample size calculator script
🛠️ Sample size calculator script
💡 Use Case
- The A-B test sample size sheet rewritten as a short script, so that the four assumptions sit in one place, at the top, with their reasons written next to them.
- Written two weeks after the triage test was designed, when I found I had changed the power on one tab of the sheet and not on the other.
- It implements 📐 Sample size for comparing two proportions and prints a small table: the sample per group for the difference I care about, and for half and double that difference.
- The half and double rows are the lesson of 🎓 Lecture 9 - A-B tests and their traps: halve the effect you hope for and the sample you need roughly quadruples.
A practice run, not the real test
Baseline 40 percent, smallest difference worth having 5 points, significance 0.05, power 0.80: about 1,530 per group. Halve the difference to 2.5 points and it is over 6,000.
⚖️ Pros & Cons
Pros
- Assumptions first. Anyone can read lines 1 to 8 and disagree with me before any arithmetic happens.
- The three-row table ends the conversation that starts can we not just run it for a week.
- It is saved with each test, so the sizing is part of the record.
Cons
- It assumes every dispatcher is independent of every other. Dispatchers share an office and a list.
- It answers how many, not how long. How long depends on how many dispatchers a customer has, and for some customers the answer is never.
🔄 Alternatives
- The spreadsheet, for anyone who will not open a script.
- An online calculator. Fine for the number, useless as a record of what was assumed.