2026-07-26
A Game That Cannot Be Gamed — Finishing the San Francisco First Half Marathon, six months after a 10K got me hooked.
2026-06-01
Beyond Error Bars — a nine-part series on the statistics of LLM evaluation.
- Part 1Every Eval Era Created the Next Measurement Problem
- Part 2An Eval Is a Decision Instrument
- Part 3Error Bars, Applied
- Part 4How Many Examples — and When Can You Stop?
- Part 5Humans Are Instruments — Rater Ops Is the Eval
- Part 6Your LLM Judge Is a Biased, Noisy Instrument. Debias It.
- Part 7Rankings Under Uncertainty: What Arena Numbers Mean
- Part 8Agent Evals: the Statistics of pass@k and pass^k
- Part 9Building a Product Eval, Instrumented
2026-01-01
What Five Years at a Startup Taught Me — Sixteen things I believe now that I didn't believe as a new data scientist.