Gold Set Design
Train, dev, test — and why tweaking prompts against your test set quietly lies to you.
Why this matters
A is the carefully labelled data you trust to tell you how good your system is. With LLMs there's no training loop to overfit — so it's tempting to think overfitting can't happen. It can: every time you tweak a prompt because the test score went up, the test set becomes part of your design process.
After 30 tweaks evaluated on the same 150 calls, the best score is inflated by selection (the winner's curse) and by rules that fit those specific calls. The only defence is discipline: splits, a test set touched once, and fresh samples to re-check.
Core concept
Technical explanation + simple intuition + example
Three splits, three jobs
- Train: where you pick few-shot examples and get rule ideas. - Dev: where you iterate — as often as you like. - Test: touched once, at the end, to estimate how the system will do on new data. If you iterate on the test set, you no longer have a test set.
Prompt overfitting and the winner's curse
Every evaluation score is the true quality plus some luck. If you try 20 prompts and keep the best-looking one, you've probably kept one that was also lucky on this particular sample. Its score on new data will be lower.
The 10 questions
Every important concept should answer all of these
1What is it?
The labelled datasets used to build, tune and finally evaluate a system, split so that the final estimate is unbiased.
2Why does it exist?
Reported numbers are only meaningful if they come from data that didn't influence design decisions.
3What problem does it solve?
It prevents optimistic, non-reproducible performance claims.
4How does it work internally?
Split into train/dev/test (stratified, disjoint), iterate on dev, evaluate once on test, refresh periodically.
5What does it look like on real transcript data?
Calls from the same customer or agent can leak patterns across splits; traffic drifts with campaigns and seasons.
6What can go wrong?
Iterating on test, leakage (same customer in train and test), few-shot examples drawn from test, stale gold sets, too-small test sets.
7How do I evaluate it?
Compare dev vs test vs fresh-sample scores; a large drop signals overfitting or drift.
8When should I use it?
From the first prompt iteration.
9When should I NOT use it?
Never skip; for tiny projects at least keep a small untouched test set.
10How does it affect downstream systems?
Every go/no-go decision, model comparison and stakeholder report depends on it.
Interactive lab: The prompt-iteration trap
Try prompt tweaks and keep the ones that raise your score — tuning on the TEST set or the DEV set. Then ship and see the hidden production sample. Watch the gap open when you tune on test.
Try it yourself
Conceptual (A), applied (B) and engineering (C) exercises
Graded by a transparent keyword rubric (not an LLM) — it checks for the key ideas, then shows a model answer for self-assessment.
Real-world application
A team reported 91% accuracy to leadership after three weeks of prompt iteration on a 120-call "test" set. A fresh sample of 500 calls a month later scored 83%. Most of the gap wasn't drift: ~20 rules in the prompt referenced patterns seen in specific test calls. They rebuilt the process: 300-call dev set for iteration, 300-call test set held by the QA team, evaluated only at release.
Common mistakes
Selection and item-specific rules inflate the score.
Fix: Iterate on dev; test once.
Direct leakage.
Fix: Examples from train only.
Near-duplicates leak across splits.
Fix: Split at the call (or customer) level.
Traffic drifts; the set stops representing production.
Fix: Add fresh, stratified samples periodically.
Engineering decision
How big and how split should your gold set be?
- Test ≥ 300 calls for ±~4–5 pt CIs on accuracy; more for rare-class metrics
- Stratified by language, quality and intent
- Hidden test set for high-stakes releases
- Reporting dev-set numbers as expected production performance
Quiz
Pass mark 60%. Missed questions go to your mistake log and spaced review.