Evals before demos
The demo went well. The model answered eight rehearsed questions, the room nodded, and the pilot got funded. Six months later the same system is quietly switched off, and the postmortem says the model “wasn’t reliable enough.”
That’s rarely true. What was missing wasn’t reliability. It was a definition.
The real failure mode
Every AI system is probabilistic somewhere. The mistake isn’t accepting that — it’s not deciding where the probability is allowed to live. When teams skip that decision, they end up with a system where nobody can answer three basic questions:
- What does a correct answer look like, concretely?
- How often does the system produce one?
- When it fails, how does it fail, and does anyone find out?
Without answers, “reliable enough” becomes a feeling. And feelings don’t survive contact with a CFO.
Build the harness first
An evaluation harness is just a test suite for behaviour that isn’t binary. At minimum it needs:
- A golden set — real inputs with known-good outputs, drawn from actual usage, not synthetic examples.
- A scoring rule — exact match where you can, rubric or model-graded judgement where you can’t.
- A baseline — today’s number, so every change has a before and after.
- A gate — the score runs in CI, and a regression blocks the merge.
None of this is exotic. It’s the same discipline that made software deployment boring, applied to a component that happens to be non-deterministic.
Why it has to come first
Retrofitting evals is expensive because by then the architecture has calcified around vibes. Prompts have grown to paper over failures nobody recorded. Nobody remembers which edge cases were “known issues” and which were never seen.
When the harness exists first, every one of those decisions has a place to live. Prompt changes become measurable. Model swaps become a diff. And the question “should we ship this?” has an answer that isn’t an anecdote.
Determinism, by design. Put the randomness where it belongs — and measure everything around it.
The uncomfortable part
This is not the fun work. There’s no demo of a test suite. But it is the difference between an AI initiative that compounds and one that gets quietly retired — and it’s cheap when it’s done before the first line of prompt code.
Start with the harness. The demo can wait a week.