Start from real cases
The best test set is 200 to 300 examples of the work the agent will actually do, with the right answer written down by the person who does it today.
Test the behaviour, not the wording
We check outcomes: the right customer matched, the amount within 1%, no email sent without approval, the source invoice cited.
Run on every change
Prompts, models and tools change. The eval suite runs in CI on every pull request, and accuracy is reported weekly after launch.
