APIIntermediateClaude API
Evals: code for the objective, a judge for the interpretive
Validating format or data against a source is solved with automatic checks; tone or reasoning quality needs a model-based judge.
Build the eval before the production code: define the task and the success criterion, a golden dataset with edge cases, automatic checks for the unambiguous (valid JSON against the schema) and a judge for the interpretive. Also look at the breakdown by category: a rising average can hide a drop on edge cases.
Example
Examples of the split: - Code: is the output valid JSON according to the schema? - Code: does the amount match the database? - Judge: does the tone respect the style guide? - Judge: is the reasoning consistent with the evidence?