Tips // Claude in practice

Tips for working better with Claude

One short tip at a time, with an example you can copy. Move through them at random or search for what you need.

APIBasicClaude API

Do not test the exact text of a Claude response

The same prompt can give different wordings that are all correct; assert properties (field present, value in range), not strings.

The model samples each token from a distribution, so there is no guarantee of identical output, not even at temperature 0. A test that compares exact text will fail intermittently. Check structure and ranges in code; to judge meaning, use an eval with a model-based judge.

Example
# Instead of: assert resp == "It is a billing problem."
data = json.loads(resp)
assert "category" in data
assert data["category"] in {"BILLING", "TECHNICAL", "ESCALATION"}
#testing#evals#non-determinism