An agent says it created a support ticket. Its final message includes a plausible title and a confident confirmation. The ticket system contains no new ticket. If the evaluation grades only the reply, it can reward the precise failure the product needs to prevent.
Check the result in the target system
Use a synthetic ticket queue for this example. Define success as one ticket with the expected project, requester, category and description. A ticket in the wrong project is a failure. Two correct-looking tickets are a failure if the task authorized only one. A well-written confirmation cannot repair either result.
Anthropic’s guide to evaluating agents distinguishes the interaction record from the actual outcome. It also separates a task, repeated trials and the graders that assess performance. That vocabulary helps keep the evaluation focused on what happened beyond the conversation.
Match checks to requirements
Some requirements have direct checks: the ticket exists, its project matches, the restricted field was not changed and no duplicate was created. Other requirements need judgment, such as whether the description preserves the customer’s meaning. Keep those assessments separate so a strong writing score cannot hide a failed permission check.
For an assistant that only drafts tickets, the expected outcome is different. Verify the saved draft and the review state. The evaluator must not penalize the agent for refusing to submit something it was never authorized to submit. Define the allowed endpoint before measuring completion.
Include difficult and incomplete cases
A happy path shows that the components can cooperate. Add cases that reveal how the agent handles incomplete evidence and uncertain actions. Run repeated trials where behavior can vary, and retain enough trace information to explain a failure without collecting unnecessary private content.
- The tool rejects the write: the agent must not claim success.
- The tool times out: establish the operation state before repeating it.
- The requested project is inaccessible: preserve the denial.
- The user leaves out a required detail: ask or produce a clearly incomplete draft.
Use findings to improve the system
If failures come from an ambiguous tool result, rewriting the prompt may only disguise the problem. If the tool is clear but the agent ignores a denial, inspect the orchestration and confirmation rules. If the draft is accurate but unusable, revisit the product’s definition of a useful output.
A good evaluation narrows the next decision. Keep a small set of representative tasks that expose different failure causes, then expand it from observed gaps. The aim is a feature whose completion claim can be checked against reality.