Agent evaluation: measuring what an AI agent actually gets done
Agent evaluation covers the methods used to check whether an AI agent really completes a task, not just whether it produces a plausible answer. It looks at the final result, the path taken, consistency from one run to the next, cost and safety.
Data as of October 22, 2026
Key figures
| Level | Advanced |
|---|
Why agents are harder to evaluate
A question-and-answer benchmark compares a piece of text with a reference answer. An AI agent takes actions: it searches, calls tools, edits files. Two agents can reach the same result by very different routes, and the same agent can pass and then fail on the same task. Evaluation therefore has to look at the environment’s end state, the execution trace, consistency, cost and unwanted side effects.
The core vocabulary
In its guide “Demystifying evals for AI agents” (January 2026), Anthropic sets out vocabulary that has become common:
- Task: a test with defined inputs and a success criterion.
- Trial: one attempt; you run several because results vary.
- Grader: the logic that scores. It can be code (fast and objective), a model (LLM-as-a-judge, flexible but needs calibration) or a human (the gold standard, but slow and expensive).
- Transcript: the full record of a trial, tool calls included.
- Outcome: the final state of the environment.
The same guide separates two measures: the chance of succeeding at least once across several trials, and the chance of succeeding every time. The first captures potential; the second captures the reliability you can expect in production.
A concrete example
A team is testing an agent that changes travel bookings. Each task defines an expected database state (“the flight moves to the next day with no extra fee”). A script checks that state; a judge model checks that the message to the customer strikes the right tone; a human reviews a sample. Each task is replayed several times to measure consistency.
Public benchmarks and their limits
Benchmarks such as AgentBench (Liu et al., 2023), SWE-bench for code and Terminal-Bench for the command line give everyone shared reference points. But each measures one specific setting and can be contaminated by training data. Our methodology explains how the site displays these scores.
Common myths
- “A strong public score means a strong agent for me.” Only your own tasks measure your own use case.
- “The overall score is all you need.” Reading failure transcripts often teaches more than the average, and LLM observability carries that work into production.
Related terms
- Benchmark · A standardised test, a fixed set of scored questions or tasks, used to compare how different AI models perform.
- LLM-as-a-judge · An evaluation method in which a language model scores or compares another model's answers against a rubric, standing in for a human grader.
- SWE-bench · A benchmark that asks an AI to resolve real GitHub issues from Python projects, with each fix checked against the project's own tests.
- Terminal-Bench · A benchmark that measures whether an AI agent can complete hard, realistic tasks on its own inside a command-line terminal.
- LLM observability · The traces, metrics and logs that show what an AI application is doing: prompts, responses, tools called, cost and latency.
Full glossary
Frequently asked questions
How is this different from a regular benchmark?
A regular benchmark asks a question and compares the answer with a reference. An agent acts over several steps in an environment, so you need to check the end state (is the file fixed, does the booking exist?) and often review the path, not just the final message.
Can another AI grade my agent?
Yes, that is the LLM-as-a-judge approach, useful for open-ended tasks such as writing. It needs calibrating against human judgments because the judge model has its own biases. Code-based checks are still better whenever the result can be verified objectively.
Where should I start with my own agent?
With a small set of realistic tasks drawn from failures you have actually seen, each with a success criterion two experts would agree on. Run them after every change of prompt, tool or model, and read the transcripts of the failures.
Why run the same task several times?
Because agents are not deterministic: one may succeed once and fail the next time. Measuring success across several trials separates an agent that can do it “sometimes” from one that does it “every time”.
Data sources
Data as of October 22, 2026
Report an error
Does a figure, a date or a fact look wrong? Tell us: every report is checked.