Evaluating AI Agents: Final Answer vs Trajectory vs Per-Turn

Most teams evaluate their agent the way they evaluated their chatbot: run a set of inputs, check the outputs. It is necessary and it is not enough. An agent produces a trajectory, a sequence of reasoning, tool calls, and results that can be badly wrong even when the last message is right, and right by luck when the trajectory was a mess. If your eval only looks at the end, you cannot tell those cases apart, and you will ship regressions you never saw.

Layer 1: Final Answer

Did the agent complete the task? For structured outcomes this is a check on world state (the record was updated, the file exists with the right content) or on the answer (matches, parses, passes tests). For free-text outcomes it is a rubric and a grader.

What it catches: outright failures. What it misses: everything about how.

Layer 2: Trajectory

Did the agent get there sensibly? The trace from a well-built harness contains every proposed, refused, and executed tool call with arguments and results. Most of the interesting grading happens here, and most of it is code.

pythonCopy
def grade_trajectory(task, trace):
    calls = [e for e in trace if e["event"] == "executed"]
    names = [c["tool"] for c in calls]
    refused = [e for e in trace if e["event"] == "refused"]
    verbatim_retries = sum(
        1 for a, b in zip(calls, calls[1:])
        if a["tool"] == b["tool"] and a["args"] == b["args"] and not a["ok"]
    )
    return {
        "required_tools_present": all(t in names for t in task.expected_tools),
        "forbidden_tools_absent": not any(t in names for t in task.forbidden_tools),
        "expected_order": is_subsequence(task.expected_tools, names),
        "within_turn_budget": len(calls) <= task.max_turns,
        "recovered_from_errors": verbatim_retries == 0,
        "args_valid_first_try": len(refused) == 0,
    }

Each check catches a specific failure:

• Required tools present. The agent answered from memory instead of looking it up.
• Forbidden tools absent. The read-only task wrote something.
• Expected order. It added a note before reading the ticket. Grade as a subsequence, not an exact match, so the agent has latitude.
• Within budget. Loops.
• Recovered from errors. A tool failed and the next call was identical. That is a loop wearing a disguise.
• Args valid first try. Retries after schema rejection mean the tool description or schema is unclear.

Report these separately. A blended "trajectory score" hides which one moved, and which one moved tells you what to fix.

Layer 3: Per-Turn, in Production

The suite catches what you imagined. Production catches the rest. Instrument the signals that indicate a turn went wrong without anyone labeling it:

• the user restates or corrects in the next turn
• the run escalates to a human or a bigger model
• the user abandons mid-task
• explicit feedback
• escalation rate or cost per task drifts without a deploy (the inputs changed)

Every triaged production failure becomes a suite task. That flow is the whole system.

Why Not Just Use a Benchmark

Public benchmarks measure models. τ-bench measures how a model does with a fixed set of tools and a simulated user; SWE-bench measures resolving real GitHub issues. Both are useful for choosing a model. Neither tells you whether your agent, with your tools, descriptions, harness, and prompts, works. They also saturate and get optimized for. Your suite is the only eval that measures your system.

Stochasticity

Agents are not deterministic. Run each task several times and grade on pass rate. τ-bench formalizes this as pass^k: the probability that all k attempts succeed. It is the number a user experiences, because users do not get silent retries. A task that passes 4 of 5 runs is an 80% task, not a passing one.

Judges

Free-text outputs need an LLM judge, and a judge is only trustworthy after validation: two humans grade 100 outputs against a specific rubric; if they agree, the judge must agree with them at a high rate on the same 100. Re-validate on any judge model change. Known biases: longer answers win, first option wins, models prefer their own family. Mitigate with pairwise comparisons in both orders and a judge from a different family than the agent.

The Gate

The suite earns its keep in CI. On every change to prompt, tools, harness, or model:

1. Run the full suite, each task several times, against a fixed sandbox.
2. Compare each metric to the baseline. Fail on a drop past a threshold you chose beforehand.
3. Attach the trace diff for failed tasks to the pull request, so the reviewer sees what changed in behavior.
4. Record cost and latency per task next to quality.

What to Practice Next

Write 30 tasks for an agent you have: 10 from failures, 10 representative successes, 10 adversarial. Implement the six checks, run five times per task, and wire it into CI. Then break something on purpose (delete a tool description) and confirm the gate catches it. The module agent-evals-trajectories-tool-calls-and-regression-suites covers suite construction, judges, and production signal in depth.