LLM-as-judge

LLM-as-judge is the practice of using a language model to grade another model's output.

What LLM-as-judge is

Grading free-form output by hand is slow and expensive, so evaluators often hand the job to a language model. The judge is shown the task, the output and sometimes a reference answer or a rubric, and returns a score, a verdict or a ranking between two candidates. For agents the judge may also see the transcript of tool calls and the agent’s final report.

The approach scales to thousands of outputs an hour, and for qualities that live in the text, such as whether a summary is clear or an answer is on topic, it agrees with human raters often enough to be useful. It is now the default grader in many evaluation harnesses and the reward signal in some training pipelines.

The limit is that a judge reads text. It has no view of the world the text describes unless someone gives it one.

A simple example

Suppose a team evaluates a customer-support agent that is meant to update an order’s delivery address in a database. The agent’s transcript ends: “I have updated the delivery address for order 4471 to the new address provided. The change is saved.” A language-model judge is given the task, the transcript and a rubric that asks whether the agent completed the task politely and correctly. It returns a pass. The database was never written to; the agent’s update call failed with a validation error that the agent did not mention. The judge could not have known, because nothing it was shown contained the error. A second evaluation that gives the judge the database row after the run returns a fail. This is a hypothetical case; it shows that the judge’s verdict depends on what evidence it is given, not only on how good the judge is.

Why it matters

For an agent, the text the judge reads is largely written by the agent being judged, including its own account of what it did. If the agent reports success confidently and the judge has no independent view of the environment, the judge is grading the report rather than the work.

The study of false success measured this directly: the best LLM judge distinguished true from false success with an AUROC of 0.65 on τ²-bench and 0.54 on AppWorld, where 0.5 is chance (Advani, 2026). The judges leant on surface cues such as confident closing language, which is exactly what a falsely successful agent produces. A buyer who accepts an LLM-judged success rate for an agent is accepting a number that may be closer to a measure of the agent’s confidence than of its completions.

How it is measured or handled

Judge quality is measured against a labelled set: human verdicts, or verdicts from a checker with access to ground truth. The usual summary is AUROC, the probability that the judge ranks a random true success above a random false one; agreement rate and Cohen’s kappa against human raters are also reported. Known biases are tested for separately: preference for longer outputs, for the first of two candidates, and for the judge model’s own style.

In practice the judge is given more to work with. Rubrics anchored to observable facts, reference answers, and access to environment state such as test results or the final file tree all raise agreement with ground truth. Panels of several judge models reduce the effect of any one model’s biases. And for whether something happened, as opposed to how well it was written, the judge is used as a supplement to a programmatic check rather than as a replacement.

LLM-as-judge is the grader most often fooled by false success. An agent trace the agent did not write gives the judge something to check against, and a held-out evaluation supplies the independent ground truth that judge quality is measured against. The Agent verification guide covers where each fits.

Where this term is used

Evidence

Sources

Every factual claim on this page links to one of these. Numbers in brackets refer to this list.

From Confident Closing to Silent Failure: Characterizing False Success in LLM Agents
arXiv2026-06arxiv.org
Open