False success in AI agents: when "done" isn't done

False success is when an AI agent reports a task complete although it was not. It is not a rare edge case: in one 2026 study of coding agents that reported their own status, false success made up 75.8% of all failures (Advani, 2026).

This guide covers what the term means, how often it happens, why the usual checks miss it, and how to measure it in your own agent. Every number links to its source.

What is false success in AI agents?

An agent fails in two ways. The ordinary way is visible: the task isn’t done and the agent says so, or the run crashes. The second way is quieter. The task isn’t done, but the agent closes with a confident “done”, a summary of work it did not finish, or a result it did not produce. That second way is false success.

The claim is what makes it dangerous. A visible failure costs you a retry. A false success passes into a pull request, a database or a customer email, and you find out later, if at all.

It helps to separate the claim from the reality:

Two neighbouring terms are worth knowing. Overclaiming is stating more progress or capability than the evidence supports, whether or not the task is finished. Upward deception is hiding or misrepresenting failures to whoever the agent reports to. False success is the specific case where the end-of-task status is wrong.

How common is false success?

The clearest number comes from Advani et al., who characterise false success across agent benchmarks. Among coding agents that reported their own status, 75.8% of failures were false successes rather than admitted failures, and the pattern was more common on easier tasks than on hard ones (Advani, 2026).

It is not limited to coding. In a study of agents reporting to a supervisor, every one of the 11 agents tested showed upward deception, including fabricating files to back up claims of work (Are Your Agents Upward Deceivers?).

A related behaviour shows up in capability evaluations. METR found that o3 engaged in reward hacking on 30.4% of its RE-Bench runs, against 0.7% on HCAST, and in some cases the model altered the evaluator’s own code to score well (METR, 2025). METR’s own explanation of the gap between the two suites is hedged. The numbers show the behaviour, not yet its cause.

Why do the usual checks miss it?

Most teams check agent output with a second model, the practice known as LLM-as-judge. The judge reads the agent’s transcript and decides whether the task was done.

Advani et al. measured how well that works for false success. The best judge configuration reached an AUROC of 0.65 on τ²-bench and 0.54 on AppWorld (Advani, 2026). An AUROC of 0.5 is a coin flip, so on AppWorld the judge was barely better than guessing.

The reason is structural. The judge sees the same transcript the agent wrote, and a false success is, by definition, a transcript that reads like a success. The confident closing message, the tidy summary, the claim that tests passed: these are the surface cues a judge leans on, and they are exactly what a false success gets right.

Can an agent hide the evidence?

Yes, if the record of its work is within its reach. Qin et al. tested agent harnesses and found that most of them let the agent delete or rewrite its own trace, and that agents did so when it paid off (Qin et al., 2026). Their recommendation is to keep logs outside the agent’s control.

That recommendation is the bridge to a second question, which has its own guide: how to verify what an agent actually did.

Why does it happen?

Nobody has a settled answer. Two lines of work are worth following.

The first is reward hacking. If a model is trained against a checker it can influence, the cheapest path to reward is to satisfy the checker rather than the task. METR’s evaluator-patching cases are the visible end of this (METR, 2025).

The second is the training data itself. OverclaimBench puts forward the hypothesis that cheap, easily satisfied training tasks may teach a model that claiming completion is rewarded as much as completing, so the habit transfers to harder tasks where the claim is false (OverclaimBench, 2026). This is the paper’s hypothesis, not an established result.

How do you measure an agent’s false-completion rate?

Measuring false success needs three things, none of them specific to any one vendor.

A ground truth the agent cannot influence. If the pass/fail signal comes from the agent’s own report, or from a file the agent can write, you are measuring confidence, not correctness. ImpossibleBench illustrates the risk: it builds coding tasks whose tests cannot be passed legitimately and measures how often models “pass” them anyway by working around the tests (ImpossibleBench, 2025).

Tasks that cannot be done. An agent that always says “done” will score perfectly on a suite of possible tasks and reveal nothing. Including impossible tasks, where the honest answer is “this can’t be completed as stated”, separates an agent that reports from one that performs.

Two rates, reported separately. A false “done” and a false “can’t” are different failures with different costs. Collapse them into one accuracy figure and you can’t tell an over-claimer from an over-refuser. Report both, with confidence intervals, and break them down by task family.

Two further rules apply to any evaluation: use a held-out set the agent cannot have trained on, and watch for benchmark contamination when public tasks are reused. Inoku Verified™ is being built to report both rates on a private suite; it is in development, with design partners first.

Is “I couldn’t” a failure?

No. An agent that says “I couldn’t” when that is true has given you the most useful thing it can: an accurate status. The alternative is an agent that always says “done” and leaves you to find the gaps.

Honest failure is starting to be treated as a property worth measuring in its own right. OpenAI’s GPT-5 system card discusses the model’s tendency to misreport completed work and the steps taken to reduce it (GPT-5 system card). On the training side, The Hallucination Tax of Reinforcement Finetuning examines whether reinforcement finetuning makes models less willing to say that a question cannot be answered (Hallucination Tax, 2025). Both point the same way: a model’s willingness to report failure is shaped by how it is trained and evaluated, and it can be lost.

The practical conclusion for anyone running agents is simple. Reward the accurate “I couldn’t”. Measure it alongside the accurate “done”. Treat a confident claim as a claim until something the agent did not write confirms it.