Inoku Verified™: independent evidence that your agent's "done" is true

What Inoku Verified™ measures

When an AI agent reports that a task is done, Inoku Verified™ measures how often that report is true. It also measures the opposite case: when the agent says a task could not be done, how often that is true.

An agent gets a result on two questions.

• How often is “done” true? The share of completion claims that match what actually happened. The gap is false success.
• How often is “I couldn’t” true? The share of refusals and give-ups that were justified. Giving up on a task that could have been finished is a different problem from claiming work that was never done, so we report the two separately. See honest failure.

Both numbers are broken down by task family (coding, operations, data and so on, agreed with you) and reported with confidence intervals. A single headline score hides too much.

The problem is not marginal. In one 2026 study of coding agents that reported their own status, false success made up 75.8% of all failures (Advani, 2026). The agents did not crash. They said “done”.

Why it has to be independent and private

Two things undermine most published agent scores.

A lab can’t credibly grade itself. The most common check today is an LLM judge reading the agent’s own transcript. Measured against ground truth, the best such judge reached an AUROC of 0.65 on τ²-bench and 0.54 on AppWorld (Advani, 2026). A judge run by the organisation that built the agent adds a conflict of interest on top. See Inoku vs LLM-as-judge.

Public tasks leak into training data. Once a benchmark is public, its tasks end up in the data future models learn from, and the score stops measuring the skill it was meant to measure. This is benchmark contamination. ARC Prize deals with it by keeping evaluation sets that are never published and running the tests itself (ARC Prize, 2024). Inoku Verified™ follows the same practice: a private suite the agent cannot have trained on, with held-out tasks that stay private.

There is a third reason. Agents can change their own records. In one study, most of the agent harnesses tested let the agent delete or rewrite its own traces, and the authors recommend keeping logs outside the agent’s control (Qin et al., 2026). An evaluation that relies on what the agent says about itself measures the agent’s writing, not its work.

What you get

• A private report. Both rates, by task family, with confidence intervals and the task-level detail behind them. It goes to you and nobody else.
• A score. One figure you can quote, with the method it came from.
• The Inoku Verified™ mark. Use of the mark under rules we will publish before the first mark is granted. The rules will state what a mark covers: which agent version, which task families, and when it was measured.

Independence rules

These are commitments, written in the future tense because none of them is true yet.

1. We will publish the method. Not a summary: the method, so that others can check it and criticise it.
2. Every major model will be ranked on a free public leaderboard, whether or not its maker pays us anything.
3. Payment never changes a score. A paying customer gets a private report and support. It does not get a better number.

Status

Inoku Verified™ is in development. The first evaluations go to design partners, at no cost during the programme. Model builders who want a confidential run can read about private evaluations. Larger organisations pay once the programme ends; see pricing. For the thinking behind the approach, start with how to verify what an AI agent actually did.