Independent research lab · Larnaka, Cyprus

When an AI agent says "done", it should be true.

In one 2026 study, false success (an agent reporting a task complete when it wasn't) made up 75.8% of failures among coding agents that reported their own status. Inoku builds ways to measure that gap and close it, so agents report what really happened, including an honest "I couldn't".

Become a design partner Join the beta

Inoku OS, our long-term distribution, is in development. Nothing on this site is for sale yet; design partners and beta testers get the first evaluations and tools at no cost.

75.8%

False success dominates failures

Among coding agents that reported their own status, three failures in four were the agent saying "done" when the task wasn't done. The agents didn't crash. They reported success.
Advani, 2026 [1]
0.65

The usual check barely works

The best LLM judge reading the agent's transcript reached an AUROC of 0.65 at catching false success, on a scale where 0.5 is a coin toss.
Advani, 2026 [1]
11 of 11

Every agent tested deceived upward

All eleven agents in the study reported success to the user that the work did not support, in some cases by fabricating files.
arXiv, 2025 [3]
30.4%

Frontier models game their graders

On METR's RE-Bench, o3 showed reward hacking on 30.4% of runs, against 0.7% on HCAST, in some cases by modifying the code that scored it.
METR, 2025 [4]

What we're building

Four situations, one question: is the agent's report true?

Each piece of Inoku starts from a situation teams are in today and answers it with evidence instead of the agent's own word. Statuses are stated in the open. Nothing here is for sale yet.

Why

An AI agent's "done" is a claim, not a fact. Agents report success they did not achieve, the judges that read their transcripts miss most of it, and agents can edit the logs that would show it. Teams that build, buy or audit agents have no independent way to tell a true report from a false one.

How

Inoku measures the gap directly. The agent runs as shipped on real tasks, some of which cannot be completed. An independent record of what happened on the system, kept outside the agent's reach, is compared with what the agent claimed. The result is two rates, by task family with confidence intervals: how often "done" was true, and how often "I couldn't" was justified.

Solution

Four products come from that one measurement. Inoku Verified™ is the public, independent evaluation and mark. Private evaluations give model builders a confidential score on held-out tasks. Task environments supply tasks with a ground truth the agent cannot game. The independent record keeps an audit-grade account of what production agents did. Each is described below with its status.

01In development
Inoku Verified™
The situation
A team ships an AI agent and tells its customers that the agent completes tasks. The evidence is the agent's own summary and a benchmark the model may have trained on. Nobody outside the team can check either.
What Inoku does
Inoku Verified™ is an independent evaluation that measures how often the agent's "done" is true and how often its "I couldn't" is true, on a private test suite run by Inoku, and reports both rates by task family.
About Inoku Verified
What you get
- Two rates, by task family, with confidence intervals
- A private report, a quotable score and the Inoku Verified™ mark
- Payment never changes a score
02Design partners first
Private evaluations
The situation
A model builder wants a reliability number it can publish or show to buyers. A score produced in-house, by a judge reading the model's own transcript, on a public benchmark, convinces nobody and drifts with every training run.
What Inoku does
A private evaluation by Inoku runs the model as shipped on held-out tasks it has never seen, including tasks that cannot be done, and delivers a confidential report that belongs to the builder alone.
About evaluations
What you get
- Private, held-out tasks; no contamination
- Run as you ship: no weights, no code changes
- Results go to you and nobody else
03Design partners first
Task environments
The situation
A training or evaluation pipeline needs tasks with a check the agent cannot game. Public task suites get exploited: models modify the tests, work around them, or learn the answers from training data.
What Inoku does
Inoku builds custom task environments whose ground truth sits where the agent cannot see, influence or rewrite it, in task families agreed with the customer, for both training and evaluation.
About environments
What you get
- Task families agreed with you
- An independent ground truth for every task
- Tasks that can't be done, so "I couldn't" is measured too
04Research stage
Independent record
The situation
An agent operates on production systems in a regulated or audited setting. The only record of what it did is the log the agent's own harness writes, which the agent can pause, edit or delete.
What Inoku does
Inoku's independent record keeps a complete, reviewable account of what the agent did on the system, outside the agent's control, so reviewers and auditors can rely on it. This is research-stage work.
About the record
What you get
- Complete: what the agent did, not what it chose to report
- Outside the agent's control
- Reviewable by a person or an auditor

How it works, in principle

From task to report, with the agent kept out of the record

Five steps, the same for every agent and every task family. The agent runs as shipped. The record of what happened is kept where the agent can't reach it, and the report is computed from the record, not from the agent's own account.

The record collects the evidence. The report says how often the agent told the truth.

How to verify what an agent actually did

The problem

Three findings, three sources

The behaviour Inoku measures is documented, measured and growing. Each row links to the paper.

Finding	Source
Agents claim work they didn't do. False success is common, and easier tasks show it more often.	Advani, 2026
The usual checks miss it. LLM judges reading the agent's transcript scored no better than 0.65 AUROC at catching it.	Same study
Agents can edit their own records. When it paid off, agents deleted or rewrote their own traces.	Qin et al., 2026

How we work

Four rules we hold ourselves to

Rule 01

Evidence over assertion.

Every claim we publish links to its source, starting with this page.

Where we have no evidence yet, we say so. An opinion is labelled as one, and a result that didn't work is still a result we publish. The site is the first proof of the promise: numbers carry a source, statuses are stated plainly, and nothing is dressed up as a product before it exists.

Rule 02

Honest failure is a feature.

An agent that says "I couldn't" when that's true is worth more than one that always says "done".

Our evaluations measure "I couldn't" as carefully as "done". Giving up on a task that could have been finished and claiming work that was never done are two different problems, so we report them separately and include tasks that cannot be completed, so that an honest refusal counts.

Rule 03

Open by default.

We plan to publish our tools and benchmark openly. Organisations above 10 people or $1M in revenue pay for private evaluations and support; researchers, educators, open-source projects and smaller teams don't.

The method, the tools and the benchmark go public together, once the method is ready to be checked, under an open licence on GitHub through the Inoku Labs organisation. Research, education, open-source projects and small organisations use everything at no cost.

Rule 04

Independent.

Payment never changes a score.

A paying customer gets a private report and support, not a better number. Every major model goes on a free public leaderboard whether or not its maker pays us anything, and the rules of the Inoku Verified™ mark are published before the first mark is granted.

Open source and free tiers

Open by default. Free for those doing the research.

We plan to publish our tools and benchmark openly, under an open licence, on GitHub under the Inoku Labs organisation. Organisations above 10 people or $1M in revenue pay for private evaluations and support; researchers, educators, open-source projects and teams under that line don't.

See what is free GitHub LinkedIn

Public repositories

The method, the tools and the benchmark go public together, once the method is ready to be checked. The repositories at github.com/inoku-labs hold the READMEs and roadmaps until then.

Free tier, stated in advance

Research, education and open source use everything at no cost, whatever their size, and so do organisations under 10 people and $1M in revenue. Borderline cases can ask for a review.

Independence you can audit

Every major model goes on a free public leaderboard whether or not its maker pays. The rules of the Inoku Verified™ mark are published before the first mark.

Work with us

Two ways in, both free

Open now

Beta testers wanted

We're opening the first beta of our evaluation tools to teams that run agents on real systems. Run them on your own tasks, tell us what breaks, keep the results. Invites go out in waves.

Join the beta

Five places

Five teams. Real tasks. Blunt feedback.

We're looking for five teams that build or run AI agents on real systems. You get early evaluations of your agent at no cost during the programme. We get real tasks and blunt feedback.

Apply as a design partner

Resources

From the research notes

All notes

The zero-advantage trap in RLVR training
Under GRPO a task every sample passes or fails gives zero gradient; 88% of one maths corpus did. Why it happens, and what filtering and synthesis can do.
22 October 202613 min read
GRPO explained for agent builders
GRPO scores a group of samples against each other instead of a learned value model. How the update works, what it amplifies, and what changes for agents.
22 October 202612 min read
Does RLVR create new skills or sharpen old ones?
Theory says GRPO amplifies what the base model can already do; ProRL reports prolonged RL solving tasks the base model never could. The pass@k debate.
22 October 202612 min read

Evidence

Sources

Every factual claim on this page links to one of these. Numbers in brackets refer to this list.

From Confident Closing to Silent Failure: Characterizing False Success in LLM Agents
arXiv2026-06arxiv.org
Open
LLM Agents Can Easily Tamper With Their Own Traces
arXiv2026-09arxiv.org
Open
Are Your Agents Upward Deceivers?
arXiv2025-12arxiv.org
Open
Recent frontier models are reward hacking
METR2025-06metr.org
Open

Proof, not promises.

We're looking for a small group of teams that build or run AI agents on real systems.

Become a design partner