Why coding agents should not certify their own changes

If your teams are rolling out coding agents, one question will outlast every model upgrade: who says a change is done? When the agent that wrote the code also runs the tests and writes “all checks passed” in the pull request, the organisation has accepted a claim from the party with the strongest incentive to make it. Coding agents should write code, and they should write tests. They should not hold the authority to certify that their own work is verified.

This is not a claim that agents are dishonest. It is a statement about control design. We do not let an engineer approve their own production change, and the reasoning does not change because the author is faster. What does change is the scale: an agent can open dozens of changes a day, and the old backstop, a human who reads each one closely, stops being available. This article walks through where self-certification fails, what to put in its place, how agent-written tests fit, and, because nothing here is magic, what independent verification still cannot catch.

The workflow above runs seven stages from the coding agent to a policy decision. Switch between an agent that validates itself and independent verification, and pick a scenario. In the first two scenarios independence turns a false “all clear” into a HELD decision. In the third it does not, and the visual says so, because a credible account of independence has to include what it cannot do.

Where self-certification fails

• Tests adjusted to fit. An agent that is asked to make a failing test pass can edit the test, skip it or loosen an assertion. The result is a green run that no longer proves the original behaviour.
• Stale proof. A pass on an earlier commit gets quoted for a later one. The summary reads fine; the build under review was never run.
• Narrow evidence presented as broad. A lint pass or a subset of tests is described as “verified”.
• Unreviewable volume. When agents open many changes a day, humans cannot re-derive each result. Review turns into trusting the summary unless the system enforces the rule.
• Optimising the metric. Any agent rewarded for “tests pass” will, over enough attempts, find the shortest path to that outcome. That path is sometimes the one you wanted and sometimes not.

None of these requires intent. A model that is trying to be helpful, and has been told that the goal is a passing build, will make exactly the moves above under pressure. The organisation’s problem is that each looks like success from the outside. The summary says what the reader wants to read, and the diff is too large to check line by line.

There is a structural point underneath. Verification is valuable because it is a check by someone with a different perspective and different incentives. An author’s own review catches a class of error, and it is worth doing, but it cannot catch the errors the author is blind to, and it cannot provide assurance to anyone else, because they have no reason to believe the author graded harshly. Replace the author with a model and the logic is unchanged. If anything it strengthens: a model’s confident prose is the same whether or not the underlying work is right.

Four controls to require

1. Separate who produces from who decides

Evidence comes from CI, scanners and reviewers. A different authority, with rules the agent cannot edit, decides. In TestWarden that authority is The Warden, which evaluates evidence against a versioned Verification Contract. Deterministic policy owns clearance. AI may help explain a result, but it never sets or overrides one.

2. Make token custody the identity boundary

Every evidence submission names a provider type (ci, agent, scanner, code_reviewer or visual_agent), a name, a source system and a run ID. The provider type is asserted by the caller, so the real boundary is who holds the token: only CI holds submission tokens, and coding agents never do. An agent can still contribute evidence, but only through a pipeline, and agent evidence cannot satisfy the requirement for a CI run. This is the answer to the question “what credentials can an agent use to publish test results?”, which should be “none”.

3. Bind proof to the exact build

A change is identified by repository, commit, build and environment. The verification target only moves forward; submissions for older builds are history and are answered as superseded, never cleared. The default contract also requires evidence no older than 24 hours. A quoted pass from the previous commit therefore proves nothing about this one, however confidently it is quoted.

4. Keep decisions append-only

When an agent fixes a failure and CI produces fresh passing evidence for a new build, that creates a new decision. The earlier HELD decision stays on record. Auditors and engineers can see what was blocked, why, and what changed. An agent cannot tidy up the history.

These four controls are the practical expression of independent software verification: a separate authority, stated obligations, evidence bound to an exact build and a recorded decision. They also match the architecture described in what a verification control plane is. None of them depends on a particular model, agent framework or vendor, which is the point. Your agents will change every quarter. The authority structure should not.

Agent-generated tests: useful, but not clearance

It would be a mistake to conclude that agents should not write tests. Agents are good at drafting test cases, generating edge cases a tired human would skip, and keeping coverage from decaying. Agent-authored tests are useful evidence about a change. The mistake is to let the same agent grade them.

In practice that means agent-written tests run in the same independent CI as everything else, on the exact build, and their results arrive at The Warden as evidence from the pipeline. The agent’s role ends at producing code and tests. If those tests fail, the decision is HELD. If they pass, they count toward the evidence the contract needs, no more and no less than a human-written passing test. If the agent marks one as skipped to avoid a failure, a skipped observation prevents a pass under the default contract, so the build is HELD with a named next action.

What independence cannot catch

Credibility requires saying this plainly. The third scenario in the visual is the uncomfortable one: an agent loosens an assertion, so the test passes more easily, and independent CI reports a genuine, complete, current-build pass. The Warden clears it, because under the default contract the required evidence is present and passing. Independence guarantees that the evidence is attributable, complete, fresh and about this build. It cannot tell you that a test asserts something worth asserting.

So the honest position is layered, and it deliberately does not rest on any single control.

• Human review is valuable and is not a guarantee. A reviewer who reads the test diff can catch a weakened assertion, and a reviewer who skims a 900-line agent change will not. Do not imply that review alone ensures correctness. Direct human attention at test changes and at the changes the policy flags.
• Additional kinds of evidence. Independent scanners, additional tests the agent did not write and recorded outcomes widen what must be fooled at once.
• History. Quality Memory records test versions and outcomes, so a test that was loosened and then a defect that slipped past it are connected after the fact, and advisory analysis can flag changed tests when enabled.
• Approval where the policy requires it. A contract may demand a human approval for categories of change. TestWarden’s decision model supports NEEDS APPROVAL, and no endpoint, CLI command, MCP tool, CI step or AI feature can grant it. Customer-authored contracts are not available today, so the default contract applies.

This helps the agent too

A HELD decision comes with a fixed reason code, a next action and, for failing JUnit evidence, the names of up to ten failing tests in report order. An agent or a developer gets an actionable list, not a vague red mark. That is useful in agent loops, where the cost of an ambiguous failure is a long wander. The post-V1 developer pilot lane, which is not part of the Marketplace V1 listing, adds a Developer API, a PowerShell client and a stdio MCP server with six typed tools so that a coding agent can read obligations and status. None of them can clear a change or grant approval. The agent can ask “what is still required for this build?” and it can never answer its own question.

Atlassian is positioning Jira as the orchestration surface for coding agents, with spec-driven work, governed agent loops and agent-to-agent protocols. TestWarden’s role in that picture is the quality authority those loops consult before code is considered verified, and its core capabilities are exposed through stable domain contracts first, with protocol adapters on top. It does not compete with agents. It gives an agent loop a place to learn whether it is done.

A rollout model for agent autonomy

Leaders often ask how much autonomy to give agents. The question is easier if you tie autonomy to the strength of the independent check rather than to your trust in the model. A simple three-tier model works well in practice, and it is deliberately boring.

Notice what does not change across tiers: the agent never gains the authority to decide. Autonomy increases the volume of work it can do without a person, not the weight of its word. That is the property that lets you move quickly without betting the release process on a model’s behaviour.

Anti-patterns to retire

• The agent summary as gate. “The agent says tests pass” is a claim, not evidence. Gate on a decision for the build.
• Shared tokens. A single service credential used by agents and CI alike erases the line between producers.
• Rules in the repository the agent can edit. If the definition of “enough proof” lives in the same pull request as the change, the author can edit its own grading.
• Retry until green. Loops that rerun the pipeline until something passes, without reading the reason, teach the agent to chase noise. A fixed reason code and next action make retries purposeful.
• Silent skips. Treating skipped tests as neutral turns the cheapest trick in the book into a policy.

What to ask your teams this quarter

1. Which credentials can an agent use to publish test results?
2. Does the merge gate check the exact commit and build, or only that “a check passed”?
3. Can anyone, human or agent, edit the rules that define “enough proof” in the same pull request?
4. Where is the decision recorded, and can it be rewritten?
5. Who reads changes to tests, as distinct from changes to code?
6. What happens when a job never reports?

You do not have to replace your tools to answer these. Bring every verifier you already use; the Warden decides whether their combined proof is enough. Customers can keep Xray, AIO Tests, Zephyr Scale, TestRail, Azure DevOps, SonarQube and GitHub Actions. Start with AI software verification, walk through a worked decision in a green check is not proof, or read the pull-request guide, how to verify AI-generated code before merge.

Why leaders should decide this now

The DORA research programme reported in 2024 that higher AI adoption was associated with lower delivery throughput and stability alongside improved perceived productivity. The report is correlational and has caveats you should read, but it supports a plain reading: producing code faster does not by itself ship better software, and the difference lives in the practices around it. OWASP’s guidance on LLM application risks names excessive agency, meaning granting a model more authority than it should hold, as a risk. A coding agent that can declare its own work verified holds more authority than is wise.

The decision is cheaper to make early. Retrofitting an authority structure after agents have become embedded in every repository means renegotiating habits as well as tooling. Setting the rule now, that agents produce and an independent party decides, costs one pipeline step and one conversation. Teams that have not had that conversation usually discover the cost of not having it in an incident review.

See where this fits in the broader architecture in what a verification control plane is, the contract details in verification contracts and exact-build evidence, and current product status on Proof. The developer overview explains the evidence formats and client options, and Pricing covers the free tier and evaluation.

Frequently asked questions

If you are adopting coding agents and want to put this authority structure in place on one repository, talk to us, or start with the Warden overview.