What is independent software verification?

Independent software verification is the practice of deciding whether a specific change is ready to ship using a party, a rulebook and a body of evidence that are separate from whoever made the change. The author can claim a change works. Independent verification is what lets anyone else rely on that claim. It is not a tool category so much as a property: any process can have it, and most processes that say they do, do not.

The idea is old in safety-critical engineering and newly urgent everywhere else. It is urgent because code is increasingly written by agents that also run the tests, read the results and report that everything passed. Writing and approving collapse into one actor, and the organisation is left trusting a statement from the party with the strongest incentive to make it. This article defines the term precisely, explains the five duties that independence separates, shows how it is applied in TestWarden, and gives you a rubric to judge any system’s claim of independence yourself.

Use the toggle above to compare two operating models. In the first, one actor creates the code, generates the evidence, vouches for it, applies the rules and approves progression. In the second, five duties are held by distinct parties. The text below defines each duty, so the page reads fully without the interaction.

Where the idea comes from

Independent verification and validation, usually shortened to IV&V, has a long history in aerospace, defence, medical devices and nuclear systems, where a failure costs lives and a developer’s own assurance is not enough. The IEEE’s standard for system, software and hardware verification and validation, IEEE 1012, describes independence along three dimensions: technical independence (the verifier uses its own methods and tools and is not led by the developers’ approach), managerial independence (it reports through a different line, so schedule pressure on the project cannot suppress a finding) and financial independence (its budget cannot be cut by the party it is checking). NASA’s software assurance standards apply the same thinking, and NASA runs a dedicated Independent Verification and Validation Program.

Those three dimensions translate neatly into modern delivery. Technical independence means the evidence comes from infrastructure and tools the author does not control. Managerial independence means the party that decides is not the party that is rewarded for shipping. Financial independence has a softer analogue: the decision authority’s configuration should not be editable by the change it evaluates. You do not need aerospace-grade bureaucracy to benefit. You need the same structural idea, applied at the speed of a pull request.

Four properties that make verification independent

1. Separate authority. The decision is made by something other than the author. If the same agent writes the change, runs the check and reports the verdict, nothing was verified, only asserted.
2. Stated obligations. Before evidence arrives, the rules say what must be shown: which kinds of checks, how fresh, covering which build. A decision made against obligations invented after the fact is an opinion.
3. Evidence tied to an exact build. Proof about one commit says nothing about the next. Independent verification names the repository, commit, build and environment, and only evidence for that target counts.
4. A recorded decision. The outcome, its reasons and the evidence behind it are stored, and later decisions are added rather than rewriting earlier ones.

Each property closes a specific escape hatch. Without separate authority, the author grades their own work. Without stated obligations, “enough” moves to suit the result. Without exact-build binding, a pass about yesterday’s commit launders today’s. Without a record, nobody can later tell what was known when the decision was made. Remove any one and you can still produce a green dashboard, but you cannot call it verification.

Five duties to separate

The four properties are easier to apply if you think in terms of duties. In any delivery process, five things must happen between “someone changed the code” and “the change progressed”. The question is who does each.

Most failures of independence are failures to separate the middle two duties. Teams separate creation from evidence reasonably well, since CI runs tests, and then let the same pipeline decide for itself what its results mean. A pipeline that runs tests, interprets the output, chooses what counts and reports the verdict to the merge button has quietly held four of the five duties. It may be an excellent pipeline. It is not independent verification.

What it is not

• Not a test runner. Verification consumes results from test runners, scanners and reviewers. It does not replace them.
• Not a bigger dashboard. A dashboard shows numbers. A verification decision says whether the numbers are enough for this change.
• Not a second opinion from the same tool. A scanner’s pass is evidence about what the scanner looked at. It does not substitute for the obligation to show a complete test run.
• Not code review. Review is valuable and complementary. It is a human reading a diff, not a guarantee of correctness, and it does not establish that the evidence belongs to the build in front of you.
• Not a guarantee of quality. Independent verification ensures the evidence is attributable, complete and sufficient under the stated rules. It cannot make a poor test meaningful.

How TestWarden applies it

In TestWarden, The Warden is the verification authority. A change is identified by repository, commit, build and environment. CI submits evidence as JUnit XML, SARIF 2.1.0 or provider-neutral JSON, and the Warden evaluates it against a versioned Verification Contract. The outcome is one of five states: CLEARED, VERIFYING, HELD, NEEDS APPROVAL or INCOMPLETE, each with fixed reason codes and a next action.

Clearance is deterministic. An external tool’s green check, approval or score is recorded as evidence and never as CLEARED. The default contract today asks for a complete, current-build automated test run, with evidence no older than 24 hours. Any failed, errored or skipped observation prevents a pass. Contracts you write yourself are not available yet. Only CI holds submission tokens, and coding agents never do, so agent evidence reaches The Warden only through a pipeline and cannot satisfy the requirement for a CI run. Decisions are append-only: a fix produces a new decision for a new build and the earlier HELD remains on record. A concrete walk-through is in a green check is not proof.

TestWarden positions this as a layer above the tools you already use. Customers can keep Xray, AIO Tests, Zephyr Scale, TestRail, Azure DevOps, SonarQube and GitHub Actions. They remain evidence producers. The Warden determines whether their combined, attributable evidence is sufficient for an exact software change under its verification policy. That architecture is described in what a verification control plane is. Where the layer reads results directly from a test-management tool, through read-only collectors, those collectors are in source and under acceptance; live vendor certification is pending, and they are not described as shipped.

Why independence matters more when AI writes the code

Three things change when coding agents enter delivery. Volume rises, because cheap changes are plentiful and reviewers cannot re-derive each result. Authorship and test-writing merge, because an agent asked to make a failing test pass can edit the test. And accountability blurs, because a pull request with an agent’s summary reads like a statement from a colleague while carrying none of the colleague’s incentives. Each pushes toward self-certification, and each is a reason to make the separation structural rather than cultural.

Atlassian’s AI SDLC playbook argues that as agents take over coding, planning, review and measurement grow in importance. The DORA research programme reported in 2024 that higher AI adoption was associated with worse delivery throughput and stability, and the OWASP guidance on LLM application risks lists “excessive agency” among the risks of giving a model authority it should not hold. Granting an agent the power to certify its own change is excessive agency in the plainest sense. The longer argument is in why coding agents should not certify their own changes, and the practical pull-request version is in how to verify AI-generated code before merge.

Three pipelines, scored honestly

Abstract properties are easier to judge on concrete examples. Consider three invented teams, each with a pipeline that ends in a merge button turning green.

Team A runs unit tests in CI, and the pull request requires the “tests” status to be green. The same workflow file defines the job, its inputs and what exit code counts as success. A developer, or an agent with repository write access, can change that file in the same pull request. Evidence and rules live together, the author can edit both and the status says nothing about which commit it checked. Team A has fast feedback and no independence.

Team B adds a required security scan and a required AI code-reviewer comment. More signals, more coverage, and still no independence: every signal is a status the pipeline reports about itself, and nothing evaluates whether the set of signals is sufficient. A pass from the scanner cannot be told apart from a pass from the test suite when the merge rule just counts greens.

Team C keeps the same pipeline but routes the results to a separate authority. CI holds the submission tokens, the author does not. Each submission is a receipt naming the producer, run and exact build. A versioned contract states what must be shown, and a decision, with a reason code and a next action, is computed per build and appended to history. The merge gate reads the decision. Team C did not buy better tests. It separated duties, and the same evidence became verification.

Adopting independence without a big-bang change

Independence is a destination, and the path to it can be gentle. Teams that succeed rarely rebuild their pipelines. They move one duty at a time, and they start by observing before they enforce.

TestWarden is built to be adopted this way. A site admin opts in, CI submits JUnit XML from an existing test job, and decisions can be read alongside your current merge rule before the gate is made required. Nothing needs replacing, and the first useful output is often a disagreement: a build your current rules would have passed that the Warden held, with a reason you can check.

A rubric for judging whether verification is genuinely independent

Interpreting the result is a matter of honesty rather than arithmetic. A tool that scores Yes on the first three and the last is probably doing real verification even if it falls short on the rest. A tool that scores No on separate authority is not independent however it scores elsewhere, because every other property assumes it. Treat the rubric as a conversation starter with vendors, including us. A vendor that cannot answer a question plainly is telling you something.

A five-question test for any tool

1. Who holds the credentials to submit evidence, and can the author hold them?
2. Where are the obligations written, and can they change after the evidence is in?
3. Does the decision name the exact build, or just “main” or “the PR”?
4. Can a green result from one check satisfy a different kind of requirement?
5. Is the decision history append-only?

A tool that answers these clearly is doing verification. One that cannot is aggregating status. Neither is bad. Aggregation is useful. The mistake is to describe it as verification, and then to rely on it as if it were.

Objections worth taking seriously

• “Our team is small and trusts each other.” Independence is not a statement about trust. It is a statement about structure, and it is cheapest to build when the team is small. It also protects the individual: a decision made by a rulebook cannot be blamed on whoever pressed the button.
• “It will slow delivery.” A computed decision costs seconds when evidence arrives. What it removes is the slow path: the discovery, weeks later, that a release was never actually tested.
• “Reviewers already do this.” Reviewers read changes. They seldom check that the evidence behind a green status belongs to the commit in front of them. The two controls are complementary, and neither guarantees correctness.
• “Independent means a separate vendor.” Not necessarily. Independence is about who holds the authority, not who sells the software. A single vendor can provide the producer and the decider if the author controls neither.

Frequently asked questions

To go deeper, read why coding agents should not certify their own changes and the release governance checklist, see the product on AI software verification and The Warden, and check Proof for what has been demonstrated versus what is still under acceptance. To evaluate it on one repository, contact us.