Independent release governance: TestWarden vs the tools already in your stack.

Four kinds of evidence, one unanswered question

A release candidate generates evidence from at least four directions, and each direction answers a different question. Test results say whether the behaviours somebody chose to check still work. Code scanners say whether the source meets quality and security rules. Delivery pipelines say whether the change moved through the agreed process and who approved each step. Signed artifacts and supply-chain records say what was built, from what, and whether the package is the one that was promised.

Each of those is valuable, and each is produced by a tool that is excellent at its own job. None of them, on its own, answers the question a release manager actually has to sign: did this exact change satisfy the proof that our verification policy requires? That question has several parts. Is the evidence about this repository, this commit, this build and this environment, or about something nearby? Is every required kind of evidence present, complete and recent? Do two reports disagree? If an earlier run failed and a later run passed, can anyone still see both? And when the author of the change was an AI coding agent, who or what decided that the agent’s own work was enough?

Teams usually answer these parts by hand, in a pull request thread, a spreadsheet or a release-day meeting. That works until volume rises. AI-assisted development raises volume sharply: more changes, produced faster, often including the tests that are meant to check them. The human answer to “is this enough” gets thinner exactly when the number of questions gets larger.

TestWarden exists for that gap. It does not replace the tools above. It sits on top of them, in Jira, and records one decision per exact change from the evidence those tools produce. This page explains how that differs from, and fits with, Harness, SonarQube, JFrog AppTrust and Quality Clouds, with Xray and AIO Tests as test-management sources. It states where each of them is stronger than TestWarden, because for most buyers the right answer is “use both”.

What TestWarden is and is not

TestWarden is a Jira Cloud app, built on Atlassian Forge, with two parts. The Warden is an independent verification authority: it receives evidence about a software change, compares it with a verification policy, and records a decision for the exact repository, commit, build and environment. Native test management is an optional second part: tests, plans, cycles, version-pinned attempts, coverage and Bug creation inside Jira, for teams that want them in one place.

What it is

• A decision layer. The outcome is one of a small set of states such as HELD, CLEARED, INCOMPLETE or NEEDS APPROVAL, each with actionable reasons.
• Deterministic. A written policy, not a model’s opinion, decides. AI and humans supply context and evidence; neither can set the outcome by asserting it. Evidence payloads that carry a caller-chosen state or an approval are rejected.
• Bound to the exact change. Evidence for a different commit, a superseded build, a stale run, or a mixed set of runs does not count. When proof is failing, stale, conflicting or about the wrong build, the answer is HELD or INCOMPLETE, never a quiet pass.
• Append-only. A HELD decision is not overwritten when a repaired build is later CLEARED. Both stay in the record.
• Tenant-isolated memory. Every receipt and decision becomes part of a per-tenant Quality Memory, described below.

What it is not

• Not a code scanner. It does not replace SonarQube’s static analysis or any SAST tool. It can consume a scanner’s SARIF output as one kind of evidence.
• Not a delivery or release-orchestration platform. It does not deploy, run pipelines, cut branches or coordinate teams the way Harness does.
• Not an artifact repository or signing authority. It does not store packages or produce supply-chain attestations the way JFrog does.
• Not a replacement for Xray, AIO Tests, Zephyr, TestRail or Azure DevOps. Customers keep them. TestWarden’s native test management is optional.
• Not an agent that releases code. Nothing in the product lets an AI agent clear its own change or authorize a release.
• Not a compliance certification. The product claims no SOC 2 or ISO certification and does not promise defect-free software.

Two boundaries on what is shipped are worth stating early because comparison pages often blur them. First, customers today use the default verification contract; customer-authored contracts, waivers and approval options are not built (see Proof). Second, evidence arrives as complete JUnit XML, SARIF 2.1.0 or a bounded JSON envelope from CI. Any tool that can write those reports can be a producer, but there are no first-party adapters for specific scanners or test runners.

Side-by-side: five tools, nine dimensions

The table below describes each product by its primary job, using each vendor’s own documentation as read on 2026-10-28. It is a map of different jobs, not a scorecard. There are no ratings, no price comparison and no claim that one product is better overall. A blank in one product’s column does not mean the product lacks the capability elsewhere; it means this page does not assert it.

Read the “limits” row as boundaries of job description rather than defects. Harness is not meant to be a test-evidence authority and does not claim to be. SonarQube is not meant to judge whether the right tests ran. Calling those “weaknesses” would be an unfair comparison, and it would also be unsupported by their documentation. The reverse applies to TestWarden: it is not a scanner, pipeline or artifact store, and says so.

Xray and AIO Tests are not in the table because they are not competitors in this comparison. They manage test cases and executions, and they are the most likely source of the “tests passed” evidence that TestWarden judges. Teams that keep them keep their system of record. The relevant pages are TestWarden alongside Xray and alongside AIO Tests; those pages concern teams deciding about test management itself, while this one concerns the decision above it.

What each tool does better than TestWarden

A comparison that only lists advantages for its author is advertising. These are the honest counterparts.

Harness: running the release, not just judging it

Harness coordinates the work of a release: phases, activities, approvals, pipeline triggers and an audit log that its page describes as immutable. If your pain is that releases are coordinated across spreadsheets and chat, TestWarden does not solve it and Harness is built for it. There is genuine overlap in one place: both can hold an approval gate and both keep an audit record of decisions. The difference is what the gate is about. A Harness approval typically records that a person or a process agreed to move forward. A TestWarden decision records whether the evidence about one specific change satisfied a policy. Teams can reasonably use the second as an input to the first.

SonarQube: deep code analysis

SonarQube’s documentation describes AI Code Assurance as a way to label projects containing AI-generated code, apply a quality gate for them and publish badges, with checks focused on complexity, bugs and injection vulnerabilities. That is static analysis expertise TestWarden does not attempt to replicate. TestWarden’s strength begins after analysis: it can require that a scanner’s report exist, be complete and be about the right build, alongside test evidence. A passing Sonar quality gate is excellent evidence. It is not the same claim as “the required tests ran against this build”, and the two together are stronger than either.

JFrog AppTrust: the closest conceptual neighbour

JFrog describes AppTrust as embedding “evidence-based policy gates” in the SDLC so that only verified, compliant application versions reach production. Versions progress through stages and, when controls pass, receive a signed Trusted Release attestation. It also collects evidence from several sources, including SonarQube, and expresses policies as code. JFrog does have evidence-based policy gates; this page does not claim otherwise. The overlap is real, and for teams already on the JFrog Platform AppTrust is the natural home for artifact-centred governance. The differences are of centre of gravity. AppTrust’s unit is the application version and its artifacts; TestWarden’s unit is the exact change against Jira-resident verification obligations, with Jira roles, issues, native test management and a durable history of attempts. They can coexist, and an organization could feed one’s outcome to the other as evidence.

Quality Clouds: governance for specific platforms

Quality Clouds positions itself around quality, governance and maintainability for enterprise platforms such as Salesforce, ServiceNow and Microsoft 365, analysing code, configuration and metadata. Its LivecheckAI capability examines AI-generated logic from development tools. If your estate is heavily Salesforce, that platform-specific knowledge is something a general verification layer cannot match. TestWarden would simply treat any report from such a tool as evidence, if it can be expressed as JUnit or SARIF. Note that the documentation page supplied for this review was unreachable when checked, so this description is intentionally brief.

Keep your stack: the architecture

The principle is simple: evidence producers stay where they are; TestWarden consumes what they emit and decides. Concretely:

1. Producers run in your CI: tests, scanners, builds, coverage, and any authorized AI agent. They write standard reports.
2. Provenance validation binds each report to a repository, commit, build, run and environment. Reports that cannot be bound do not count toward clearance.
3. The verification contract lists the evidence obligations for the change.
4. The Warden evaluates the obligations and records HELD, CLEARED or another state, with reasons.
5. Jira, a native Check and the audit history carry the outcome back to where people work.

The table below labels each input by what is true today, using the same four labels as the integrations page.

That table is the limit of the integration claim. It does not say “live Xray integration”, because there is none yet. It says a collector is implemented and waiting for a real vendor account to prove it.

A HELD to CLEARED example, honestly labelled

The scenario below is illustrative and synthetic: invented names and numbers, there to show the mechanics.

1. A developer’s AI agent changes a payment-retry module and opens a pull request. CI builds commit a1b2c3.
2. The pipeline’s unit tests pass. The integration suite is cut short by an infrastructure timeout, so its JUnit report is incomplete. A scanner report is attached and clean.
3. The Warden evaluates the contract. A passing unit report and a clean scan exist; the integration obligation is not satisfied by an incomplete report. Decision: HELD, with the reason “integration results incomplete for build a1b2c3”.
4. The developer fixes the infrastructure issue and the agent pushes a corrected commit d4e5f6. Evidence for the old commit does not carry over.
5. New complete reports arrive for d4e5f6. All obligations are satisfied. Decision: CLEARED, for d4e5f6 only.
6. The record now shows both: HELD for a1b2c3, CLEARED for d4e5f6. Nothing was rewritten.

What is real. The pattern above is the one observed, with real data, in TestWarden’s own development installation. The public Proof page records that real GitHub Actions runs against TestWarden’s repository produced a HELD decision for a failing build and a CLEARED decision for the fixed build, with the earlier decision retained. A newer internal acceptance record dated 2026-10-27 (draft engineering pull request, installed development version 3.18.0) additionally documents a CLEARED decision confirmed by an independent read of the native Check for the exact source and run, and a Quality Agent that dispatched a real workflow whose outcome was read back independently, with the original HELD decision still on record. That record also says the engineering result is NO-GO for customer release: broad AI comparison and some model-driven capabilities were not fully accepted, vendor integrations are not certified, and non-owner roles and licensing journeys are unverified. It is a genuine development result, not a customer outcome and not a release.

AI advice versus governance authority

When an AI coding agent writes the change and also writes or edits the tests, it is both the author and its own examiner. Even a well-behaved agent can be wrong in ways that make its tests agree with its code. The remedy is not to forbid agents; it is to separate who produces from who decides.

In TestWarden’s model the agent can do useful things: propose tests, review evidence, explain why a change was HELD, and, where an owner has authorized it, dispatch a verification run. What it cannot do is decide. The decision belongs to the deterministic policy, working only from evidence that is bound to the exact build. In the 2026-10-27 development acceptance, a Quality Agent’s dispatched run produced evidence, and a separate protected worker fetched the actual run and Check before Forge read the new and original decisions. The agent did not clear anything; the policy did, from independent readback.

This matters for buyers because AI features in adjacent tools are not all the same. Some label AI-generated code and apply a stricter gate; some generate release processes; some check AI-produced logic. Those are useful. The distinct property here is the rule that an agent’s own assertion is never evidence of its own success.

Quality Memory, precisely

Quality Memory is the tenant-specific, append-only history that results from using The Warden and native test management: evidence receipts, decisions, version-pinned test attempts, who changed what and when, and the provenance of each. It is the authoritative record, and it is what makes the second decision about a service better informed than the first.

• It is not model training. TestWarden does not continually train model weights on your data, and does not pool one customer’s data with another’s. Memory is stored history that grounded features can retrieve.
• Provenance travels with it. A summary is only as good as the receipts beneath it, so each record points to its source.
• It is tenant-isolated. Records belong to a Jira site. Two-site development isolation has been tested by the owner, which is not the same as commercial customer acceptance.
• What is proven. The immutable history of receipts and decisions is proven in development. A bounded, Forge-hosted Claude explanation citing real execution history (three executions, two failures, test versions 4, 5 and 6) passed in the 2026-10-27 acceptance.
• What is not proven. Broad AI comparison, general model use of imported source-control context, and continuous production-outcome monitoring are not claimed as accepted capabilities. Anything described as “learning” in the future is a direction, not a feature.

Who should use what

SonarQube alone, or TestWarden alongside it

If your main risk is code quality and security findings, and test evidence is already trustworthy and reviewed by people, SonarQube alone can be right. Add TestWarden when the open question is whether the full set of proof was complete and about the right build, especially with AI-authored changes and tests.

Harness alone, or TestWarden alongside it

If your pain is coordinating releases across teams, Harness is the more direct answer. Add TestWarden when approvals in the release flow need a defensible, independent answer to “was the verification obligation met for this change”. TestWarden is not a substitute for release orchestration.

JFrog AppTrust alone, or TestWarden alongside it

If your estate lives in the JFrog Platform and your governance is about artifacts, vulnerabilities and signed releases, AppTrust is built for that. Add TestWarden when your verification obligations live in Jira: requirements, test attempts, defects and release coverage. If you only need artifact-centred policy gates, you likely do not need TestWarden.

Quality Clouds alone, or TestWarden alongside it

For a Salesforce-centred team whose governance is platform quality, Quality Clouds may be all you need. TestWarden is relevant when the same team also needs a Jira-resident decision across suites of evidence.

Instead of TestWarden

Choose none of the above, and not TestWarden, if you do not use Jira Cloud, if no change needs formal evidence obligations, or if you require customer-authored contracts or verified customer production references today: those remain outside the listed Marketplace V1.

How costs layer

TestWarden is priced for the Jira site: free for sites of up to 10 Jira users, a 30-day evaluation for larger sites, then user-tiered Standard (see pricing). It is an additional subscription alongside the tools you keep. TestWarden does not bill or resell subscriptions for any tool named on this page. Whether the layering is worth it is a question for your own review; this page quotes no competitor prices.

Twelve evaluation questions

Use these with any vendor, including us.

1. Which repository, commit, build and environment does a given pass refer to, and can the tool show that binding?
2. What happens when evidence is stale, incomplete, or about a different build? Does the gate fail closed?
3. If a failed run is followed by a passing one, can you still see the failure?
4. Who or what is allowed to change the decision, and is that change recorded?
5. Can an AI agent that authored the change influence the decision about it?
6. Where does the evidence come from, and is it independently collected or asserted by the pipeline itself?
7. How are test edits handled: does an old result silently point at a new test version?
8. What does the tool require from your existing stack, and what does it not touch?
9. Which capabilities are released, in development, in source, or planned, and where is that written?
10. What is the audit record, who can read it, and how long is it kept?
11. What does the full cost look like when layered with the tools you keep?
12. What would we need to do to leave, and what would we take with us?

Request a guided pilot

The fastest way to test the claims on this page is a guided pilot or early-access conversation. We will walk through the Warden workflow with your stack in mind, discuss whether TestWarden should sit above your current tools or whether you need something else, and plan guided onboarding, migration and pricing with you. Marketplace V1 is publicly listed on Atlassian Marketplace; the optional developer pilot does not include live third-party provider connectors or a complete live AI experience. Get TestWarden on Marketplace, or read the developer documentation first.

Method, sources and non-affiliation

This page was written and checked on 2026-10-28. Vendor descriptions use each vendor’s own documentation, read on that date; sources are listed below with notes about what each supports. TestWarden statements come from this site’s Proof and integrations pages and an internal engineering acceptance record dated 2026-10-27. Where those differ, the more conservative public statement governs and newer development proof is labelled as such. Products change; confirm current details with each vendor.

Harness, SonarQube, SonarSource, JFrog, AppTrust, Artifactory, Xray, AIO Tests, Quality Clouds, Salesforce, Atlassian, Jira and other names are trademarks of their respective owners and appear only as text to identify products. TestWarden is an independent product of Alt Datum. It is not made, endorsed, certified or sponsored by any of them. This page contains no customer testimonials, benchmarks, rankings or conversion figures, because none are available.

Objections and answers

We already have quality gates in SonarQube and approvals in Harness. Isn’t this redundant?

It would be if both of those answered whether the full verification obligation for this exact change was met. A Sonar gate answers a question about analysis results; a Harness approval records that a gate was passed or a person agreed. Neither inherently checks that required test evidence is complete, fresh and bound to the build. If your process already enforces that elsewhere, you may not need TestWarden.

Doesn’t JFrog AppTrust already do evidence-based gates?

Yes. Its documentation says it embeds evidence-based policy gates and issues a signed attestation for trusted releases. If your governance is artifact-centred and you are on JFrog, it covers much of that ground. TestWarden’s focus is Jira-resident verification obligations for the exact change, including test attempts and requirements.

Will this replace Xray or AIO Tests?

No. The assignment of this product is to sit above them. Native test management is optional and exists for teams that want one place. Migration is optional and CSV-based, and it brings test cases rather than historical executions.

Can an AI agent approve its own pull request here?

No. The policy decides from evidence bound to the build. Agents can propose and dispatch authorized verification, and the decision is read back independently.

Is the Quality Memory trained on our data?

No. It is a tenant-isolated history that grounded features can retrieve. It is not model weights trained on yours or anyone else’s data.

Is it available now?

TestWarden is publicly listed on Atlassian Marketplace for Jira Cloud. The proof published here comes from development installs and tests, not independently verified customer production deployments; direct connectors and optional developer capabilities remain separately staged.

Does TestWarden make us compliant or certified?

No. It produces a decision record and history that can support your own process. It carries no SOC 2 or ISO certification claim and does not guarantee that software is free of defects.