We hold TestWarden to its own standard.

VERIFYING → HELD on a failing JUnit observation, then VERIFYING → CLEARED after fresh evidence for the fixed build

Real GitHub Actions runs against TestWarden’s own repository submitted evidence to the development installation. The HELD decision remained in history after the CLEARED decision.

Exact retry of an evidence submission returns the identical decision (idempotent)

Proven at the real CI boundary.

Evidence payloads carrying a caller-selected state, progression flag or approval are rejected

Contract tests on the App REST API routes.

Stale, wrong-build, superseded-build, conflicting and overdue-run evidence cases

Scenario tests. These cases have not been exercised in a live installation.

JUnit XML accepted over the App REST API with OAuth 2.0 (3LO) custom scopes

A site administrator completed 3LO consent with the custom evidence scopes; token rotation and persistence succeeded on repeated attempts; real GitHub Actions runs submitted JUnit evidence.

SARIF 2.1.0 and generic JSON normalised into receipts

Covered by envelope and contract tests. A live SARIF or JSON submission from a real producer has not been run.

App REST reads require a current Jira role and a repository grant; evidence writes require a site administrator holding the write scope

Covered by tests. Live runs with a non-admin user and with a second installation have not been done. Write authority is installation-wide for that administrator; per-repository write grants do not exist.

Server-side role mapping (Viewer, Tester, QA Manager, QA Admin) with fail-closed defaults

Implemented and covered by tests; a live smoke of Admin writes with multiple user identities is still to be completed.

Create, edit (versioned), clone, archive and restore tests; add, duplicate, reorder and delete steps; folders, tags, saved filters, bulk tag/move/archive

Exercised on the development site with disposable data.

Cycles composed from repository selection and tags; version-pinned attempts; attempt history; Run again

Step-level Failed disables overall Passed; Pass all remaining; history viewer.

Jira Bug created from a failed step with cycle, release, environment and attempt auto-filled; link persists after reload

A real issue was created in the development Jira project.

Requirement coverage for a linked Jira issue and release coverage with a cycle picker

Derived from recorded executions and requirement links.

Generic CSV: upload, column mapping, preview, background commit, fidelity report

Including a 900-row preview and multi-column samples with skipped-row reporting.

Zephyr Scale CSV: automatic mapping detection, preview, commit, fidelity

Run end to end on a development Jira site against a documented Zephyr Scale export-layout fixture. Not every Zephyr edition or export has been tested, and no customer export has been run, so preview your own file first.

Xray Test Case Importer CSV, and Jira/Xray CSV with Manual Test Steps JSON: detection, preview, commit, fidelity

Run end to end on a development Jira site against documented Importer-layout fixtures, including a semicolon-delimited layout. Not every Xray edition or export has been tested. Test Sets, Plans, Executions, Preconditions and attachments are disclosed as not imported.

AIO Tests export through the Generic CSV mapping

There is no AIO-specific importer and no AIO fixture has been verified. Data can arrive through the Generic CSV mapping flow; preview your own file first.

Checkpoint/resume and exactly-once row delivery under concurrent duplicate batches

Single-pass completion with checkpoint rows was observed live; resume after interruption is covered by automated tests.

Free, evaluation, active and inactive (read-only) states

Marketplace licensing is enabled in the manifest; live license-state proof on a clean site is pending.

Repository list at 50,000 rows and 10,000-row migration jobs

Benchmark harness only. No performance number is published as a live result.

Developer API (six operations), PowerShell client and MCP server read the same Warden decision

Implemented in source and deployed to an isolated development installation (version 2.3.0); the deployment script’s CI run passed 2,376 tests with none failing or skipped. Live client acceptance needs owner 3LO consent, App REST enablement and non-admin and second-installation sessions, and has not been run.

Failing build → HELD with an actionable reason → repair → CLEARED, driven by an independent verifier

Blocked until the verifier workflow and its credentials can be protected with enforceable repository controls. Earlier HELD and CLEARED pairs in the development installation predate this pilot installation.

Optional AI explanation of a decision (Forge llm module)

Response handling and fail-closed behaviour are covered by tests. A real, budgeted model answer with citations has not been produced, because an administrator has not yet enabled and invoked it in the Jira UI.

Scoped developer pilot readiness

Current verdict: BLOCKED, with acceptance not run. The full-release gates (authentic changes, governed repairs and external-team acceptance) remain open.

Recorded manual executions as inputs to Warden decisions

The Warden currently decides on automated CI evidence. Manual execution results are kept in test management but are not evidence inputs.

Customer-authored verification contracts, approval and waiver options

Only the default contract is available to customers today.

Direct result adapters for Playwright, Cypress, Semgrep or Sonar

Evidence arrives as JUnit XML, SARIF 2.1.0 or JSON. Tools that can write those reports are compatible; they are replaceable producers, not first-party adapters.

Read-only connectors for AIO Tests Cloud, Xray Cloud, Zephyr Scale Cloud, TestRail Cloud and Azure DevOps Test Plans

Implemented in source as customer-CI collectors that write a JUnit report, with mocked-API tests only. The most recent integration review (2026-10-25) still reproduced failing tests, including malformed-UTF-8 handling and error disclosure in the TestRail and Azure DevOps path, and these are being corrected. No vendor account, real provider result or customer connection has been used. Nothing here is released.

Independent attribution of a provider result to the exact build

A signed-attribution preflight exists as a library with tests. No independent attestor, key custody, revocation or server-side verification is in place, so provider-derived evidence stays asserted by CI.

October 23 engineering release candidate

Hosted CI passed on the exact source, with 1,418 product tests and no failures, skips or errors. It is not released: the customer and Marketplace gates remain open, and it is separate from the version submitted to Marketplace.

Historical execution results and attachments from other tools

CSV migration brings test cases, steps and their metadata. Executions and attachments are not imported.

Immutable, per-tenant history of evidence receipts and Quality Decisions

The HELD to CLEARED sequence above stayed in history in the development installation.

Expanded historical ingestion and retrieval, Verification Intelligence analyses (coverage gaps, changed-code test impact, flaky-test and failure patterns, release risk) and Quality Agent proposals

Implemented in source in a draft integration, as advisory analyses. The Quality Agent runtime is deliberately blocked until authorized write and verification adapters exist. No real model usage, cost or acceptance has been recorded, and none of these is counted as an operational capability.

Autonomous repair and a live agent loop

No agent repairs software or grants clearance. Nothing learns from customer data, and no customer data is pooled across customers.