Runbook: move Xray test cases into TestWarden with CSV

To migrate Xray test cases into TestWarden, get your manual tests into a CSV the Xray Test Case Importer source understands, upload it in the Migration Center, read the preview, import in the background and reconcile the report against Xray. The adapter understands two file shapes and brings across the test cases themselves: summary, structured steps, priority, labels and preconditions. It does not bring executions, and that single fact shapes every planning decision in this runbook.

This is a runbook for engineers and QA leads who have decided, or are seriously considering, to make TestWarden their Jira test manager. If you only want to keep Xray and add independent verification above it, nothing here is required; skip to the distinction between CSV transfer and live integration. TestWarden is an independent, policy-driven verification and intelligence layer that sits above test-management tools, CI systems, scanners and coding agents, and customers can keep using Xray alongside it. The same Jira app also includes full native test management, which is the destination for the cases this runbook moves.

The walkthrough above uses invented rows to follow a file from export to reconciliation, shows how Xray columns map into TestWarden, and lets you filter a coverage matrix by what transfers, what is kept as a custom field, what you must rebuild and what does not transfer at all. The prose below states the same facts in full.

What this migration is, and what it is not

A migration runbook is only useful if it is honest about its boundary. Here is the boundary in one table.

Nothing in the migration modifies Xray. It reads a file you exported, so your source of truth is untouched, and you can run the whole process as a rehearsal and discard it.

What the Xray adapter accepts

The Migration Center has a source called Xray Test Case Importer CSV. It reads two shapes of file, and knowing which one you have saves most of the troubleshooting.

• The Test Case Importer layout. One row per manual step. The test is identified by Issue Id (or TCID, or Issue key), test-level columns appear on the first row (Test Summary, Test Priority, Labels, Description, Precondition, Test Repository Path) and the step columns are Action, Data and Result. Rows that share an identifier fold into one test with several steps.
• A Jira CSV export of Test issues that includes the Manual Test Steps field. That field contains JSON describing the steps. Use it, and not the HTML-rendered variant of the export: the importer rejects HTML with an explicit error rather than guessing.

Cucumber and Generic tests that carry a Definition column keep that script as the steps text. Priority is mapped, labels become tags, and preconditions are appended to the description. Every unrecognised column is preserved as a custom field named Xray: <column>, so a column you forgot you had is not silently dropped. That behaviour is deliberate: a migration that quietly discards data is worse than one that keeps too much.

Xray documents the CSV layout it expects on import in its Test Case Importer examples, which shows one row per step sharing an issue id and repeating only action, data and result. If you already hold files in that layout, you can skip the export step. If you do not, the Jira export route below produces a usable starting point.

Step 1: export from Xray

To produce a file from an existing Xray project, filter the Test issues you want in Jira, then export from the Jira issue view. Atlassian documents the procedure in Export issues from Jira Cloud in CSV format: open the list, use More actions, then Export, and choose the all-fields CSV. Add or keep the Xray Manual Test Steps column so that the steps are in the file. Jira writes UTF-8, so inspect the file in a UTF-8-aware editor.

Choosing what to export

Use JQL to decide the unit of work. Splitting by project is the most natural boundary; splitting by label or by component works when one project is large. Each file becomes one migration job, which keeps reconciliation tractable: you compare the imported count with the number of Test issues returned by the same JQL, and any difference has a short list of possible causes. A single enormous file concentrates every problem into one hard-to-read report.

Decide also whether to include archived or obsolete tests. A migration is a rare chance to leave behind tests nobody has run in two years. Many teams export only the tests that appear in a recent plan or carry a recent execution, then keep the remainder in Xray read-only. That decision belongs to a QA lead, not to the importer, and it is far easier to make before the import than after.

Step 2: prepare the file and the account

• Role. You need the QA Admin role in TestWarden. Migration is admin-only.
• Size. One file per job, up to 10 MB and 10,000 rows. In the importer layout a test with many steps spans many rows, so row count can be several times the test count. If you are near the limit, split by JQL.
• Detectable columns. Open the CSV and confirm there is a summary column and a step, definition or Manual Test Steps column. Without one of those the adapter reports that the file does not look like an Xray test export, which is the correct and helpful outcome for a wrong file.
• Encoding and line breaks. Multi-line step text inside quoted cells is normal. Opening and re-saving in a spreadsheet application can change quoting or encoding, so keep an untouched copy of the original export.

Steps 3 and 4: map, preview and import

1. Open Migration in TestWarden, choose Xray Test Case Importer CSV and upload the file. The mapping is detected for you, so there is usually nothing to configure.
2. Read the preview: Will import, Will skip, Total rows, the reason for every skipped row and a sample of mapped tests. Check that grouped step rows came out as one test with several steps rather than several one-step tests.
3. Commit. The import runs in the background in batches of 200 with checkpoints. You can close the panel and it resumes. Re-previewing the same file and mapping reuses the job instead of duplicating rows.
4. Reconcile with the report, as described in the next section.

Because the adapter detects the mapping itself, the preview is where your judgment goes. The three most informative things to look at are the skipped rows, the sample of a test with many steps, and the custom fields. A skipped row is usually a row without a summary or a stray continuation line. A many-step test shows whether grouping worked. The custom fields confirm that Test Repository Path and any other unrecognised columns were preserved as Xray: <column> rather than lost.

Cucumber and Generic tests

If your Xray project mixes manual tests with Cucumber or Generic tests, expect those to arrive differently. Where a Definition column exists, its content is kept as the steps text; the importer does not attempt to parse Gherkin into structured steps. For a Cucumber scenario that is usually the right outcome, because the scenario text is the test. But it means such tests will show a script block in the steps area, and you should decide whether automated tests belong in a test manager at all once their results flow from CI. Often the more durable answer is that automated tests live in the repository, and TestWarden receives their results as evidence for the exact build.

Step 5: reconcile

Reconciliation is the evidence that a migration happened correctly. In the report, filter by Imported, Imported with warnings, Skipped and Failed.

• Count. Compare the imported count with your Xray count for the same JQL. Differences should be explained by skipped rows you agree with, not by surprise.
• Side by side. Open a handful of tests with many steps and compare each one with Xray. Check step order, data and expected results. Many-step tests are where grouping errors hide.
• Warnings. Each imported-with-warnings row is a decision. Accept it, or fix the value after import with bulk edit.
• Custom fields. Confirm that the Xray columns you care about appear as custom fields on a sample of tests.

Step 6: structure what the CSV cannot carry

• Folders. The Test Repository path is kept as a custom field. Recreate the folders and tags in the shared Library, then use bulk actions to file tests, filtering on that field.
• Sets and plans. Rebuild Test Sets and Test Plans as TestWarden plans and cycles built from the imported library. Do not recreate every historical set; recreate the ones you will run next.
• Requirement coverage. Re-link tests to Jira issues. Coverage is then computed in Jira from the links, rather than imported from a file whose links may have gone stale.
• Automation. Decide which tests are executed manually in TestWarden and which are covered by automated CI evidence submitted to The Warden.

Once cycles start, every attempt is version-pinned from the first run, so editing an imported test during a cycle does not rewrite what earlier attempts verified. That is one of the practical reasons teams use migration as an opportunity to tidy tests that were previously too risky to touch.

What to do with old execution history

Historical Xray executions are not imported, and nothing in the migration changes them. Keep Xray readable for as long as you need those records, or archive them before you let the subscription lapse. Test Executions are Jira issues, so the Jira CSV export captures their issue-level fields. Check the file for the result detail you require, because per-step results may not appear in it.

Plan the cutover as a dated event. Pick a day after which new attempts are recorded in TestWarden, write it down, and keep Xray access until your retention period for older releases has passed. For a release audited a year later, the answer to “where is the evidence” is then unambiguous: before the date, in Xray; after the date, in TestWarden’s Quality Memory, where decisions, evidence receipts and attempts are retained per tenant and append-only.

CSV transfer is not a live Xray integration

It is easy to confuse two things people call “Xray integration”. The CSV migration copies manual test cases once. A live integration reads results from Xray’s API while Xray remains your system of record. TestWarden’s Xray Cloud collector is the second kind: a read-only collector that runs in your own CI, authenticates to Xray Cloud with its GraphQL API using credentials you hold in your CI secret store, and produces evidence that an attestor signs and The Warden evaluates for an exact build.

Be careful about what that means today. The Xray collector passes documented-contract HTTP tests and a signed offline HELD-to-repaired-CLEARED test, and its live vendor certification is pending; no vendor is certified. The Forge application does not store your Xray credentials, and the collector needs only read access. Do not describe it to colleagues as a shipped, certified Xray integration. If your aim is “keep Xray, add independent verification”, you can also do that today by having CI submit the JUnit XML that your automated tests already emit. The Integrations page tracks the status of each provider, and Warden evidence for developers explains the submission formats.

Troubleshooting the preview

Most failed or surprising previews trace back to a short list of causes. Work through them in this order, because the early ones are the most common and the cheapest to fix.

A final habit worth adopting: change one thing at a time and re-preview. Because re-previewing the same file and mapping reuses the job, a preview is cheap, and the fastest route to a clean import is a handful of small corrections rather than one large rewrite. When the preview reads the way you intended, the import itself is the least eventful step in the whole process, which is exactly how a migration should feel.

Frequently asked questions

For the decision itself, see the Xray alternative comparison and choosing test management for Jira Cloud. Sibling runbooks cover AIO Tests and Zephyr Scale, and the Migration Center overview lists every supported source. To try it on one project, get in touch or open the Pricing page to see what an evaluation involves; the preview shows what will happen before any test is created.