Version-pinned test execution for QA leads

A test result is only meaningful relative to what was tested. If a test case is edited after it was executed and the old result now points at the new wording, your record says something that never happened. Version-pinned test execution prevents that: each attempt keeps a reference to the exact, immutable version of the test it ran, so history stays true no matter how the test evolves afterwards.

This is a data-integrity property, and it is easy to underrate until it fails. Teams discover it during an audit, an incident review or a release dispute, which is the worst time to learn that “passed” on the fourteenth meant something different from what the page now says. This article explains the failure precisely, shows pinning in an interactive timeline, describes what it changes for QA leads, states where it stops, and gives you a five-minute trial to run against any tool, including ours.

In the timeline above, a test is created and executed on Monday, edited on Tuesday, executed again on Wednesday, and Monday’s result is opened on Thursday. Switch between a result linked to the live test and one pinned to a version, and open each execution. The text below states what you are seeing, so it does not depend on the interaction.

The failure, in four lines

1. Monday: a tester runs “Refund a partial payment” in the release cycle. Step 3 says “Check the confirmation email”. It passes.
2. Tuesday: a lead rewrites step 3 to “Check the confirmation email and the ledger entry”.
3. Wednesday: the release manager opens Monday’s result. If the tool shows the live test, the attempt now appears to have verified the ledger entry.
4. Nobody checked the ledger on Monday. The record is wrong, and it is wrong in the reassuring direction.

Notice the direction of the error. Records that drift toward saying less than was done are an annoyance. Records that drift toward saying more than was done are a liability, because the person relying on them is being told a check happened that did not. And the drift is invisible: nobody touched the result. The test changed under it.

This happens in any tool where a run links to the living test rather than to a snapshot. It is not a flaw peculiar to one product; it is a design choice with consequences, and different tools make different choices, sometimes with workarounds. You should not assume your tool’s behaviour in either direction. You should test it, which is what the last section helps you do.

Three ways a tool can relate an execution to a test

Copy-per-cycle is the model many teams grew up with, and it does protect the past, but at a price. A shared test case is the whole point of a library: one definition used by many plans. When every cycle holds its own copy, correcting a step means choosing between leaving old copies wrong and editing dozens by hand. Teams adapt by freezing the library during a release, which is exactly the behaviour a good tool should make unnecessary. Pinning gets the benefit of both: one shared definition that can evolve, and attempts that never change.

What pinning means in TestWarden

• Test cases have version history with archive and restore. An edit creates a new version.
• Plans group tests; cycles schedule them. Starting an attempt binds it to the test version current at that moment.
• Each re-run is a new attempt on the same case. Earlier attempts stay in the attempt history with their own steps and results.
• A contextual Jira Bug raised from a failed step carries the test, the step and the attempt, so the defect points at what actually ran.

Three consequences follow from this structure, and each is a thing a QA lead can say out loud. First, a result is a fact about a version. “Passed” is never a property of a test in the abstract; it is a property of one attempt against one version. Second, history is additive. A re-run does not overwrite the last result, so the story of a flaky test is visible as a sequence rather than reduced to its latest colour. Third, a defect is anchored. Because the bug carries the attempt, anyone opening it later sees the steps that failed, not whatever the test says today.

Vocabulary that keeps conversations precise

• Test case. The shared definition in the library. It evolves.
• Test version. An immutable snapshot of a test case at a moment. Versions do not change.
• Attempt. One execution of one test version, with its results, who ran it and when.
• Cycle. A scheduled set of tests for a purpose, such as a release candidate.

What it changes for a QA lead

• You can edit tests during a cycle. Fixing a typo or tightening a step no longer risks corrupting past results, so teams stop freezing the library.
• Re-test decisions are visible. If a test changed after it last passed, the history shows it, and you can decide whether a re-run is needed.
• Coverage statements hold up. Requirement and release coverage built on attempts refers to what was executed, not to the present state of the library.
• Disputes have an answer. “What exactly did we run on the 14th?” is a lookup, not an investigation.
• Handovers are safer. A new lead can read why a result was recorded without trusting that nobody edited the test since.

A concrete example of the second point: a test passed in Cycle 3. Between Cycle 3 and Cycle 4, someone strengthened its expected result. In Cycle 4 the test is due to run again anyway, but in a different situation, say a hotfix cycle that includes only tests marked unchanged, the history tells you the passing result is for an older, weaker version. Without pinning, the test looks stable and it is silently untested against its current definition.

A worked example: the release dispute

Here is how the absence of pinning typically surfaces, using an invented company. A payments team ships release 5.2 on the fourteenth. A month later a customer reports that refunds of partial payments sometimes leave the ledger out of step. The incident lead opens the release cycle and finds “Refund a partial payment: Passed, 14th”, with a step that reads “Check the confirmation email and the ledger entry”.

Everyone relaxes: the test covered the ledger and it passed, so the defect must be a rare race. The investigation spends two days on concurrency. Then someone notices from the audit log that the ledger wording was added on the sixteenth. On the fourteenth the step said only “Check the confirmation email”. The ledger was never checked before release. The two days were spent chasing a theory the record had wrongly ruled out.

With pinned execution, the attempt from the fourteenth shows version 1, whose step 3 has no ledger check, and the incident lead sees it in the first minute. The wording change on the sixteenth is version 2, with its own attempts, if any. The cost of the wrong record was not the test; it was the misdirected investigation. That is the practical argument for treating execution history as evidence rather than as a convenience.

What a pinned attempt must store

It is useful to be specific about what a trustworthy attempt record contains, because it gives you a way to question any tool without relying on its marketing vocabulary. Whatever the implementation, a pinned attempt should let you recover each of the following without guessing.

• The exact test version. A version identity, not a pointer to the live test. From it you can read the steps, data and expected results as they stood.
• Per-step results. Which step passed, failed or was skipped, with any note, so “Failed” is never a single colour for a long test.
• Who and when. The person or system that ran it and the time, which makes the attempt attributable and contemporaneous.
• Context. The cycle or plan it belonged to, and, where relevant, the build or release it was run against.
• Linked defects. Bugs raised from a step, each pointing at the attempt and the step.

Notice that nothing on this list is exotic. It is the minimum for a record to be attributable, accurate and original. Tools differ mainly in whether they store the version identity or only a pointer, and in how easily you can see it in the interface.

Pinned manual attempts and CI evidence are different things

QA leads sometimes ask how pinned manual attempts relate to automated results from CI. They are different kinds of record serving different questions. A pinned attempt answers “what exactly did a person or tool execute, and what did the test say?”. A CI result submitted to The Warden answers “is the evidence for this exact build complete, current and sufficient?”. The first lives in test management, the second in the verification layer.

TestWarden keeps them separate on purpose. Manually recorded executions are not currently inputs to Warden decisions, which evaluate the evidence your pipeline submits. That is not a hierarchy of importance. Manual exploratory testing, usability checks and edge-case investigations are valuable and belong in test management. But a release decision that depends on a result must be able to say which build it belongs to, and manual attempts do not carry the build identity that automated runs do. Keeping the two honest and distinct is better than blending them into a score. The reasoning is developed in test management versus release governance.

Questions to put to any vendor, including us

1. When a test is edited after an attempt, what does the old attempt display, and where can I see the version it ran?
2. Is a re-run a new attempt or an overwrite? Show me both attempts on screen.
3. If I archive or delete a test, what happens to its attempts?
4. Can I correct a typo in a shared test without changing the record of past cycles?
5. Does a defect raised from a failed step identify the attempt?
6. What is imported when I migrate, and does it include execution history? If not, where will it live?

Ask for answers on screen with your own data, not in a document. Several mature Jira test-management products make deliberate decisions here, and you should not assume that any of them is better or worse than another without trying the five-minute check below. The point of the questions is to replace a feature-checklist tick with an observed behaviour.

Why this is an integrity property, not a convenience

Regulated industries have long had a vocabulary for records that can be trusted. Data-integrity practice, often summarised as ALCOA, asks that a record be attributable, legible, contemporaneous, original and accurate. You do not need to work in a regulated industry to see why those words apply to test history. A pinned attempt is attributable (who ran it), contemporaneous (when), original (the steps as they stood) and accurate (what it says matches what happened). A live-linked result can fail the last two without anyone doing anything wrong.

The same argument matters for evidence in AI-assisted delivery. When agents propose changes to tests as well as to code, the question “which version of this test did the passing result come from?” stops being academic. A pinned history is what lets you notice that a green result belongs to a version an agent had edited, and decide what that means for your confidence. It is also one of the foundations of Quality Memory, where Test Version is a first-class node in the chain from requirement to outcome.

Where pinning stops

Pinning protects the link between an attempt and the test text. It does not tell you whether a green result is enough for a given build. That is a separate question, handled by The Warden from evidence submitted by your pipeline: is the evidence complete, current, attributable to this exact build and sufficient under the contract? A beautifully pinned manual attempt can still be the wrong evidence for a release decision.

• Manual executions are not Warden inputs today. Manually recorded executions are not currently inputs to Warden decisions. The Warden evaluates evidence your CI submits.
• Imported tests bring no old executions. Migration brings test cases, not results, so pinning starts with your first attempt after migration. See the Migration Center.
• Pinning does not make a weak test strong. It preserves what ran. It cannot say whether what ran was meaningful.

This separation is deliberate. Test management records what people and tools did. Release governance decides whether the combined evidence is sufficient for an exact change. They need each other, and conflating them is how teams end up treating a tidy results page as proof. The distinction is explored in test management versus release governance.

A five-minute check for any tool

If the answers are no, treat historical results in that tool as soft. That is a finding about the tool’s data model, not about your team. For a wider evaluation method, see choosing test management for Jira Cloud, and for what an auditor will look for, audit-ready Jira test evidence.

Adopting pinned execution in a team that does not have it

1. Stop freezing. If the library is frozen during releases because edits corrupt history, say that you will lift the freeze once you have a pinned model. It is the visible win.
2. Define when edits trigger re-test. Pinning shows that a test changed after it last passed. Agree which kinds of change, such as a new expected result, require a re-run.
3. Tie defects to attempts. Make it a habit to raise bugs from the failing step so they carry the attempt.
4. Preserve old history separately. If you migrate from another tool, archive its executions and record the cutover date. Pinning starts with the first new attempt.

See what the model looks like in practice on the product page, and the test management for Jira overview. If you want to run the five-minute trial on your own data, contact us or see pricing, where the free tier covers up to ten Jira users.

Frequently asked questions