jaktestowac/tracing-requirements-to-code

Tracing Requirements to Code

Use this skill when the tests already exist and the question is what they actually verify.

Traceability is usually taught forward: take a specification, derive requirements, plan tests. That is requirements-test-coverage-mapper, and it is the right skill when you start from a document. This skill handles the other and more common situation: a live codebase, hundreds of tests, requirements scattered across a tracker and three documents, and no reliable statement of which test proves what.

The insistence that makes this worth doing: a traceability link is a claim, not a record. A test tagged @req REQ-014 claims to verify REQ-014, and nothing about writing the tag makes that true. An unverified matrix is worse than no matrix, because it converts ignorance into documented confidence, and it is exactly the artifact people reach for when deciding what not to test.

When to Use

• a suite exists and nobody can say which requirement a given test serves
• someone asks "which tests cover this requirement" and the answer takes a day
• a matrix was built once, was accurate that week, and has not been true since
• a requirement is changing and its affected tests must be found
• tests exist that nobody can trace to any requirement
• an audit, certification, or customer questionnaire needs coverage evidence

Operating Principles

• Traceability runs both ways. Requirement to test finds coverage gaps. Test to requirement finds orphan tests, and nobody looks in that direction.
• A link is verified or it is a guess. The test must fail when the requirement's behaviour is broken. Until it has been seen to, the link is unconfirmed and labelled as such.
• Extract from evidence, not from titles. A test called "validates the discount rule" is a hypothesis about what it does. The assertions are the evidence.
• The matrix lives in the code, not in a spreadsheet. A link stored beside the test moves when the test moves. A link in a document rots the first time anyone refactors.
• Orphans are findings in both directions. An uncovered requirement is a coverage gap. An orphan test is either an undocumented requirement or a test of nothing, and both are worth knowing.
• Drift is the default. Links rot silently through renames, deletions, and refactors. Without a CI check, the matrix is accurate on the day it is written and decreasingly true afterwards.

Workflow

Phase 0: Establish the requirement source

Before touching tests, fix what a requirement is in this project and where the authoritative list lives:

• an issue tracker with stable keys (PROJ-1234)
• acceptance criteria inside stories
• a PRD or specification document
• a regulatory or contractual requirement list
• none of the above, in which case the first deliverable is a requirement inventory derived from the code and the tracker, marked as derived rather than authoritative

Record the ID scheme, and whether IDs are stable. An ID that changes when a ticket moves project is not usable as a link target, and discovering that after annotating 400 tests is expensive.

./resources/id-and-annotation-conventions.md covers choosing a scheme that survives.

Phase 1: Extract what the tests actually verify

Work through ./resources/extracting-an-rtm.md. The method, in short:

1. Inventory the tests. Every test, its file, its title, its level.
2. Read the assertions, not the titles. What behaviour would have to break for this test to fail?
3. Group by behaviour, not by file. Several tests often serve one requirement, and one god test sometimes serves five.
4. Map to requirements where the mapping is defensible.
5. Label the confidence of each mapping: stated (the test or its commit names the requirement), inferred (the behaviour clearly matches), guessed (it plausibly relates).

Do not silently promote a guess. A matrix that distinguishes stated from guessed is honest and usable; one that flattens them is a document nobody should trust and everybody will.

Phase 2: Annotate at the source

Move each defensible link into the code, next to the test it describes.

test('rejects a card that expired last month @req:REQ-014', async ({ page }) => { ... });

Per-runner spellings, the machine-readable formats, and the trade-offs between tags, annotations, and comments are in ./resources/id-and-annotation-conventions.md.

Rules that decide whether this survives:

• one canonical form, chosen once and used everywhere, so a generator can parse it
• machine-readable, because a convention nothing checks is a convention that decays
• at the test level, not the file level, unless every test in the file genuinely serves the same requirement
• many-to-many is normal: one test may serve several requirements and one requirement needs several tests

Phase 3: Verify the links

The phase that separates this from bookkeeping. For each link, and at minimum for every link on a high-risk requirement:

1. Break the behaviour the requirement describes.
2. Run the linked test.
3. Confirm it fails, and that the failure names something recognizable.
4. Record the link as verified with the date.

A link that survives step 3 with a passing test is a false claim in the matrix. Fix the test or remove the link; do not leave it as documented coverage.

This is the same standard unslop-tests and migrating-tests-to-playwright apply, for the same reason: a test that has never been observed failing has unmeasured value, and a matrix built on such tests inherits that.

Verifying every link in a large suite is not affordable. Verify by risk, record the coverage of the verification itself, and state what was taken on trust.

Phase 4: Find the orphans

Cross the two directions and take all four quadrants seriously. ./resources/orphan-and-drift-detection.md has the triage.

| | Has a test | No test | | --- | --- | --- | | Has a requirement | Covered, verify the link | Coverage gap | | No requirement | Orphan test | Out of scope |

• Coverage gap: a requirement nothing verifies. Route to designing-functional-tests or requirements-test-coverage-mapper.
• Orphan test: the direction nobody looks. It is one of three things, and the distinction matters:
  • an undocumented requirement, which is the valuable case. The behaviour matters, someone tested it, nobody wrote it down. Write it down.
  • a test of a removed feature, which should be deleted.
  • a test that proves nothing, which unslop-tests should judge.

Reporting orphan tests as a single count wastes them. Classify each one.

Phase 5: Automate and gate

A matrix maintained by hand is a matrix that is true once. ./resources/traceability-automation.md has a generator that parses annotations and emits the matrix, plus the CI gate.

Enforce, in increasing order of strictness, and adopt them in this order:

1. Every annotation references a requirement that exists. Cheap, catches typos and deleted tickets immediately.
2. Every high-risk requirement has at least one linked test. Scoped to a declared critical set rather than everything.
3. New tests carry an annotation. Applied to changed files only, so the backlog does not block every pull request.
4. No requirement loses its last link without an explicit acknowledgement.

Introduce a gate against changed files first. A gate that fails on a pre-existing backlog of 300 unannotated tests gets disabled in a week, and then nothing is enforced at all.

Phase 6: Handle change

When a requirement changes, the matrix is what tells you what to revisit. The procedure is in ./resources/traceability-automation.md:

• find every test linked to the changed requirement
• classify each: still valid, needs updating, now wrong
• update the tests and the links in the same change
• when a requirement is removed, its tests are deleted or re-linked, never left pointing at nothing

Requirement changes are the main source of drift, and they are the moment traceability pays for itself. Hand the wider retest question to analyzing-regression-scope.

Phase 7: Report

./resources/traceability-automation.md has the report shape. It states:

• coverage per requirement, with the link confidence and verification status
• uncovered requirements, ranked by risk
• orphan tests, classified rather than counted
• links that could not be verified, and why
• drift found since the last run
• what the matrix does not cover, which is the line that keeps it from being over-quoted

Common Failure Modes

• a matrix in a spreadsheet, accurate the day it was written, quietly wrong within a month
• links inferred from test titles, so a misleadingly named test becomes documented coverage
• stated and guessed mappings flattened into one column
• links recorded and never verified, converting ignorance into documented confidence
• orphan tests reported as a number rather than classified
• annotating in a format nothing parses, so nothing checks it
• a CI gate switched on against the whole backlog, then disabled and never restored
• a requirement edited without touching the tests linked to it
• treating full traceability as achievable everywhere, rather than mandatory on the risky part and best-effort elsewhere

Resource Map

• ./resources/id-and-annotation-conventions.md - choosing a stable ID scheme, per-runner annotation spellings for Playwright, Vitest, Jest, and Cucumber, and the many-to-many cases
• ./resources/extracting-an-rtm.md - building the matrix from an existing codebase: reading assertions rather than titles, confidence labels, and a worked extraction
• ./resources/orphan-and-drift-detection.md - the four quadrants, orphan-test classification, the ways links rot, and the triage table
• ./resources/traceability-automation.md - a generator script, the staged CI gate, the change-impact procedure, and the report template

Related Skills

• requirements-test-coverage-mapper - the forward direction: designing coverage from a PRD before the tests exist. Use that one to plan, this one to verify what was built.
• verifying-acceptance-criteria - when the question is whether a specific build meets its criteria, rather than which test covers what
• analyzing-regression-scope - when a requirement change needs its full retest blast radius, beyond the linked tests
• unslop-tests - when an orphan test needs judging on whether it proves anything
• designing-functional-tests - when an uncovered requirement needs its tests designed
• documenting-test-suites - where the ID convention and the matrix's location belong permanently
• assessing-release-readiness - which consumes the uncovered-requirement list as release evidence

Definition of Done

This skill is complete when:

• the authoritative requirement source and its ID stability are recorded
• every mapping carries a confidence label of stated, inferred, or guessed, and guesses are not promoted silently
• defensible links live in the code beside their tests, in one canonical machine-readable form
• every link on a high-risk requirement has been verified by breaking the behaviour and observing the test fail, with the date recorded
• links that could not be verified are labelled unverified rather than omitted
• both directions are crossed: uncovered requirements ranked by risk, and orphan tests classified as undocumented requirement, dead test, or proves-nothing
• a generator produces the matrix from the annotations rather than a human maintaining it
• a CI gate is enforced against changed files at minimum, and the staged escalation is recorded
• the change procedure updates tests and links together
• the report states what the matrix does not cover