nicolas-codemate/qa-sync

QA Sync Skill

Keep docs/qa/*.md in sync with the codebase. This skill:

1. Bootstraps a test plan structure on first run.
2. Computes the application code diff since the last test plan commit.
3. Delegates analysis to the qa-writer agent.
4. Applies the agent's proposed operations (ADD / UPDATE / REMOVE) to the markdown files.
5. Refreshes docs/qa/README.md as an index.

This skill writes files. It does not commit them — the user reviews the diff and commits when ready.

Arguments

| Flag | Default | Effect | |---|---|---| | --from <commit> | auto-detected from git log -- docs/qa/ | Override the diff base commit | | --flow <name> | all flows | Restrict the sync to a single flow file (e.g., auth) | | --dry-run | off | Print the operations without writing anything |

Workflow

Step 1 — Bootstrap check

Read .claude/ticket-config.json (the existing project workflow config). Check whether it has a qa section. If .claude/ticket-config.json does not exist at all, instruct the user to run /init-project first and stop. Do NOT create it from this skill — that is /init-project's job.

If a qa section already exists, skip to Step 2.

If no qa section exists, run the bootstrap wizard. Do not ask blind questions: detect what you can, then ask only the things you cannot infer.

1a. Detect the project architecture

Run the detection probes from references/architecture-detection.md, then print the resulting one-block summary to the user before going further. If something essential is ambiguous, ask one targeted question via AskUserQuestion before proceeding — do not roll into Synthesis with a wrong picture.

1b. Auto-detect the base URL

Build a candidate list from the project:

1. Read compose.yaml / compose.yml / docker-compose.yml / docker-compose.yaml at the project root. For each service exposing ports: of the form <host>:<container>, compute http://localhost:<host>. Prefer (in order): a service named frontend, web, app, then the first service exposing an HTTP-looking port (80/443/3xxx/4xxx/5xxx/8xxx).
2. Read package.json (root and frontend/) for default dev-server ports:
   • Vite → 5173
   • Next.js → 3000
   • Astro → 4321
   • Nuxt → 3000
   • CRA → 3000
3. Read .symfony.cloud.yaml, Procfile, manage.py runserver defaults (8000), rails server default (3000) for additional hints.

Present the auto-detected candidates as the first options of the AskUserQuestion. Always keep "Autre" as the last option for manual entry.

AskUserQuestion:
  questions:
    - question: "Quelle URL utiliser pour la QA en local ?"
      header: "Base URL"
      options:
        - label: "<auto-detected #1>"
          description: "Détecté via <source: compose.yaml service `frontend`>"
        - label: "<auto-detected #2>"
          description: "Détecté via <source: package.json + Vite default>"
        # ... up to 3 detected candidates
        - label: "Autre"
          description: "Saisir manuellement"

If detection finds nothing (rare), fall back to the static list:

options:
  - label: "http://localhost:8000"
    description: "Stack PHP/Symfony classique"
  - label: "http://localhost:3000"
    description: "Stack Node/Next/Vite"
  - label: "Autre"
    description: "Saisir manuellement"

1c. Other bootstrap questions

AskUserQuestion:
  questions:
    - question: "Activer la détection automatique de l'environnement Docker pour le préflight ?"
      header: "Docker preflight"
      options:
        - label: "Oui (Recommended)"
          description: "Détecte docker-compose.yml et propose `docker compose up -d` si l'app est down"
        - label: "Non"
          description: "Le user gère le démarrage de l'app manuellement"

Apply references/pre-run-detection.md to detect a pre_run candidate (Makefile target / package.json script / Symfony bin/console command) and let the user choose. The reference returns a string (the chosen command) or null (skip). Store the result as pre_run_answer for use in Step 1e.

1d. Symfony-specific proposal

If Symfony was detected in 1a, surface the dedicated QA environment pattern. Do not scaffold anything by default — the pattern (separate APP_ENV=qa, dedicated DB, swap commands, idempotent seed) is invasive.

AskUserQuestion:
  question: "Symfony détecté. Veux-tu que je documente / scaffolde l'infra QA dédiée (APP_ENV=qa, DB séparée, app:qa:reset+seed) ?"
  header: "Infra QA Symfony"
  options:
    - label: "Documenter seulement"
      description: "Je crée references/symfony-qa-env.md à lire ; tu décides après"
    - label: "Scaffolder maintenant"
      description: "Je génère .env.qa, compose overrides, commandes app:qa:* (modifie le code)"
    - label: "Non"
      description: "Configuration QA simple, tu gères le reset à la main"

The reference doc is at references/symfony-qa-env.md (read by the user, not by Claude during sync). It is self-contained.

1e. Persist the config

Write the merged config back. Auth profiles default to a richer set than just user/admin, since onboarding/verification flows usually exist:

# pre_run is the value returned by the Step 1c pre-run detection (string or null)
jq --arg pre_run "${pre_run_answer:-}" '.qa = {
  "test_plan_dir": "docs/qa",
  "base_url": "<base_url answer>",
  "browser": "playwright",
  "env_file": ".env.qa",
  "label": "qa",
  "auth_profiles": ["anonymous", "user", "admin", "user_fresh", "user_unverified"],
  "preflight": { "docker_autostart": <answer> },
  "pre_run": (if $pre_run == "" then null else $pre_run end)
}' .claude/ticket-config.json > .claude/ticket-config.json.tmp \
  && mv .claude/ticket-config.json.tmp .claude/ticket-config.json

The five default profiles mean:

| Profile | Meaning | |---|---| | anonymous | Not logged in. | | user | Logged-in user with onboarding completed and email verified. | | admin | Logged-in user with admin role. | | user_fresh | Logged in, but onboarding not yet completed (used to test onboarding gates / first-run flows). | | user_unverified | Logged in, email not yet verified (used to test email-verification gates). |

The seed command on the backend materializes whichever profiles the project actually uses. The point of naming all five up front is that the names are stable across projects, so scenarios can reference them without the user having to invent new conventions later.

1f. Supporting files

Create on first run only:

• docs/qa/ directory if absent (mkdir -p docs/qa)
• docs/qa/README.md (placeholder index, regenerated in step 5)
• For Symfony projects: .env.qa.local.dist template at the project root (Symfony convention — the user copies it to .env.qa.local which is gitignored). For other stacks: .env.qa.example (the user copies it to .env.qa).

Template (adapt the filename):

# QA test credentials and fixture refs (gitignored target — copy this dist file)
[email protected]
QA_USER_PWD=changeme
[email protected]
QA_USER_FRESH_PWD=changeme
[email protected]
QA_USER_UNVERIFIED_PWD=changeme
[email protected]
QA_ADMIN_PWD=changeme

Add the gitignored target (.env.qa or .env.qa.local) to .gitignore if a .gitignore exists and the entry is missing. Do not create .gitignore if it doesn't exist (respect project conventions).

Step 2 — Compute the diff

Resolve the base commit:

if [ -n "$FROM_FLAG" ]; then
  BASE="$FROM_FLAG"
  IS_FIRST_RUN=0
else
  BASE=$(git log -n 1 --format=%H -- docs/qa/ 2>/dev/null)
  IS_FIRST_RUN=0
fi
if [ -z "$BASE" ]; then
  BASE=$(git rev-list --max-parents=0 HEAD | tail -n 1)
  IS_FIRST_RUN=1
fi

Collect the application diff (filter to web-relevant files; exclude tests and tooling):

git diff --stat "$BASE"..HEAD -- \
  ':!*.test.*' ':!*.spec.*' ':!__tests__/' \
  ':!*.lock' ':!*.lockb' ':!*.lock.json' \
  ':!.github/' ':!docs/qa/' \
  '*.tsx' '*.ts' '*.jsx' '*.js' '*.vue' '*.svelte' \
  '*.php' '*.py' '*.rb' '*.go' \
  'src/' 'app/' 'pages/' 'routes/' 'controllers/' 'views/' 'templates/'

If the diff is empty, output No application changes since last QA sync. Test plan is up to date. and exit.

Otherwise capture the same range with git diff (full content) and measure the size:

DIFF_BYTES=$(git diff "$BASE"..HEAD -- <same path filters as above> | wc -c)

Diff handling strategy

| Scenario | Diff size | Strategy | |---|---|---| | Incremental sync | ≤ 200 KB | Inline the full diff in the agent prompt (current default). | | Incremental sync | > 200 KB | Print: Le diff dépasse 200 KB depuis le dernier sync (<X> KB). Plusieurs syncs ont probablement été manqués. Relance avec --from <commit> pour cibler un sous-ensemble. Stop the skill. | | First run | ≤ 200 KB | Inline the full diff (e.g., a small repo or a brand-new project). | | First run | > 200 KB | Switch to route-map mode (below). Do NOT inline the diff — at first run on a mature codebase, the entire app is "diff", which means inlining produces a payload the agent can't usefully exploit. |

Route-map mode (first run, large codebase)

When IS_FIRST_RUN=1 and DIFF_BYTES > 200000:

1. Build a route map by grepping the project's routing surface. Adapt to the detected stack (from Step 1a):

   # React Router (component-style routes)
   grep -rEn '<Route[^>]+path=' frontend/src 2>/dev/null
   
   # File-based routers
   ls frontend/src/pages/ frontend/src/app/ pages/ app/ 2>/dev/null
   
   # Symfony controllers
   grep -rEn "#\[Route\(" src/ 2>/dev/null
   
   # Express / Fastify / Koa
   grep -rEn "(app|router)\.(get|post|put|delete|patch)\(" src/ app/ 2>/dev/null
   
   # Django urls.py
   grep -rEn "path\(|re_path\(" */urls.py 2>/dev/null
   
   # Laravel routes
   cat routes/web.php routes/api.php 2>/dev/null
   

2. Run git diff --stat "$BASE"..HEAD -- <filters> and keep the file list (no content).

3. Pass to the qa-writer agent:

   • The route map (route → file mapping).
   • The git diff --stat output (filenames only).
   • The architecture summary built in Step 1a.
   • Explicit instruction: "Use your Read / Grep / Glob tools to drill down into specific files. Do not expect a full diff — at first run on a mature codebase, treat the entire surface as new and prioritize coverage of the routes listed in the map."

The agent's Synthesis phase will then operate by reading the source on demand instead of waiting for an inlined diff.

Step 3 — Delegate to qa-writer

Launch the agent with the existing plan + diff context. The prompt shape depends on the mode picked in Step 2.

Mode A — inline diff (default)

Agent:
  subagent_type: qa-writer
  description: "Synthesize QA operations for diff <BASE>..HEAD"
  prompt: |
    SYNTHESIS PHASE

    ## Architecture summary

    <one-block summary from Step 1a>

    ## Existing Test Plan

    <inline contents of docs/qa/*.md, joined>

    ## Diff Context

    Base commit: <BASE>
    Head commit: <HEAD>

    ### Files changed (--stat)

    <git diff --stat output>

    ### Full diff

    <git diff output>

    Produce a single ## Operations YAML block following the format in your agent prompt.
    If you flag fallback selectors, include the `selectors_to_harden:` list.

    ### Preconditions

    Emit a `precondition:` block on every fixture-sensitive scenario. Heuristics:
    - `auth: user_fresh` or `auth: user_unverified` → guard the fixture flag
      (e.g., `http_status: GET /api/users/me/onboarding = '{"completed":false}'`).
    - Any `${QA_<RESOURCE>_ID}` reference in `start:`, `setup:`, `steps:`, or `expect:`
      → guard the resource exists (`http_status: GET /api/<resource>/${QA_<RESOURCE>_ID} = 200`).
    - Otherwise, no precondition needed.

    See `~/.claude/skills/qa-sync/references/scenario-format.md` §Preconditions for syntax
    and the canonical SKIPPED reason format.

    ### Textual assertions — source, never invent

    Any value used in `text:`, `text_not_present:`, `text_present_in:`, or
    `text_not_present_in:` that references a **translation**, **domain term**
    (card names, factions, traits, rule labels), or any **non-English common
    word** MUST come from a canonical source. Do not paraphrase, translate,
    or guess. The rule applies equally to negative assertions — `text_not_present:
    "Bonjour"` still depends on the canonical value of "Bonjour".

    Acceptable sources, picked by **where the string actually lives**:
    - **Static UI labels** (button text, navigation, form copy — e.g.
      "Welcome back", "Submit"): read the **frontend i18n bundle**.
      Common locations: `frontend/public/locales/<lang>/*.json`,
      `frontend/src/locales/<lang>.json`, `src/i18n/<lang>.json`. Copy
      the value verbatim.
    - **Backend-served translations** (entity translations, dictionary
      tables — card traits, faction names, rule labels): the **backend
      translation store** is authoritative. Examples: Gedmo Translatable,
      Symfony `translations/`, Django `django-modeltranslation` /
      `django.utils.translation` (`.po`), Rails `globalize`. Grep the
      project for the key, read the persisted value from the DB row,
      `.po` file, YAML bundle, or migration seed.
    - **Pure data values** (proper nouns served by the API, never
      localized client-side — card names, faction codes): capture an
      **API response sample** with `curl` (or read a fixture file under
      `fixtures/`, `seeders/`, or a test snapshot) and quote it verbatim.

    Do not fall through the list "in order" — pick the source that owns
    the string. A card trait that is not in the frontend bundle does NOT
    mean "no canonical source exists"; it means look at the backend.

    If no canonical source can be found, do NOT emit the textual assertion.
    Two acceptable fallbacks:
    - Replace the textual assertion with a **structural** one in the same
      `expect:` block (`selector_visible:`, `selector_count:`) — it
      validates the UI shape without depending on the literal string.
    - If even structural validation is impossible, mark the whole scenario
      as TODO by putting a `check_manual: <description>` line in `steps:`
      (per §Step actions). The runner skips it with a warning instead of
      asserting fabricated text.

    `check_manual:` is a step action, NOT an assertion — it goes in
    `steps:`, not in `expect:`.

    Trip-wire: before emitting any textual assertion whose value contains a
    word that is not plain English (FR / DE / ES / IT accents, non-ASCII,
    or a domain proper noun like a card name), confirm you read the value
    from one of the sources above. If you cannot point to a file:line for
    the source, drop the assertion.

    See `~/.claude/skills/qa-sync/references/scenario-format.md`
    §"Textual assertions — sourcing rule" for the full spec.

Mode B — route-map (first run, large codebase)

Agent:
  subagent_type: qa-writer
  description: "Bootstrap QA plan from route map (first run, large codebase)"
  prompt: |
    SYNTHESIS PHASE — first-run bootstrap

    ## Architecture summary

    <one-block summary from Step 1a, including detected seed commands and i18n setup>

    ## Existing Test Plan

    (empty — this is the first sync)

    ## Route Map

    <route → file mapping built in Step 2>

    ## Files in scope (--stat)

    <git diff --stat filenames>

    ## Important

    No inlined diff — the codebase is too large for that to be useful. Use your
    Read / Grep / Glob tools to drill down into the files you need. Prioritize
    coverage of the routes listed in the map. Run your Pre-synthesis discovery
    pass (selector convention, seed commands, i18n) before generating scenarios.

    Produce a single ## Operations YAML block following the format in your agent
    prompt. Include `selectors_to_harden:` for any fallback selector you used.

    ### Preconditions

    Emit a `precondition:` block on every fixture-sensitive scenario. Heuristics:
    - `auth: user_fresh` or `auth: user_unverified` → guard the fixture flag
      (e.g., `http_status: GET /api/users/me/onboarding = '{"completed":false}'`).
    - Any `${QA_<RESOURCE>_ID}` reference in `start:`, `setup:`, `steps:`, or `expect:`
      → guard the resource exists (`http_status: GET /api/<resource>/${QA_<RESOURCE>_ID} = 200`).
    - Otherwise, no precondition needed.

    See `~/.claude/skills/qa-sync/references/scenario-format.md` §Preconditions for syntax
    and the canonical SKIPPED reason format.

    ### Textual assertions — source, never invent

    Any value used in `text:`, `text_not_present:`, `text_present_in:`, or
    `text_not_present_in:` that references a **translation**, **domain term**
    (card names, factions, traits, rule labels), or any **non-English common
    word** MUST come from a canonical source. Do not paraphrase, translate,
    or guess. The rule applies equally to negative assertions — `text_not_present:
    "Bonjour"` still depends on the canonical value of "Bonjour".

    Acceptable sources, picked by **where the string actually lives**:
    - **Static UI labels** (button text, navigation, form copy — e.g.
      "Welcome back", "Submit"): read the **frontend i18n bundle**.
      Common locations: `frontend/public/locales/<lang>/*.json`,
      `frontend/src/locales/<lang>.json`, `src/i18n/<lang>.json`. Copy
      the value verbatim.
    - **Backend-served translations** (entity translations, dictionary
      tables — card traits, faction names, rule labels): the **backend
      translation store** is authoritative. Examples: Gedmo Translatable,
      Symfony `translations/`, Django `django-modeltranslation` /
      `django.utils.translation` (`.po`), Rails `globalize`. Grep the
      project for the key, read the persisted value from the DB row,
      `.po` file, YAML bundle, or migration seed.
    - **Pure data values** (proper nouns served by the API, never
      localized client-side — card names, faction codes): capture an
      **API response sample** with `curl` (or read a fixture file under
      `fixtures/`, `seeders/`, or a test snapshot) and quote it verbatim.

    Do not fall through the list "in order" — pick the source that owns
    the string. A card trait that is not in the frontend bundle does NOT
    mean "no canonical source exists"; it means look at the backend.

    If no canonical source can be found, do NOT emit the textual assertion.
    Two acceptable fallbacks:
    - Replace the textual assertion with a **structural** one in the same
      `expect:` block (`selector_visible:`, `selector_count:`) — it
      validates the UI shape without depending on the literal string.
    - If even structural validation is impossible, mark the whole scenario
      as TODO by putting a `check_manual: <description>` line in `steps:`
      (per §Step actions). The runner skips it with a warning instead of
      asserting fabricated text.

    `check_manual:` is a step action, NOT an assertion — it goes in
    `steps:`, not in `expect:`.

    Trip-wire: before emitting any textual assertion whose value contains a
    word that is not plain English (FR / DE / ES / IT accents, non-ASCII,
    or a domain proper noun like a card name), confirm you read the value
    from one of the sources above. If you cannot point to a file:line for
    the source, drop the assertion.

    See `~/.claude/skills/qa-sync/references/scenario-format.md`
    §"Textual assertions — sourcing rule" for the full spec.

Capture the agent's ## Operations YAML block (and the optional selectors_to_harden: list).

Guard-rails for textual assertions

The synthesis prompt enforces a "source canonical text values, never invent" rule for text:, text_not_present:, text_present_in:, and text_not_present_in: assertions. The rule covers four cases:

1. i18n bundle — when the project has locales/<lang>/*.json or an equivalent: read the bundle, copy the literal value. Detected during architecture detection (Step 1a) and surfaced in the synthesis prompt so the agent knows where to look.
2. DB-backed translations (Gedmo Translatable, Symfony translations/, Django django-modeltranslation or django.utils.translation (.po), Rails globalize): grep the project for the translation key, read the persisted value from the migration / fixture / .po file. Never re-translate.
3. Domain terms (card names, factions, traits, rule labels): treat these as canonical strings owned by the backend. Source from the API sample, fixture file, or seeded reference data. Do not localize them client-side and do not invent them.
4. Non-English common words: if a textual assertion contains an FR / DE / ES / IT word (or any non-ASCII character), the agent must point to a source file:line. If it cannot, the textual assertion is dropped and replaced with a structural assertion (selector_visible:, selector_count:) in the same expect: block — or, if structural validation is impossible, the scenario is marked as TODO via a check_manual: step. check_manual: lives in steps:, not in expect:.

These rules are injected directly into the synthesis prompt (Mode A and Mode B) via the "Textual assertions — source, never invent" sub-block. The sub-block is duplicated verbatim in both prompts so each one stays self-contained — if you change one, change the other. Project consumers that enforce a strict "no self-translation" policy in their own CLAUDE.md rely on this guard-rail to keep /qa-sync aligned with their convention.

Step 4 — Apply operations

Parse the YAML. For each operation:

• ADD: append the scenario block (and a top-level ## <Scenario name> heading derived from the id) to target_file. Create the file if missing with this header:

  # <Flow name from filename, capitalized>
  
  Scenarios for the <flow> user flow.
  

• UPDATE with changes: locate the scenario by id (search for id: <ID> in YAML blocks), present the change description as a comment under the scenario heading, then ask the user via AskUserQuestion whether to apply automatically (with current → proposed selector mapping) or open the file for manual edit. Avoid silent surgical edits — selector changes are best reviewed.

• UPDATE with replace_with: replace the existing YAML block in place, keeping the section heading.

• REMOVE: delete the scenario heading and YAML block. If the file becomes empty, also remove the file (confirm with the user first).

If --flow <name> is set, skip operations whose target_file is not docs/qa/<name>.md.

If --dry-run, print the operations as a table and exit without writing.

Step 5 — Regenerate docs/qa/README.md

After all operations applied, scan docs/qa/*.md (excluding README), count scenarios per file (one scenario = one YAML fenced block with a top-level id: key), and rewrite the index:

# QA Test Plan

Living test plan for this project. Maintained by `/qa-sync`, executed by `/qa-run`.

## Coverage

| Flow | File | Scenarios | Last sync |
|------|------|-----------|-----------|
| Auth | [auth.md](auth.md) | 3 | <date du dernier commit du fichier> |
| Checkout | [checkout.md](checkout.md) | 5 | ... |

**Total:** N scenarios across M flows.

## How to update

- Run `/qa-sync` after merging features to keep this plan current.
- Run `/qa-run [--flow <name>]` to execute scenarios via Playwright.
- Edit individual `<flow>.md` files manually for fine-grained tweaks.

## Format

Each scenario is a fenced YAML block. See [scenario-format reference](../../.claude/skills/qa-sync/references/scenario-format.md) (read by Claude) for the schema.

Use git log -n 1 --format=%cs -- "<file>" for the last sync date per file (ISO date). Fall back to — if no commit yet.

Step 6 — Recap

Show the user what changed:

git diff --stat docs/qa/

Print a final block:

## QA sync done

- N operations applied (X added, Y updated, Z removed)
- Files touched: docs/qa/auth.md, docs/qa/checkout.md
- Run `/qa-run` to execute the updated plan, or `git diff docs/qa/` to review.

Commit when ready (skill does not commit automatically).

If the agent returned a selectors_to_harden: list (see qa-writer output format), append a "Selectors to harden" block right after the recap so the user sees the technical debt left by this pass:

## Selectors to harden

The QA plan currently uses fallback selectors for the following elements. Add a stable
test attribute (matching the project's convention) to each so future syncs can use it:

- `frontend/src/pages/NotFoundPage.tsx` — Go Home link → suggested `data-testid=not-found-go-home-link` (used by NOT-FOUND-01)
- `frontend/src/components/AnnotationCard.tsx` — Delete button → suggested `data-testid=delete-annotation-btn` (used by ANNOTATIONS-DELETE-01)

Treat this as a small follow-up PR — once these attributes land, re-run `/qa-sync` to tighten the scenarios.

Skip this block entirely if the list is empty or absent.

Edge cases

• No qa-writer output: if the agent returned an empty operations: [], do not regenerate the README. Print the agent's note and exit.
• YAML parse error: log the offending block, stop before any file write, and surface the error to the user verbatim.
• Conflicting IDs: if an ADD op uses an id already present in another file, stop and ask which file should own it.
• Missing scenario for UPDATE/REMOVE: log a warning, skip that op, continue with the rest.

What this skill does NOT do

• It does not commit. The user reviews git diff docs/qa/ and commits manually (or via /commit).
• It does not execute tests. That's /qa-run.
• It does not modify application code.
• It does not propose product features. That's /express-need.