aganesy/qfai-verify

<!-- QFAI Skill Body (SSOT) - This file is intended to be referenced by tool-specific wrappers (e.g., GitHub/Claude/Codex skills). - Keep wrappers thin and route users to this skill body. -->

/qfai-verify — Quality Gates and Evidence

[DRIFT-PROTOCOL:MANDATORY]

User Questions (AskUserQuestion Protocol)

• When a question to the user is needed (e.g., gate failure triage, fix approach confirmation), the agent MUST use AskUserQuestion if the tool is available.
• When AskUserQuestion supports structured choices (radio/multi-select), the agent MUST prefer structured choices over free-text input.
• If AskUserQuestion is technically unavailable, present the same question as a normal message with explicit numbered choices. The agent SHOULD preserve structured choice semantics (enumerated options, selection constraints). The reason for unavailability MUST be stated.

FORMAT SSOT (Mandatory)

• **Before writing or editing any .qfai/** artifact**, read and follow the relevant directory README template and sample:
  • .qfai/require/README.md
  • .qfai/specs/README.md
  • .qfai/contracts/**/README.md
  • .qfai/evidence/README.md
• Do NOT copy templates/samples into this prompt or into other prompt markdown.
• The generated artifacts must match the README-defined structure (headings, ordering, table columns).
• Completion requires a Format Self-Check in the evidence: list each artifact and confirm “matches README template”.

Inputs Priority (Preflight)

When unsure, read inputs in this order:

• P1: .qfai/assistant/instructions/*
• P2: .qfai/assistant/steering/*
• P3: .qfai/specs/<spec-id>/09_delta.md (Decision Records; if no spec yet, state "not applicable")
• P4: other artifacts (01_Spec.md, contracts, evidence, optional legacy scenario.feature / coverage ledgers)

Verify Scope Rule (Mandatory)

• /qfai-verify MUST always run full-scan verification.
• Do NOT use Preflight Diff (or any diff-only shortcut) in this skill.
• Preserve the DR-0007/spec-0011 intent: verify is the safety gate and must not be reduced to incremental checks.

Sub-agent Delegation (MANDATORY)

This section is mandatory and overrides any conflicting fallback text in this file.

Orchestrator Protocol (MUST)

• Orchestrator may only create work orders, delegate tasks, integrate outputs, and present results to the user.
• Orchestrator MUST NOT generate the primary artifact first draft.
• Orchestrator MUST NOT serve as Reviewer or skip delegation for convenience.

Capability Probe (MUST)

1. Run one harmless Probe Task (for example: "reply with ok") once at stage start.
2. If subagents are unavailable, explicitly ask the user for Simulation mode approval.
3. Without explicit approval, stop the stage and do not continue.

Simulation mode (Opt-in only)

• Allowed only when the user explicitly states Simulation mode allowed.
• When used, record both of the following in outputs/evidence:
  • Subagents: simulated (reason: <why unavailable>)
  • User approval: <quote or reference>

Work Orders Summary (MANDATORY evidence)

Every major artifact in this stage MUST include a ## Work Orders Summary section with this fixed table schema:

| Step | Role (sub-agent) | Task title | Input (refs) | Output (refs) | Status (PASS/REVISE) | | ---- | ---------------- | ---------- | ------------ | ------------- | -------------------- | | 1 | <role> | <task> | <refs> | <refs> | PASS/REVISE |

• Output (refs) must point to in-file anchors or relative evidence file paths.

Stage Minimum Roles (MUST)

• Delegate: Runner, ReportWriter create first drafts of execution evidence and verification summary.
• Integrate: Orchestrator consolidates delegated outputs and presents them to the user for confirmation.
• Gate: Reviewer is delegated independently and returns only PASS or REVISE.
• Orchestrator must not draft the primary artifact body and must not self-approve.

Reviewer Gate (MUST)

• Final completion gate MUST be delegated to an independent completion-reviewer.
• Reviewer checks (minimum):
  • Required roles were delegated (no orchestrator self-authoring).
  • DoD satisfied (validate gate, test-layer hard gate, evidence, DR-IDs).
  • Validate gate evidence exists: qfai validate --fail-on error completed with error=0.
  • Drift Protocol enforced:
    • No upstream artifact edits were made without an explicit user-approved Change Request.
    • If upstream changes exist, the correct owner skill was re-run after approval; downstream did not patch upstream directly.
  • Test-layer policy enforced:
    • E2E/API/Integration coverage aligns with steering/test-layers.md and the project’s plan.
    • Do not use pyramid ratios as a gate; use floors/ratios only as signals. Coverage obligations are the gate.
• Route specialist reviewers from .qfai/assistant/steering/agent-routing.yml.
• Default verify review set:
  • qa-gatekeeper
  • completion-reviewer
• Add implementation-reviewer only when code fixes are in scope.
• Do not declare DONE or handoff until all routed blocking reviewers return PASS.
• Every reviewer MUST provide a concrete alternative or fix proposal when returning FAIL.

Work order template (copy/paste)

Task title: <short>
Role: <sub-agent role>
Goal: <what to decide/produce>
Inputs (refs):
- <file/section>
Constraints:
- must: enforce Drift Protocol (no upstream edits without user approval + CR)
- must: verify plan/test-layer adherence (`steering/test-layers.md` + plan)
- must: check `qfai validate --fail-on error` passes with evidence (`error=0`)
- must: enforce `.qfai/assistant/steering/test-layers.md` hard gates
- must_not: accept test-volume ratios/floors as a hard gate
- must_not: accept upstream edits made directly by downstream phase
Output format:
- <headings / bullet schema>
Quality bar:
- PASS if ...
- REVISE if ...

Reviewer response template

Result: PASS | REVISE
Findings:
- <issue>
Required fixes:
- <action>
Evidence checked:
- <refs>

Stage 0 — Steering completion refresh (mandatory)

Before moving forward in this stage, refresh these files:

• .qfai/assistant/steering/manifest.md
• .qfai/assistant/steering/product.md
• .qfai/assistant/steering/structure.md
• .qfai/assistant/steering/tech.md

Rules:

• Detect incomplete content (empty sections, placeholder-only lines, <...>, TBD, stale facts).
• Fill what is verifiable from repository evidence (tree, docs, require/spec artifacts, package.json, CI definitions).
• If something cannot be verified, record it as an Open Question and ask the user.
• Even if steering is already complete, update it when new facts are discovered in this stage.

Delta Rejected Guard (Mandatory)

• Do NOT reintroduce options marked as rejected in 09_delta.md.
• If a rejected option must be reconsidered, create a [RE-OPEN] Decision Record in 09_delta.md that references the prior DR-ID, states what changed + new criteria, and includes explicit approval (user or instructions/steering).

CRITICAL CONSTRAINTS (Read First)

• Do NOT declare completion without running the defined gates.
• You MUST produce the required evidence file: .qfai/evidence/verify-<spec-id>.md.
  • .qfai/evidence/ is intentionally NOT tracked by Git (it ships with a local .gitignore).
  • Do NOT commit evidence files; summarize key outcomes in the PR description instead.
• You MUST run the mandatory checks listed below and record outcomes.
• In CI, you MUST keep QFAI validation on default/full mode (qfai validate --fail-on error). Do NOT use --phase refinement.
• Waivers are only for warning / info findings. If a waiver attempts to suppress an error, treat it as a failure and fix the root cause.
• You MUST stop and escalate if any gate fails without an actionable fix list.
• Completion must be approved by a reviewer who did not run the gates.

Completion Contract (Shared)

Before declaring completion, you MUST:

• OQ / undefined resolution: detect undefined or ambiguous items; resolve them or explicitly defer them with documented rationale and (when required by this prompt) user approval.
• Deliverable completeness: verify every expected artifact listed in this prompt (and required README templates) exists and is fully populated; no missing required sections.
• OQ / placeholder scan: scan all generated artifacts (including evidence) for placeholders such as "TBD", "TODO", "TBA", "TBC", "XXX", "???", "OQ", "OPEN QUESTION", "UNDEFINED", "PLACEHOLDER", and localized equivalents in the user's language. Resolve or explicitly defer; do not leave silent placeholders.
• Smoke check (if applicable): when the prompt produces runnable code, tests, or configs, execute the smallest command that proves basic run/start/operate and record evidence. If not applicable, state "not applicable" with a short rationale.

Goal

Run quality gates and produce evidence that the change is correct and safe.

Success Criteria (Definition of Done)

• Repo quality gates PASS (format/lint/type/test/build/etc).
• QFAI checks PASS (at minimum: qfai validate, and optionally qfai report).
• A concise evidence summary exists (copy‑paste for PR).
• The PR-ready summary includes Change Classification (Primary/Tags) per .qfai/assistant/instructions/change-classification.md.
• Evidence file exists: .qfai/evidence/verify-<spec-id>.md.
• Completion is approved by a reviewer who did not run the gates.

Mandatory checks

• Run listed commands and record outputs.
• If failing, produce an actionable fix list (not vague).
• Static policy checks:
  • .qfai/assistant/instructions/drift-protocol.md exists.
  • .qfai/assistant/steering/test-layers.md exists.
  • all .qfai/assistant/skills/*/SKILL.md include [DRIFT-PROTOCOL:MANDATORY].
  • reviewer-related agent docs include drift-protocol and test-layer review viewpoints.

Not-done criteria

• "Seems ok" without actual command outputs.

Non‑Negotiable Principles (QFAI Articles)

These principles are inspired by “constitution / articles” patterns used by other agent frameworks, but tailored to QFAI.

1. SDD First (Specification is the source of truth)
   If there is a conflict between code and spec, treat the spec as authoritative and either (a) fix code or (b) raise an explicit Open Question to change the spec.

2. Traceability is mandatory
   Every meaningful change must be traceable: Require → Spec → Scenario → Tests → Code → Verification evidence.

3. Evidence over confidence
   Prefer observable proof (logs, commands, file diffs, test results). If you cannot verify, say so and record it.

4. Minimize scope, but never hide gaps
   Keep changes minimal, but do not “paper over” missing decisions. If something blocks correctness, stop and ask.

5. Quality gates are the decision mechanism
   Use tests/lint/typecheck/build/pack verification (whatever the repo defines) as the primary guardrail. Fix until PASS.

6. Make it runnable
   Outputs must be executable in terminal/CI. Provide copy‑paste commands.

7. User time is expensive
   Ask only the questions that are truly blocking. Everything else: make reasonable assumptions and label them clearly.

README Rule

Do not edit any .qfai/**/README.md file; raise an Open Question instead.

• READMEs are reference guides. Follow their structure, templates, and checklists.

Absolute Rule — Output Language

All outputs MUST be written in the user’s working language for this session.

• If the user writes in Japanese, output Japanese.
• If the user writes in English, output English.
• If the user mixes languages, prefer the dominant language unless explicitly instructed otherwise.
  This rule overrides all other stylistic preferences.

Multi‑Role Orchestration (Subagents)

This workflow assumes the environment may support subagents (e.g., Claude Code “Task” tool) or may not.

If subagents are supported

Delegate to multiple roles and then merge the results. Use a “real‑world workflow” order:

• discovery-analyst -> requirements-analyst -> delivery-planner -> solution-architect -> test-design-analyst -> qa-strategist -> devops-ci-engineer -> completion-reviewer / qa-gatekeeper

Pseudo‑invocation pattern (adjust to your tool):

Task(
  subagent_type="planner",
  description="Create an execution plan and DoD",
  prompt="Context: ...\nGoal: ...\nConstraints: ...\nReturn: phases + risks + DoD"
)

If subagents are NOT supported

Only with explicit user approval (Simulation mode allowed), simulate roles by running the same sequence yourself:

• Write a short “role output” section per role, then consolidate into the final deliverable(s).

Completion Separation (mandatory)

• Gate execution (devops-ci-engineer) and completion approval (completion-reviewer) must be separate.
• qa-gatekeeper must confirm gate coverage before approval.

Context Refresh (mandatory for long tasks)

Every 5 major actions, pause and restate:

• DoD and prohibited "done" criteria
• Gates already executed vs remaining
• Evidence captured so far and what is missing

Step 0 — Load Context (always)

1. Read relevant project steering (if present):

   • .qfai/assistant/steering/structure.md
   • .qfai/assistant/steering/tech.md
   • .qfai/assistant/steering/product.md
   • any additional files under .qfai/assistant/steering/

2. Read project constitution / instructions (if present):

   • .qfai/assistant/instructions/constitution.md
   • .qfai/assistant/instructions/workflow.md (or equivalent)

3. Read existing artifacts for the current work item (if present):

   • .qfai/require/
   • .qfai/specs/spec-*/
   • .qfai/contracts/

4. Inspect repo conventions:

   • package manager (pnpm/npm/yarn), test runner, lint/typecheck scripts, CI definitions
   • existing test patterns (unit/integration/e2e)

Step 0 — Project Analysis (mandatory)

Before producing any deliverable, thoroughly analyze the current project so your outputs fit the repo’s:

• background and goals
• directory structure and conventions
• chosen technologies and versions (runtime, package manager, test runner)
• architecture boundaries (packages, CLI, core modules)
• existing patterns for tests, docs, and CI

Minimum analysis checklist

• [ ] Read key repo docs: README / CHANGELOG / RELEASE (if present)
• [ ] Inspect .qfai/ layout and existing SDD/ATDD/TDD artifacts (if present)
• [ ] Inspect packages/qfai structure (CLI entrypoints, core modules, validators, assets/init)
• [ ] Identify standard gate commands (format/lint/type/test/verify-pack) and where they are defined
• [ ] Search for existing examples/patterns of similar changes in tests (if available)
• [ ] Note constraints: Node versions, CI matrix, packaging rules, verify-pack expectations

If analysis cannot be performed (missing access), clearly state what could not be verified and proceed with minimal-risk assumptions.

Step 0.5 — Steering Bootstrap / Refresh (mandatory when incomplete)

QFAI expects assistant/steering/ to contain project‑specific facts so all subsequent design/test/implementation fits this repository.

What to do

1. Open these files:

• .qfai/assistant/steering/product.md
• .qfai/assistant/steering/tech.md
• .qfai/assistant/steering/structure.md

1. If they are missing, mostly empty, or still have placeholders (e.g., a lone - only), populate them by analyzing the current repository:

• derive “what/why/users/success/non-goals” from README/docs/issues (product.md)
• derive runtime/tooling versions + constraints from package.json, CI config, lockfiles (tech.md)
• derive repo layout + key directories + gate commands from the file tree and scripts (structure.md)

1. Do not invent facts. If something cannot be verified, write it as:

• TBD + what evidence is missing, or
• an Open Question (if it blocks correctness)

Steering refresh checklist

• [ ] product.md: what we build / users / success / non-goals / release posture
• [ ] tech.md: Node / package manager / TS / test / lint / CI constraints
• [ ] structure.md: repo layout, key packages, entrypoints, standard gate commands, how to run locally

Step 1 — Discover project gate commands (DevOps/CI Engineer)

Prefer existing scripts:

• package.json scripts
• Makefile / task runner
• CI config

If unknown, propose defaults and mark assumptions.

Step 2 — Run QFAI gates

Run (adjust as needed):

• qfai validate --fail-on error
• qfai report (if used in this repo)

Notes:

• CI must run default/full validation only. --phase refinement is local-only.
• If QFAI-WAIVER-002 appears, remove the invalid waiver and resolve the underlying error finding.

Capture:

• exit codes
• key errors/warnings
• file paths affected

Step 3 — Run repo gates

Run the repo’s standard pipeline in a stable order:

1. format
2. lint
3. typecheck
4. unit tests
5. scenario/e2e tests
6. build/package (if relevant)

Step 4 — Fix loop (Code Reviewer + QA)

If anything fails:

• Identify whether it’s spec mismatch, test issue, or implementation defect.
• Fix the root cause (do not silence tests without reason).

Step 5 — Produce Evidence Summary (Delivery Planner)

Output this format:

Verification Evidence

• Change classification (SSOT: .qfai/assistant/instructions/change-classification.md):

  • Primary:
  • Tags:
  • rationale (1-3 lines):

• QFAI:

  • command:
  • result:

• Repo gates:

  • command:
  • result:

• Notes:

  • assumptions:
  • risks:

Evidence (MANDATORY)

Create and update: .qfai/evidence/verify-<spec-id>.md

Evidence must include:

• command list + pass/fail + next actions

Required sections

• Objective
• Inputs reviewed (files/paths)
• Decisions made (with rationale)
• Work performed (what changed, where)
• Commands executed + key outputs
• Gaps / Open risks (must be explicit; "none" is acceptable if justified)
• Final status (PASS/FAIL) + who confirmed

Template

# Verify Evidence: <spec-id>

## Objective

## Inputs reviewed (files/paths)

## Decisions made (with rationale)

## Work performed (what changed, where)

## Commands executed + key outputs

- command:
- result:

## QFAI gates

- command:
- result:

## Repo gates

- command:
- result:

## Next actions (if any)

## Gaps / Open risks

## Final status (PASS/FAIL) + who confirmed

Completion Criteria (Final Gate)

All of the following must be verified and PASS:

1. QFAI validation:

   qfai validate --fail-on error
   

2. Repository standard gates (discover from package.json/CI/docs):

   • format check
   • lint
   • typecheck
   • tests
   • pack/verify (if distributed)

   Record the exact commands and results.

If you cannot run these commands (environment limitation):

• Request the user to run them and provide the output.
• Do NOT assume PASS without evidence.

Output

• Evidence summary with all gate results
• All gates: PASS confirmed
• Next action suggestion: proceed to PR creation (use your platform workflow)

DONE Declaration (Mandatory Output)

When you declare DONE, include:

• Referenced inputs: instructions/steering and the 09_delta.md spec-id
• DR-IDs referenced (or "none" + propose adding a Decision Record)
• Confirmation that no rejected options were reintroduced (or list RE-OPEN DR-IDs)

FINAL CHECKLIST (Check Last)

• [ ] CRITICAL CONSTRAINTS were followed.
• [ ] Evidence file exists and is complete.
• [ ] All mandatory checks were executed and recorded.
• [ ] No untracked gaps remain (or they are explicitly documented).
• [ ] Completion approved by a reviewer who did not run the gates.

Completion Checklist (MUST)

• [ ] This skill's Definition of Done is satisfied.
• [ ] Required artifacts were produced or updated (if applicable).
• [ ] Open questions were logged to the proper OQ file (if applicable).
• [ ] The completion message was presented to the user.
• [ ] Next actions were enumerated for all available options.

Completion Message & Next Actions (MUST)

When this skill is complete, provide a final user-facing completion message and enumerate all actionable next steps.

• Proceed (recommended): Create a PR on your hosting platform. Action: use the verified evidence to write the PR description.
• Any gate failed: Action: return to the owning skill, fix the issue, then rerun /qfai-verify.
• Need a report artifact: Action: run qfai report after validation outputs are up to date.