fearovex/sdd-verify

sdd-verify

Verifies that the implementation complies with the specs, design, and task plan. The quality gate before archiving.

Triggers: /sdd-verify <change-name>, verify implementation, quality gate, validate change, sdd verify

---

Purpose

Verification objectively validates that what was implemented meets what was specified. It fixes nothing — it only reports.

Sibling reference file

• REPORT_TEMPLATE.md — persisted compact format + conversational detail blocks.

Skill resolution

Project-local → global, in order:

1. .claude/skills/sdd-verify/SKILL.md     (project-local — highest priority)
2. ~/.claude/skills/sdd-verify/SKILL.md   (global catalog — fallback)

See docs/SKILL-RESOLUTION.md for the full algorithm.

---

Process

Step 0 — Preload

Follow skills/_shared/sdd-phase-common.md Section F (Project Context Load) and Section G (Spec Context Preload). Both non-blocking.

Step 1 — Load all artifacts

Read in this order:

• Tasks — mem_search(query: "sdd/{change-name}/tasks") → mem_get_observation(id). Engram unreachable → orchestrator passes inline.
• Spec — same pattern with sdd/{change-name}/spec.
• Design — same pattern with sdd/{change-name}/design.
• Code files created/modified.

Step 2 — Completeness check

Count total tasks vs completed. Emit Completeness block from REPORT_TEMPLATE.md.

Severity:

• Incomplete core logic tasks → CRITICAL.
• Incomplete cleanup/docs tasks → WARNING.

Step 3 — Correctness check (specs)

For each requirement in the spec:

1. Look for evidence in the code that it is implemented.
2. For each Given/When/Then scenario:
   • GIVEN handled (precondition/guard)?
   • WHEN implemented (action/endpoint)?
   • THEN verifiable (correct result)?

Emit Correctness + Scenario Coverage blocks from REPORT_TEMPLATE.md.

Step 4 — Coherence check (design)

Verify design decisions were followed. Emit Coherence block from REPORT_TEMPLATE.md.

Step 5 — Testing check

Verify test presence and scenario coverage. Emit Testing block from REPORT_TEMPLATE.md.

Step 6 — Run tests

Three-level priority model. Check config.yaml at project root in order:

Level 1 — verify_commands (highest priority, checked first):

• Present → use the listed commands in order; do NOT check levels 2–3; for each command run via Bash, capture exit code + stdout/stderr; record in ## Tool Execution with source label "verify_commands (config level 1)".
• Absent → proceed to Level 2.

verify_commands is NOT additive — it overrides all lower levels. Commands are assumed non-destructive; the user is responsible for this.

Level 2 — verify.test_commands (when Level 1 is absent):

| verify.test_commands shape | Behavior | |------------------------------------|----------| | Not a list | WARNING: "verify.test_commands is not a list — treating as absent". Fall through to Level 3. | | Empty list [] | Treat as absent. Fall through to Level 3 (prevents silent zero-command success). | | Non-empty list | Use commands in order; record source label "verify.test_commands (config level 2)". Skip Level 3. |

Level 3 — auto-detection (only when 1 and 2 are absent or invalid). First match wins:

| Priority | File | Condition | Command | |----------|-----------------------------------------------|---------------------------|----------------------------------------------------------------------------------------| | 1 | package.json | scripts.test exists | npm test (or yarn test if yarn.lock, pnpm test if pnpm-lock.yaml) | | 2 | pyproject.toml / pytest.ini / setup.cfg | pytest indicators present | pytest | | 3 | Makefile | test target exists | make test | | 4 | build.gradle / gradlew | file exists | ./gradlew test | | 5 | mix.exs | file exists | mix test | | — | none of the above | — | Skip with WARNING |

Execution: run via Bash, capture exit code (0 = pass) + stdout/stderr, record runner name + command + exit + failure summary.

Error handling:

| Condition | Report | Status | |--------------------------------------------|-------------------------------------------------------------------|---------| | Command not executable (deps missing, etc.) | "Test Execution: ERROR — [error message]" | WARNING | | Tests run but some fail | Report failure count + failing test names if parseable | (varies) | | No test runner detected | "Test Execution: SKIPPED — no test runner detected" | WARNING |

Save the full test output for Steps 8 and 9.

Step 7 — Build & type check

Detect and execute build/type-check command.

Config overrides (checked before auto-detection):

| Key | Behavior | |----------------------------------|----------| | verify.build_command (string) | Use as build/type-check command. Skip auto-detection for build. | | verify.build_command (non-string) | WARNING "verify.build_command is not a string — treating as absent". Fall back to auto-detection. | | verify.type_check_command (string) | Use as type-check command. Skip auto-detection for type check. | | verify.type_check_command (non-string) | WARNING. Fall back to auto-detection. |

Both overrides are independent — one can be set without the other.

Build command auto-detection (when override absent or invalid). First match wins:

| Priority | File | Condition | Command | |----------|----------------------------|---------------------------------------------|--------------------------------------| | 1 | package.json | scripts.typecheck exists | npm run typecheck | | 2 | package.json | scripts.build exists | npm run build | | 3 | tsconfig.json | file exists + TypeScript in devDependencies | npx tsc --noEmit | | 4 | Makefile | build target exists | make build | | 5 | build.gradle / gradlew | file exists | ./gradlew build | | 6 | mix.exs | file exists | mix compile --warnings-as-errors | | — | none of the above | — | Skip with INFO |

Execution: run via Bash, capture exit code, capture error output.

Error handling:

| Condition | Report | Status | |----------------------------|-------------------------------------------------------------|----------| | Command not executable | "Build/Type Check: ERROR — [error message]" | WARNING | | Build fails | "Build/Type Check: FAILING" + error output in detail | (varies) | | No build command detected | "Build/Type Check: SKIPPED — no build command detected" | INFO |

Step 8 — Coverage validation (optional)

Active ONLY when coverage.threshold is configured. Advisory only — never CRITICAL, never blocks.

1. Read config.yaml for coverage.threshold (e.g., coverage: { threshold: 80 }).
2. Not configured → "Coverage Validation: SKIPPED — no threshold configured".
3. Configured:
   • Parse coverage % from Step 6 test output.
   • Actual ≥ threshold → "Coverage: [X]% (threshold: [Y]%) — PASS".
   • Actual < threshold → "Coverage: [X]% (threshold: [Y]%) — BELOW THRESHOLD". Status WARNING.
4. Cannot parse coverage data → "Coverage Validation: SKIPPED — could not parse coverage from test output". Status WARNING.

Step 9 — Spec Compliance Matrix

Cross-reference every Given/When/Then scenario from the change's spec files against verification evidence.

1. Read all spec content from the active persistence mode (same source as Step 1).
2. Extract every Given/When/Then scenario.
3. For each scenario, cross-reference against:
   • Code implementation evidence from Step 3 (Correctness).
   • Test results from Step 6 — when tests were executed.
4. Assign a compliance status per scenario (see status definitions in REPORT_TEMPLATE.md).

Emit the matrix using the template from REPORT_TEMPLATE.md. The matrix MUST include scenarios from ALL spec domains affected by the change.

Step 10 — Create verify-report.md

Evidence rule — applies to every criterion in the report:

A criterion MUST only be marked [x] when:

1. A tool command was run and its output confirms the criterion, OR
2. The user provided an explicit evidence statement.

Neither met → leave [ ] with "Manual confirmation required — no tool output available". Abstract reasoning or code inspection alone MUST NOT suffice to mark [x].

The ## Tool Execution section is mandatory in every report — even when skipped. When skipped: "Test Execution: SKIPPED — no test runner detected".

Persistence — call mem_save with topic_key: sdd/{change-name}/verify-report, type: architecture, project: {project}, content = full persisted-compact markdown from REPORT_TEMPLATE.md. Do NOT write any file.

Engram unreachable → skip persistence; return report content inline only.

Conversational output — the full detail sections (Completeness, Correctness, Coherence, Testing, Spec Compliance Matrix, Coverage Validation, SUGGESTIONS) are presented in the conversational response using the detail-block templates from REPORT_TEMPLATE.md. The user must see the full analysis; only the persisted artifact is compact.

---

Verdict criteria

| Verdict | Condition | |------------------------|-------------------------| | PASS | 0 critical, 0 warnings | | PASS WITH WARNINGS | 0 critical, 1+ warnings | | FAIL | 1+ critical |

Severities

| Severity | Description | Blocks archiving | |--------------|-------------------------------------------------------------------------------------------------------------------|------------------| | CRITICAL | Requirement not implemented, main scenario not covered, core task incomplete | Yes | | WARNING | Edge case without test, design deviation, pending cleanup task, test execution failure | No | | SUGGESTION | Optional quality improvement | No | | SKIPPED | Step preconditions not met (no test runner, no build command, no coverage config) — does NOT count toward verdict | No | | INFO | Informational note (e.g., no build command detected) — does NOT count toward verdict | No |

Verdict calculation: only the four original dimensions (Completeness, Correctness, Coherence, Testing) plus Test Execution and Spec Compliance contribute CRITICAL/WARNING. SKIPPED and INFO statuses do NOT count toward verdict. Preserves identical verdict behavior for projects without test infrastructure.

---

Output to orchestrator

{
  "status": "ok|warning|failed",
  "summary": "Verification [change-name]: [verdict]. [N] critical, [M] warnings.",
  "artifacts": ["engram:sdd/{change-name}/verify-report"],
  "test_execution": {
    "runner": "[detected runner or null]",
    "command": "[command or null]",
    "exit_code": "[0/1/N or null]",
    "result": "PASS|FAILING|ERROR|SKIPPED"
  },
  "build_check": {
    "command": "[command or null]",
    "exit_code": "[0/1/N or null]",
    "result": "PASS|FAILING|ERROR|SKIPPED"
  },
  "compliance_matrix": {
    "total_scenarios": "[N]",
    "compliant": "[N]",
    "failing": "[N]",
    "untested": "[N]",
    "partial": "[N]"
  },
  "next_recommended": ["sdd-archive (if PASS or PASS WITH WARNINGS)"],
  "risks": ["CRITICAL: [description if any]"]
}

After the report:

Continue with archive? Reply **yes** to proceed or **no** to pause.
_(Manual: `/sdd-archive <slug>`)_

---

Rules

• ONLY report — fix nothing during verification.
• Read real code — do NOT assume something works because the file exists.
• Objective: report what IS, not what should be.
• Evaluate documented deviations in tasks.md with context.
• A FAIL is information for improvement, not a personal failure.
• Run tests if possible (via Bash). Report actual results.
• ## Tool Execution is mandatory in every report — even when skipped. When skipped state: "Test Execution: SKIPPED — no test runner detected".
• A criterion marked [x] MUST have verifiable evidence: tool output OR explicit user evidence statement. Abstract reasoning or code inspection alone MUST NOT suffice.
• Test command resolution: Level 1 (verify_commands) > Level 2 (verify.test_commands) > Level 3 (auto-detection). Each level consulted ONLY when all higher levels are absent or invalid.
• Empty verify.test_commands: [] falls through to auto-detection — NEVER treated as zero-command success.
• verify.build_command and verify.type_check_command override their auto-detected commands when present AND strings. Non-string values emit WARNING and fall back to auto-detection.