sd0xdev/test-health

Test Health — Holistic Coverage Measurement

Trigger

• Keywords: test health, coverage measurement, test metrics, coverage trend, test inventory, holistic test audit

When NOT to Use

| Scenario | Alternative | |----------|------------| | Run tests | /verify | | Review test sufficiency only | /codex-test-review | | Generate unit tests | /codex-test-gen | | Feature-doc coverage only | /check-coverage | | Context-aware test execution + triage | /test-deep |

Workflow

flowchart TD
    U[User: /test-health] --> M{Mode?}
    M --> |quick| Q[Quick Mode]
    M --> |--full| F[Full Mode]

    Q --> Q1[Test Inventory]
    Q1 --> Q2[Consume Coverage Artifacts]
    Q2 --> Q3[Trend Delta]
    Q3 --> QR[Quick Dashboard]

    F --> A[Phase A: /check-coverage]
    A --> B[Phase B: Coverage Collection]
    B --> C[Phase C: /codex-test-review]
    C --> D[Phase D: Aggregate Dashboard]
    D --> T[Trend Snapshot]
    T --> FR[Full Dashboard]

Modes

| Mode | Trigger | Content | Duration | |------|---------|---------|----------| | quick (default) | /test-health | Test inventory + consume artifacts + trend delta | <15s | | full | /test-health --full | Phase A→B→C→D (feature coverage + instrumentation + qualitative + aggregation) | 2-5min |

Quick Mode Workflow

1. Test Inventory: Count test files by layer using Glob (see references/test-count-parsers.md for layer classification). If --scope <path> specified, limit Glob to that directory. If verify-runner cache exists (.claude/cache/verify/), read historical logs for test counts.
2. Coverage Artifacts: Scan for existing coverage artifacts (see references/artifact-formats.md). If --scope specified, scan within scope only. Never execute project commands in quick mode.
3. Trend Delta: Read previous snapshot, compute delta (see references/trend-schema.md). Skip if --no-trend flag is set.
4. Output: Quick Dashboard.

Full Mode Workflow

Phase A: Feature Coverage

Resolve docs path using scripts/resolve-feature.sh (same cascade as other skills). If has_tech_spec=true, dispatch /check-coverage <docs_path> via Skill tool. If feature not resolved or no tech spec, skip Phase A with advisory: "Phase A skipped: no feature docs detected".

Phase B: Test Inventory + Coverage Collection

1. Count test files by layer (same as quick mode)
2. If --collect flag: execute project coverage command (test:coverage or coverage from package.json)
3. Otherwise: consume existing coverage artifacts (same as quick mode)
4. Parse test runner stdout for test counts (see references/test-count-parsers.md)

Phase C: Qualitative Review

Dispatch /codex-test-review via Skill tool for 5-dimension quality assessment.

Phase D: Aggregate + Trend

1. Aggregate all dimensions into full dashboard
2. Write trend snapshot (see references/trend-schema.md)
3. Output Full Dashboard

Coverage Collection Strategy (Consume-First)

| Priority | Method | Trigger | Output | |----------|--------|---------|--------| | 1 | Consume existing artifact | Default (quick + full) | source_type: instrumented_artifact | | 2 | Run project coverage command | --collect flag only (opt-in) | source_type: collected_now | | 3 | Heuristic proxy (test/source file ratio) | No artifact and no --collect | source_type: heuristic |

Prohibited: Never auto-install coverage tools (c8, nyc, istanbul, pytest-cov, tarpaulin, jacoco).

Output: Quick Dashboard

## Test Health (Quick)

### Test Inventory
| Layer | Files | Tests | Source |
|-------|-------|-------|--------|
| Unit  | 25    | 47    | cached_stdout |
| Integration | 1 | 12  | cached_stdout |
| E2E   | 0     | —     | file_count |

### Code Coverage
| Metric | Value | Tool | Freshness |
|--------|-------|------|-----------|
| Lines  | 82.3% | c8   | current   |
| Branches | 76.0% | c8 | current   |

### Trend (vs previous)
| Metric | Previous | Current | Delta |
|--------|----------|---------|-------|
| Line coverage | 80.2% | 82.3% | +2.1% |
| Test count | 57 | 59 | +2 |

### Quick Verdicts
| Dimension | Status |
|-----------|--------|
| Has tests for changed files | OK |
| Coverage artifact exists | OK |
| Trend direction | Improving |

Output: Full Dashboard

## Test Health Report (Full)

### Phase A: Feature Coverage
(from /check-coverage): 12/15 documented features have tests (80%)

### Phase B: Code Coverage + Inventory
| Layer | Files | Tests | Passed | Failed | Duration |
|-------|-------|-------|--------|--------|----------|
| Unit  | 25    | 47    | 45     | 2      | 12s      |
| Integration | 1 | 12  | 12     | 0      | 45s      |
| E2E   | 0     | 0     | —      | —      | —        |

| Metric | Value | Source | Tool | Freshness |
|--------|-------|--------|------|-----------|
| Lines  | 82.3% | instrumented_artifact | c8 | current HEAD |
| Branches | 76.0% | instrumented_artifact | c8 | current HEAD |

### Phase C: Quality Findings
(from /codex-test-review):
| Dimension | Rating |
|-----------|--------|
| Happy path | 4/5 |
| Error handling | 3/5 |
| Edge cases | 3/5 |
| Mock quality | 4/5 |

### Phase D: Aggregate Dashboard

#### Trend (vs last 5 runs)
| Run | Date | Line Cov | Tests | Delta |
|-----|------|----------|-------|-------|
| a1b2c3d | 04-01 | 82.3% | 59 | +2.1% / +2 |
| f4e5d6c | 03-31 | 80.2% | 57 | -0.5% / +0 |

#### Verdicts
| Dimension | Status | Detail |
|-----------|--------|--------|
| Test inventory | WARN | No E2E tests |
| Code coverage | OK | 82.3% lines (instrumented) |
| Feature coverage | OK | 80% features covered |
| Quality | WARN | 1 P2 finding |
| Trend | OK | Improving over last 3 runs |
| Changed-file coverage | OK | All changed files have tests |

Anti-Coverage-Theater Guardrails

| Rule | Description | |------|-------------| | No composite score in v1 | Multi-dimensional dashboard, no single blended number | | Changed-file focus | Prioritize git diff files for coverage check | | Source transparency | Every metric tagged: instrumented / heuristic / missing | | Qualitative coupling | Full mode always runs Phase C even if quantitative metrics are green | | Tool change detection | tool_id change resets trend line | | Stale detection | Artifact older than HEAD marked stale |

Gate Policy

| Policy | Behavior | |--------|----------| | Advisory (default) | Output dashboard + verdicts, do not block | | Strict (v2, opt-in) | Changed files with zero tests block |

v1 implements advisory mode only.

Orchestrator Integration

| Skill | Interaction | Relationship | |-------|------------|-------------| | /check-coverage | Phase A: feature-doc coverage | Sub-step | | /codex-test-review | Phase C: qualitative review | Sub-step | | /verify | Phase B: reference output or trigger test:coverage | Optional sub-step | | /test-deep | Independent (execution + triage) | Peer | | /pre-pr-audit | Quick mode as non-blocking signal | Consumer |

Cross-Ecosystem Support

| Ecosystem | Detection | Coverage Artifact | Test Count Parser | |-----------|-----------|-------------------|-------------------| | Node.js | package.json | coverage/ dir (LCOV/Istanbul/Jest) | node:test / jest / vitest | | Python | pyproject.toml / setup.py | coverage.xml | pytest | | Go | go.mod | cover.out | go test -json | | Rust | Cargo.toml | tarpaulin-report.json / cobertura.xml | cargo test | | Java | build.gradle / pom.xml | build/reports/jacoco/ | gradle/maven | | Unknown | — | Scan for lcov.info / cobertura.xml | File count fallback |

Graceful degradation: no artifact + no coverage command = heuristic proxy + source_type: heuristic.

Verification

• [ ] Quick mode completes in <15s without executing project commands
• [ ] Full mode orchestrates Phase A→B→C→D in sequence
• [ ] Coverage artifact consumed correctly (or graceful fallback)
• [ ] Trend snapshot written to .claude/cache/test-health/
• [ ] Dashboard output includes all dimensions with source transparency

References

| File | Purpose | |------|---------| | references/artifact-formats.md | Coverage artifact formats + scan + freshness | | references/trend-schema.md | Trend storage schema + lock + comparison rules | | references/test-count-parsers.md | Framework output parsers + layer classification |