oimiragieo/proactive-audit

Proactive Audit

Overview

Automated health checks for framework artifacts that were modified during the current pipeline. This skill fills the gap between reactive verification (tests, lint) and proactive framework-level validation (wiring, syntax, security patterns).

Core principle: Framework artifact changes require the same rigor as code changes. If a skill was created, verify it is wired. If a hook was modified, verify it compiles. If an agent was changed, verify its tool/skill lists are consistent.

When to Invoke

Invoke this skill as the final pipeline step whenever ANY of the following paths were created, modified, or deleted during the session:

• .claude/hooks/**/*.cjs
• .claude/skills/**/SKILL.md
• .claude/agents/**/*.md
• .claude/workflows/**/*.md
• .claude/schemas/**/*.json
• .claude/templates/**/*
• .claude/CLAUDE.md
• .claude/lib/routing/routing-table.cjs

Invocation:

Skill({ skill: 'proactive-audit' });

Mandatory Skills

| Skill | Purpose | When | | -------------------------------- | ---------------------------------- | -------------------- | | task-management-protocol | Track audit progress | Always | | ripgrep | Fast targeted artifact search | During checks | | code-semantic-search | Pattern discovery across artifacts | When investigating | | context-compressor | Compress large audit results | When output is large | | verification-before-completion | Gate completion on zero CRITICAL | Before marking done | | memory-search | Check prior audit patterns | At start |

Step 1: Detect Changed Artifacts

Use git diff to identify which framework artifacts changed in this session:

# Primary: git diff against recent commits
git diff --name-only HEAD~5 -- .claude/hooks/ .claude/skills/ .claude/agents/ .claude/workflows/ .claude/schemas/ .claude/templates/ .claude/CLAUDE.md .claude/lib/routing/

# Secondary: check unstaged changes
git diff --name-only -- .claude/hooks/ .claude/skills/ .claude/agents/ .claude/workflows/ .claude/schemas/ .claude/templates/

# Tertiary: check untracked files
git ls-files --others --exclude-standard .claude/hooks/ .claude/skills/ .claude/agents/ .claude/workflows/ .claude/schemas/ .claude/templates/

Combine all three lists into a deduplicated set of changed artifact paths.

Step 2: Apply Check Matrix

For each changed artifact, apply the relevant checks from this matrix:

Hook Files (.claude/hooks/**/*.cjs)

| Check ID | Check | Command | Severity | | -------- | ------------------------------------------- | --------------------------------------------------------------- | -------- | | H-01 | Syntax validity | node --check <file> | CRITICAL | | H-02 | SE-02: raw JSON.parse without safeParseJSON | grep -n "JSON.parse(" <file> then verify safeParseJSON import | HIGH | | H-03 | SE-01: shell injection via shell: true | grep -n "shell:\\s*true" <file> | HIGH | | H-04 | Hook registered in settings.json | grep "<hook-filename>" .claude/settings.json | MEDIUM | | H-05 | Exit code correctness | Verify try/catch wrapping, exit 0 on non-critical errors | MEDIUM |

H-02 detail: If JSON.parse( is found, check if the file also imports safeParseJSON from .claude/lib/utils/safe-json.cjs. If not, flag as HIGH finding. Exclude test files (*.test.cjs).

Skill Files (.claude/skills/**/SKILL.md)

| Check ID | Check | Command | Severity | | -------- | ------------------------------------------- | --------------------------------------------------------- | -------- | | S-01 | Skill appears in skill-catalog.md | grep "<skill-name>" .claude/docs/skill-catalog.md | HIGH | | S-02 | At least one agent has skill in frontmatter | grep -r "<skill-name>" .claude/agents/ --include="*.md" | MEDIUM | | S-03 | Skill appears in CLAUDE.md Section 8.5 | grep "<skill-name>" .claude/CLAUDE.md | MEDIUM | | S-04 | SKILL.md has valid frontmatter | Verify name:, description:, version: fields exist | MEDIUM | | S-05 | Validate skills (if available) | pnpm validate:skills 2>&1 | LOW |

Agent Files (.claude/agents/**/*.md)

| Check ID | Check | Command | Severity | | -------- | ----------------------------------------------- | ------------------------------------------------------------------------------ | -------- | | A-01 | Agent appears in agent-registry.json | grep "<agent-name>" .claude/context/agent-registry.json | HIGH | | A-02 | Agent's skills: list references existing skills | For each skill in frontmatter, verify .claude/skills/<skill>/SKILL.md exists | MEDIUM | | A-03 | Agent's tools: list contains only valid tools | Verify each tool name against known tool list | MEDIUM | | A-04 | Agent appears in CLAUDE.md routing table | grep "<agent-name>" .claude/CLAUDE.md | MEDIUM |

Workflow Files (.claude/workflows/**/*.md)

| Check ID | Check | Command | Severity | | -------- | -------------------------------------------- | ------------------------------------------------------------ | -------- | | W-01 | Workflow referenced in WORKFLOW_AGENT_MAP.md | grep "<workflow-name>" .claude/docs/@WORKFLOW_AGENT_MAP.md | MEDIUM | | W-02 | Referenced agents exist | For each agent name in workflow, verify agent file exists | MEDIUM |

Schema Files (.claude/schemas/**/*.json)

| Check ID | Check | Command | Severity | | -------- | ----------------------------------- | --------------------------------------------------------------------------- | -------- | | SC-01 | Valid JSON syntax | node -e "JSON.parse(require('fs').readFileSync('<file>', 'utf8'))" | CRITICAL | | SC-02 | Schema appears in schema-catalog.md | grep "<schema-name>" .claude/context/artifacts/catalogs/schema-catalog.md | MEDIUM |

Root Cleanliness Check

ls -1 | grep -cvE '^(\.|node_modules|src|tests|scripts|dist|build|docs|package\.json|package-lock\.json|pnpm-lock\.yaml|tsconfig|eslint|prettier|jest|vitest|README|LICENSE|CHANGELOG|CLAUDE\.md|\.env)'

FAIL if the count is greater than 0.

Known slop patterns (any of these in project root = FAIL):

• *-debug*.txt, *-debug*.log, debug-*.json
• dump-*.cjs, dump-*.js, dump-*.json
• rename_*.cjs, revert_*.cjs, update_*.cjs
• test-out.txt, lint-output.txt, eslint.json, errors.json
• UUID-named files (e.g. a3f2c1b0-*.json)
• new_session_analysis.md or any *.analysis.md not under .claude/context/
• Any .cjs/.js/.mjs not referenced in package.json scripts or tracked project source
• Any .md not named README.md, CLAUDE.md, LICENSE, or CHANGELOG.md

Action when FAIL:

1. Delete the offending files (or move to .claude/context/tmp/ if content may be needed)
2. Log deletion to session-gap-log.jsonl with type: "cleanup"
3. Append a reflection-spawn-request.json entry with trigger: "ai-slop-found" so the root cause is investigated

Reference: .claude/rules/cleanup-always.md

Documentation Staleness Check

After any feature work, verify:

1. CHANGELOG.md — does it have an entry dated within the last session?

   • Check: git log --oneline -5 vs grep "## \[" CHANGELOG.md | head -3
   • FAIL if: feature commits exist but no matching CHANGELOG entry

2. .env.example — does it document all env vars in the codebase?

   • Check: grep -r "process.env\." .claude/skills/ .claude/hooks/ | grep -oP "process\.env\.\K\w+" | sort -u
   • Compare against entries in .env.example
   • FAIL if: env var used in code but not documented in .env.example

3. README.md — are agent/skill counts current?

   • Check counts in README vs jq '.agents | length' .claude/context/agent-registry.json
   • WARN if counts diverge by more than 2

Routing Files (.claude/lib/routing/routing-table.cjs, .claude/CLAUDE.md)

| Check ID | Check | Command | Severity | | -------- | ------------------------ | ---------------------------------------------------- | -------- | | R-01 | routing-table.cjs syntax | node --check .claude/lib/routing/routing-table.cjs | CRITICAL | | R-02 | Validate skills (full) | pnpm validate:skills 2>&1 | MEDIUM |

Step 3: Generate Report

Write a structured report to .claude/context/reports/ecosystem-audit/proactive-audit-{ISO-date}.md with this format:

<!-- Agent: qa | Task: #N | Session: YYYY-MM-DD -->

# Proactive Audit Report

**Date:** YYYY-MM-DD
**Artifacts Scanned:** N
**Findings:** N CRITICAL, N HIGH, N MEDIUM, N LOW
**Overall:** PASS | FAIL

## Changed Artifacts

- path/to/artifact1 (type: hook)
- path/to/artifact2 (type: skill)

## Findings

### CRITICAL

| ID   | File          | Check  | Detail                 | Remediation      |
| ---- | ------------- | ------ | ---------------------- | ---------------- |
| H-01 | hooks/foo.cjs | Syntax | SyntaxError at line 42 | Fix syntax error |

### HIGH

| ID   | File          | Check | Detail                                      | Remediation                                               |
| ---- | ------------- | ----- | ------------------------------------------- | --------------------------------------------------------- |
| H-02 | hooks/bar.cjs | SE-02 | JSON.parse at line 15 without safeParseJSON | Import safeParseJSON from .claude/lib/utils/safe-json.cjs |

### MEDIUM

(same table format)

### PASS

| ID   | File          | Check  | Result |
| ---- | ------------- | ------ | ------ |
| H-01 | hooks/baz.cjs | Syntax | OK     |

## Summary

- Total checks run: N
- Passed: N
- Failed: N
- Pass rate: N%

Step 4: Return Verdict

After generating the report:

• If ANY CRITICAL findings exist: return FAIL with the report path and list of critical findings
• If ANY HIGH findings exist: return WARN with the report path and list of high findings
• If only MEDIUM/LOW findings: return PASS with the report path and note about medium findings
• If no findings: return PASS with the report path

Iron Laws

1. ALWAYS run every applicable check from the check matrix — skipping "small" changes is how undetected wiring failures accumulate across sessions.
2. NEVER trust task metadata alone for change detection — use git diff as the primary source of changed artifact paths.
3. NEVER report PASS without actually executing each check command — self-attested PASS without evidence violates verification-before-completion.
4. NEVER ignore SE-02 (prototype pollution) findings in hook files — a single compromised hook can corrupt all subsequent tool calls in the pipeline.
5. ALWAYS validate hook syntax with node --check before reporting findings — broken hooks silently block the entire tool pipeline.

Anti-Patterns

| Anti-Pattern | Why It Fails | Correct Approach | | --------------------------------------------- | ---------------------------------------------------------------------------- | ----------------------------------------------------------------------- | | Skipping checks for "small" changes | Small wiring failures accumulate silently until a pipeline breaks | Run all checks regardless of perceived change size | | Trusting task metadata for change detection | Metadata can be incomplete or stale; misses unstaged changes | Use git diff --name-only + git ls-files --others as primary source | | Self-attesting PASS without running commands | Unverified PASS masks real failures; violates verification-before-completion | Execute every check command and capture output as evidence | | Ignoring SE-02 (prototype pollution) in hooks | One polluted hook corrupts Object.prototype globally across all tool calls | Flag SE-02 as HIGH severity and block pipeline until fixed | | Reporting findings without remediation steps | Developers know what broke but not how to fix it | Include specific remediation for every finding with file+line reference |

Severity Guide

| Severity | Meaning | Action Required | | -------- | -------------------------------------- | ------------------------------------------ | | CRITICAL | Framework will break | Fix immediately, block pipeline completion | | HIGH | Security risk or invisible artifact | Fix before next session, warn user | | MEDIUM | Missing integration, incomplete wiring | Fix in follow-up task | | LOW | Best practice violation, cosmetic | Track for future improvement |

Integration with Router Step 0.7

The router invokes this skill via Step 0.7 in the Router Output Contract (CLAUDE.md Section 0.1). The router:

1. Detects that framework artifacts were modified during the pipeline
2. Spawns a QA agent with this skill as the final pipeline step
3. Reads the audit report
4. If CRITICAL findings: spawns developer to fix them before claiming completion
5. If HIGH findings: warns user and notes findings in pipeline summary
6. If PASS: proceeds to claim pipeline completion

Related Skills

• verification-before-completion -- General evidence-based completion gates
• checklist-generator -- IEEE 1028 quality checklists
• sharp-edges -- Known hazard patterns (SE-01 through SE-07)

Related References

• .claude/context/plans/proactive-audit-design-2026-02-22.md -- Design document
• .claude/rules/security.md -- SE-01 and SE-02 patterns
• .claude/rules/artifact-integration.md -- Must-have integration requirements