athola/tiered-audit

Tiered Audit

Table of Contents

• When to Use
• When NOT to Use
• Tier 1: Git History Audit
• Tier 2: Targeted Area Audit
• Tier 3: Full Codebase Audit
• Output Contract

When To Use

• Auditing codebase quality, patterns, or problems
• Reviewing what changed on a branch before merge
• Investigating areas of instability or churn
• Pre-PR quality assessment

When NOT to Use

• Reviewing a specific file (use pensive:code-reviewer)
• Architecture-only review (use pensive:architecture-review)
• Single-commit review (use imbue:diff-analysis)

Tier 1: Git History Audit

Always runs first. Analyzes git log, diff stats, and blame to identify areas of concern without reading any source files.

What Tier 1 Analyzes

Run these git commands for the target commit range (default: current branch vs main):

# 1. Churn hotspots: files changed most often
git log --format="" --name-only {base}..HEAD \
  | sort | uniq -c | sort -rn | head -20

# 2. Diff stats: size of changes per file
git diff --stat {base}..HEAD

# 3. Fix-on-fix patterns: commits fixing previous commits
git log --oneline {base}..HEAD \
  | grep -iE "(fix|revert|patch|hotfix)"

# 4. New file clusters: modules with many new files
git diff --name-status {base}..HEAD \
  | grep "^A" | cut -f2 \
  | sed 's|/[^/]*$||' | sort | uniq -c | sort -rn

# 5. Large commits: single commits with big diffs
git log --format="%h %s" --shortstat {base}..HEAD

Verification: Confirm each command produces output. If a command returns empty, the commit range may be wrong; verify {base} resolves correctly with git merge-base.

Tier 1 Output Format

Write findings to .coordination/agents/tier1-audit.findings.md:

---
agent: tier1-audit
tier: 1
evidence_count: {N}
---

## Summary

{1-2 sentence overview of what the git history reveals}

## Churn Hotspots

{top 10 most-changed files with change counts}

For each flagged file, include:
- Location: path/to/file.py:line (most-changed function or block)
- Anchor: `verbatim source text at that line`

[E1] Command: git log --format="" --name-only ...
     Output: {relevant output}

## Fix-on-Fix Patterns

{commits that fix previous commits in the same area}

[E2] Command: git log --oneline ... | grep -iE ...
     Output: {relevant output}

## New File Clusters

{modules with 5+ new files}

## Large Diffs

{commits with 200+ line changes}

## Escalation Recommendation

{list of areas flagged for Tier 2, or "no escalation needed"}

Escalation Decision

After Tier 1 completes, check findings against the escalation criteria in modules/escalation-criteria.md.

If NO criteria are met: audit is complete. Report findings.

If criteria ARE met: list flagged areas and proceed to Tier 2 for each area sequentially.

Tier 2: Targeted Area Audit

Runs only for areas flagged by Tier 1. Each flagged area is audited one at a time, not in parallel.

What Tier 2 Analyzes

For each flagged area:

1. Read the source files in the area
2. Check for patterns, anti-patterns, bugs
3. Verify test coverage exists
4. Check documentation currency
5. Assess architectural fit

Tier 2 Output Format

One findings file per area: .coordination/agents/tier2-{area-name}.findings.md

Each file follows the output contract for audits (see imbue:proof-of-work/modules/output-contracts).

Tier 3: Full Codebase Audit

Requires explicit user approval. See modules/escalation-criteria.md for the gate protocol.

Tier 3 should use dedicated sessions (one per area) with file-based coordination, NOT parallel subagents.

Output Contract

All tiers use this contract:

output_contract:
  required_sections:
    - summary
    - evidence
  min_evidence_count: 3    # Tier 1
  # min_evidence_count: 8  # Tier 2
  expected_artifacts: []
  retry_budget: 1
  strictness: normal

Tier 2 raises the minimum evidence count to 8 because it reads source files and should produce deeper analysis.

Verification: After each tier completes, verify the findings file exists and contains at least the minimum evidence count ([E1], [E2], etc.) before proceeding to the next tier or reporting results.

Verify Findings Are Grounded (tiered-audit:findings-verified)

Every finding must cite a real location and a verbatim anchor. Write findings to .review/findings.json and confirm each citation resolves:

python plugins/imbue/scripts/citation_verifier.py \
  --findings .review/findings.json --repo-root .

Drop or label UNVERIFIED any finding the verifier fails (exit 1); only verified findings enter the report. See Skill(imbue:review-core) Step 5 and Skill(imbue:structured-output) for the schema.

Exit Criteria

• [ ] Tier 1 findings file exists at .coordination/agents/tier1-audit.findings.md and contains at least 3 evidence entries ([E1]–[E3]).
• [ ] Tier 2 is only started for areas explicitly flagged by Tier 1 escalation criteria.
• [ ] Every reported finding carries a Location + verbatim Anchor confirmed by citation_verifier.py (exit 0), or unverified findings were dropped or labeled UNVERIFIED.