troykelly/documentation-audit

Documentation Audit

Overview

Comprehensive documentation sync when drift is detected. Analyzes codebase and creates or updates all documentation artifacts to achieve full synchronization.

Core principle: Documentation must reflect reality. This skill brings them into alignment.

Announce at start: "I'm using documentation-audit to synchronize all documentation with the current codebase."

When This Skill Triggers

This skill is invoked when:

| Trigger | Source | |---------|--------| | API documentation drift | api-documentation skill | | Features documentation drift | features-documentation skill | | Missing documentation files | Any documentation check | | Manual request | User or orchestrator |

The Audit Process

Phase 1: Discovery

echo "=== Documentation Audit: Discovery Phase ==="

# Find all documentation files
DOC_FILES=$(find . -name "*.md" -o -name "*.yaml" -o -name "*.json" | \
  grep -E "(doc|api|swagger|openapi|feature|guide|readme)" | \
  grep -v node_modules | grep -v .git)

echo "Found documentation files:"
echo "$DOC_FILES"

# Find API documentation
API_DOC=$(find . -name "openapi.yaml" -o -name "swagger.yaml" -o -name "openapi.json" | head -1)
echo "API Documentation: ${API_DOC:-MISSING}"

# Find features documentation
FEATURE_DOC=$(find . -name "features.md" -o -name "FEATURES.md" | head -1)
echo "Features Documentation: ${FEATURE_DOC:-MISSING}"

Phase 2: API Audit

If API endpoints exist:

echo "=== Documentation Audit: API Phase ==="

# Detect API framework
detect_api_framework() {
  if grep -r "express" package.json 2>/dev/null; then echo "express"; return; fi
  if grep -r "fastify" package.json 2>/dev/null; then echo "fastify"; return; fi
  if grep -r "@nestjs" package.json 2>/dev/null; then echo "nestjs"; return; fi
  if grep -r "fastapi" requirements.txt 2>/dev/null; then echo "fastapi"; return; fi
  if grep -r "flask" requirements.txt 2>/dev/null; then echo "flask"; return; fi
  if [ -f "go.mod" ]; then echo "go"; return; fi
  echo "unknown"
}

FRAMEWORK=$(detect_api_framework)
echo "Detected framework: $FRAMEWORK"

# Extract all endpoints from code
extract_endpoints() {
  case "$FRAMEWORK" in
    express|fastify)
      grep -rh "app\.\(get\|post\|put\|delete\|patch\)" --include="*.ts" --include="*.js" | \
        sed "s/.*app\.\([a-z]*\)('\([^']*\)'.*/\1 \2/"
      ;;
    nestjs)
      grep -rh "@\(Get\|Post\|Put\|Delete\|Patch\)" --include="*.ts" | \
        sed "s/.*@\([A-Za-z]*\)('\([^']*\)'.*/\1 \2/"
      ;;
    fastapi)
      grep -rh "@app\.\(get\|post\|put\|delete\|patch\)" --include="*.py" | \
        sed "s/.*@app\.\([a-z]*\)(\"\([^\"]*\)\".*/\1 \2/"
      ;;
    *)
      echo "Unknown framework - manual inspection required"
      ;;
  esac
}

ENDPOINTS=$(extract_endpoints)
echo "Found endpoints:"
echo "$ENDPOINTS"

Phase 3: Features Audit

echo "=== Documentation Audit: Features Phase ==="

# Find user-facing components
find_features() {
  # React components in pages/views
  find . -path "*/pages/*" -name "*.tsx" -o -path "*/views/*" -name "*.tsx" | \
    xargs -I {} basename {} .tsx 2>/dev/null

  # Feature flags
  grep -rh "featureFlag\|feature:" --include="*.ts" --include="*.tsx" | \
    sed "s/.*['\"]feature[':]\s*['\"]?\([^'\"]*\)['\"]?.*/\1/" 2>/dev/null

  # Config options exposed to users
  grep -rh "config\.\|settings\." --include="*.ts" | \
    grep -v "import\|require" | \
    sed "s/.*\(config\|settings\)\.\([a-zA-Z]*\).*/\2/" 2>/dev/null | sort -u
}

FEATURES=$(find_features)
echo "Discovered features:"
echo "$FEATURES"

Phase 4: Generate Missing Documentation

Create OpenAPI if Missing

# Template for new OpenAPI file
openapi: 3.0.3
info:
  title: [PROJECT_NAME] API
  description: |
    API documentation for [PROJECT_NAME].
    Generated by documentation-audit skill.
  version: 1.0.0
  contact:
    name: API Support
servers:
  - url: http://localhost:3000
    description: Development server
  - url: https://api.example.com
    description: Production server
paths:
  # Endpoints will be added here
components:
  schemas:
    Error:
      type: object
      properties:
        code:
          type: string
        message:
          type: string
  securitySchemes:
    bearerAuth:
      type: http
      scheme: bearer
      bearerFormat: JWT

Create Features Doc if Missing

# Features

> Generated by documentation-audit skill. Update with accurate descriptions.

## Overview

[Brief description of the product]

## Core Features

### [Feature 1]

**Description:** [What it does]

**How to Use:**
1. [Step 1]
2. [Step 2]

---

## Additional Features

[List additional features discovered]

---

*Last updated: [DATE]*
*Note: This file was auto-generated. Review and enhance descriptions.*

Phase 5: Update Existing Documentation

For each discovered but undocumented item:

1. API Endpoints - Add to OpenAPI spec with:

   • Path and method
   • Parameters (from function signature)
   • Request body schema (from DTO/type)
   • Response schema (from return type)
   • Basic description

2. Features - Add to features doc with:

   • Feature name
   • Basic description
   • Placeholder for how-to-use
   • Note to review and enhance

Phase 6: Validation

echo "=== Documentation Audit: Validation Phase ==="

# Validate OpenAPI
if [ -f "openapi.yaml" ]; then
  yq '.' openapi.yaml > /dev/null 2>&1 && echo "OpenAPI: Valid YAML" || echo "OpenAPI: Invalid YAML"
fi

# Validate Markdown
for file in docs/*.md; do
  if [ -f "$file" ]; then
    # Check for required sections
    if ! grep -q "^## " "$file"; then
      echo "WARNING: $file missing section headers"
    fi
  fi
done

# Check completeness
ENDPOINTS_DOCUMENTED=$(yq '.paths | keys | length' openapi.yaml 2>/dev/null || echo 0)
ENDPOINTS_IN_CODE=$(extract_endpoints | wc -l)

echo "Endpoints in code: $ENDPOINTS_IN_CODE"
echo "Endpoints documented: $ENDPOINTS_DOCUMENTED"

if [ "$ENDPOINTS_DOCUMENTED" -lt "$ENDPOINTS_IN_CODE" ]; then
  echo "WARNING: Some endpoints still undocumented"
fi

Audit Report

Post audit results to GitHub issue:

## Documentation Audit Complete

**Audit Date:** [ISO_TIMESTAMP]
**Triggered By:** [api-documentation|features-documentation|manual]

### Summary

| Category | Before | After | Status |
|----------|--------|-------|--------|
| API Endpoints | [N] undocumented | [N] documented | [COMPLETE/PARTIAL] |
| Features | [N] undocumented | [N] documented | [COMPLETE/PARTIAL] |
| General Docs | [N] missing | [N] created | [COMPLETE/PARTIAL] |

### Files Created
- `openapi.yaml` - API documentation
- `docs/features.md` - Features documentation

### Files Updated
- `openapi.yaml` - Added [N] endpoints
- `docs/features.md` - Added [N] features

### Requires Manual Review
- [ ] Verify API response schemas
- [ ] Enhance feature descriptions
- [ ] Add usage examples
- [ ] Review security documentation

### Next Steps
1. Review generated documentation
2. Add detailed descriptions
3. Include examples
4. Validate with stakeholders

---
*documentation-audit skill completed*

Checklist

During audit:

• [ ] All documentation files discovered
• [ ] API framework detected
• [ ] All endpoints extracted from code
• [ ] All features extracted from code
• [ ] Missing documentation files created
• [ ] Existing documentation updated
• [ ] All files validated
• [ ] Audit report posted to issue
• [ ] Changes committed

Quality Standards

Generated documentation must meet:

| Standard | Requirement | |----------|-------------| | Completeness | Every endpoint/feature listed | | Validity | YAML/JSON validates | | Structure | Required sections present | | Placeholders | Clear markers for manual review | | Attribution | Generated by skill noted |

Integration

This skill is invoked by:

| Skill | When | |-------|------| | api-documentation | API drift detected | | features-documentation | Features drift detected | | issue-driven-development | Documentation step |

This skill uses:

| Tool | Purpose | |------|---------| | Glob/Grep | Discover code artifacts | | Read | Analyze existing docs | | Write | Create new docs | | Edit | Update existing docs | | yq | Validate YAML | | jq | Validate JSON |