opendatahub-io/doc-gap

doc-gap

Assess whether the gathered context is sufficient to produce quality documentation.

Prerequisites

workspace/context-package.json must exist (produced by doc-gather).

Parse arguments

$ARGUMENTS optionally contains a component name to focus the analysis on. If empty, analyze all components found in the context package.

Input validation:

• If a component name is provided, validate it against the component list in workspace/context-package.json.
• Accept only exact matches against known component names.
• If the component name is not found, halt with a clear error and do not run LLM assessment.

Step 1: Read context package

Read workspace/context-package.json and extract:

• Ticket metadata (summary, components, fix_versions)
• List of gathered context files with their source types
• Product configuration (docs conventions)

Step 2: Deterministic coverage checks

Perform these checks without LLM judgment:

1. Component coverage: For each component in the ticket, check if at least one context file from that component's repo exists.
2. Documentation existence: Check if existing documentation files are present in the context.
3. API reference availability: If the ticket involves API changes, check for CRD type definitions or API spec files.
4. Source code presence: Check if implementation source code is included.
5. Architecture docs: Check for architecture context files.

Record each check as a finding with pass/fail status.

Step 3: LLM assessment

Read the gap analysis prompt from ${CLAUDE_SKILL_DIR}/prompts/gap-analysis.md.

Construct an LLM prompt combining:

• The gap analysis prompt template
• Ticket metadata from the context package
• Summary of gathered files (file paths, source types, relevance scores)
• Content snippets from the highest-scored files (first 500 chars each, up to 20 files), after deterministic secret/PII redaction
• Results of deterministic checks from Step 2

Before prompt assembly, apply deterministic redaction to all snippets:

• Detect and mask secrets (API keys, tokens, passwords, private keys, kubeconfig credentials) with consistent placeholders (e.g., <REDACTED_TOKEN_1>).
• Mask PII fields (emails, phone numbers) unless explicitly required.
• Record redaction statistics (number of snippets scanned, items redacted by type) for inclusion in the gap report.

Ask the LLM to assess:

• Is the context sufficient to write accurate docs?
• What specific information is missing?
• What is the recommendation: proceed, gather-more, or stop?

Step 4: Synthesize findings

Combine deterministic check results with LLM assessment into a unified gap report.

Step 5: Write gap report

Write workspace/gap-report.json with this structure:

{
    "recommendation": "proceed",
    "confidence": 0.82,
    "summary": "Context is sufficient for basic documentation...",
    "deterministic_checks": [
        {
            "check": "component_coverage",
            "status": "pass",
            "details": "Found context for 2/2 ticket components"
        }
    ],
    "gaps": [
        {
            "severity": "medium",
            "category": "examples",
            "description": "No sample YAML configurations found",
            "impact": "Documentation will lack concrete examples",
            "suggestion": "Check component repo for example/ directory"
        }
    ],
    "existing_coverage": [
        {
            "topic": "Model serving overview",
            "source": "modules/serving/pages/con_model-serving.adoc",
            "quality": "sufficient"
        }
    ],
    "analyzed_at": "2026-04-14T10:35:00Z"
}

Schema requirements:

• Required keys: recommendation, confidence, summary, deterministic_checks, gaps, existing_coverage, analyzed_at.
• recommendation MUST be one of: proceed, gather-more, stop.
• confidence MUST be a float in [0.0, 1.0].
• If schema validation fails, halt and return an error instead of writing a partial report.

Output

Primary: workspace/gap-report.json Report to caller: recommendation, confidence score, number of gaps by severity.

Gotchas

• The workspace/context-package.json file must already exist (produced by doc-gather) — this skill will not create it.
• Component names must exactly match entries in the context package; partial or fuzzy matches are rejected.
• Snippets are redacted for secrets and PII before LLM assessment, so redaction artifacts in the gap report are expected, not errors.