beam-ai-team/validate-skill-functionality

Validate Skill Functionality

Archived skill. Kept for reference only; do not expose as a public active catalog skill without rewriting around registry validation and current skill QA.

Purpose: Systematic post-execution review to validate skill functionality and document findings.

Safety Contract

Before writing validation reports, modifying skill files, or updating follow-up tasks, show the skill under review, evidence sources, destination path, and proposed changes. Require explicit user approval in the current turn before writing files or modifying skill artifacts. Reviewing execution evidence and reporting findings in chat do not require approval.

When to Use: After a skill has completed its full execution workflow

Workflow

Follow these steps to validate skill functionality:

Step 1: Review Execution Context

• Identify which skill was just executed
• Review what the skill was supposed to accomplish
• Check the SKILL.md to understand expected behavior
• Review conversation history to identify all tool calls made during execution

Step 2: Validate File Loading

Check that all required files were loaded correctly:

• Review all Read tool calls in the conversation
• Verify SKILL.md was loaded (for skill execution context)
• Check if skill references other files (references/, scripts/, assets/)
• Confirm referenced files were actually loaded when needed
• Look for "File not found" errors or truncated reads
• Verify file paths match expected locations

Example checks:

[OK] SKILL.md loaded: Yes (line 1-88, complete)
[OK] references/workflow.md loaded: Yes (when needed in Step 2)
[ERROR] references/error-handling.md loaded: No (should have been loaded but wasn't)
[OK] scripts/bulk-complete.py executed: Yes (correct parameters)

Step 3: Validate Skill Nesting/Wrapping

Check if skills correctly loaded nested skills:

• Identify if the skill called other skills (e.g., execute-project calls create-skill)
• Verify nested skills were loaded using beam-next-loader.py or explicit Read
• Confirm nested skill workflows were followed correctly
• Check that context was passed properly between skills
• Validate that nested skill outputs fed back correctly

Example checks:

Primary Skill: execute-project
  [OK] Loaded: Yes (via beam-next-loader.py --skill execute-project)

  Nested Skill: create-skill
    [OK] Loaded: Yes (via beam-next-loader.py --skill create-skill)
    [OK] SKILL.md read: Yes (complete)
    [OK] Workflow followed: Yes (all 7 steps)
    [OK] Context passed: Yes (user's workflow → create-skill)

  Nested Skill: close-session
    [OK] Loaded: Yes (auto-triggered)
    [OK] workflow.md loaded: Yes (as required)
    [OK] All 8 steps executed: Yes

Step 4: Verify Expected Outputs

• Confirm the skill completed its workflow
• Check that outputs match expectations
• Verify all steps executed correctly
• Validate files were created/modified as expected

Step 5: Check for Errors or Edge Cases

• Look for any errors or warnings during execution
• Identify edge cases or unexpected behavior
• Note any deviations from expected workflow
• Check for incomplete reads or missing context

Step 6: Report Findings (≤5 lines)

Report to user verbally:

• [OK] What worked
• [ERROR] Issues found (if any)
• 💡 Recommendations (if any)

NO documentation files - Follow orchestrator.md ≤5 line rule