beam-ai-team/validate-skill-functionality

Validate Skill Functionality

Purpose: Systematic post-execution review to validate skill functionality and document findings.

When to Use: After a skill has completed its full execution workflow

Workflow

Follow these steps to validate skill functionality:

Step 1: Review Execution Context

• Identify which skill was just executed
• Review what the skill was supposed to accomplish
• Check the SKILL.md to understand expected behavior
• Review conversation history to identify all tool calls made during execution

Step 2: Validate File Loading

Check that all required files were loaded correctly:

• Review all Read tool calls in the conversation
• Verify SKILL.md was loaded (for skill execution context)
• Check if skill references other files (references/, scripts/, assets/)
• Confirm referenced files were actually loaded when needed
• Look for "File not found" errors or truncated reads
• Verify file paths match expected locations

Example checks:

[OK] SKILL.md loaded: Yes (line 1-88, complete)
[OK] references/workflow.md loaded: Yes (when needed in Step 2)
[ERROR] references/error-handling.md loaded: No (should have been loaded but wasn't)
[OK] scripts/bulk-complete.py executed: Yes (correct parameters)

Step 3: Validate Skill Nesting/Wrapping

Check if skills correctly loaded nested skills:

• Identify if the skill called other skills (e.g., execute-project calls create-skill)
• Verify nested skills were loaded using beam-next-loader.py or explicit Read
• Confirm nested skill workflows were followed correctly
• Check that context was passed properly between skills
• Validate that nested skill outputs fed back correctly

Example checks:

Primary Skill: execute-project
  [OK] Loaded: Yes (via beam-next-loader.py --skill execute-project)

  Nested Skill: create-skill
    [OK] Loaded: Yes (via beam-next-loader.py --skill create-skill)
    [OK] SKILL.md read: Yes (complete)
    [OK] Workflow followed: Yes (all 7 steps)
    [OK] Context passed: Yes (user's workflow → create-skill)

  Nested Skill: close-session
    [OK] Loaded: Yes (auto-triggered)
    [OK] workflow.md loaded: Yes (as required)
    [OK] All 8 steps executed: Yes

Step 4: Verify Expected Outputs

• Confirm the skill completed its workflow
• Check that outputs match expectations
• Verify all steps executed correctly
• Validate files were created/modified as expected

Step 5: Check for Errors or Edge Cases

• Look for any errors or warnings during execution
• Identify edge cases or unexpected behavior
• Note any deviations from expected workflow
• Check for incomplete reads or missing context

Step 6: Report Findings (≤5 lines)

Report to user verbally:

• [OK] What worked
• [ERROR] Issues found (if any)
• 💡 Recommendations (if any)

NO documentation files - Follow orchestrator.md ≤5 line rule