renatocaliari/cali-skill-validator

Skill Validator

Validate skills against 10 criteria from industry best practices.

Contents

| Section | What | When to consult | |---------|------|------------------| | Sources | Which standards this implements | When citing criteria | | 10 Rules | The validation criteria | When understanding checks | | Run Validator | Execute the validation script | Before publishing skills | | Fix Common Issues | How to fix each failure | When a check fails | | Test Cases | Prompts that should/shouldn't activate | When testing trigger accuracy |

Sources

| Source | What it defines | |--------|----------------| | AgentSkills Spec (19.5k ⭐) | Open specification: frontmatter, directory structure, progressive disclosure | | Microsoft Learn | Enterprise guidance: when to create skills, portable packages | | Addy Osmani | Directory layout, SKILL.md anatomy, references/ pattern | | Perplexity Research | 3-tier disclosure, description optimization, testing with evals | | OpenAI Skills | name + description as routing mechanism | | Claude API Docs | Trigger conditions, degrees of freedom, test with all models | | Trail of Bits | 500-line rule, what goes where | | Lalit Madan | "Operating manual for repeatable work" | | agentskills.io | Description optimization |

10 Rules

| # | Rule | Source | Severity | |---|------|--------|----------| | R1 | SKILL.md exists | AgentSkills spec | FAIL | | R2 | Frontmatter has name + description | AgentSkills spec, OpenAI | FAIL | | R3 | Description has keywords, <1024 chars | agentskills.io, Perplexity | FAIL/WARN | | R4 | SKILL.md < 500 lines | Trail of Bits | FAIL | | R5 | references/ exists when SKILL.md > 50 lines | Addy Osmani | WARN | | R6 | Examples section with I/O | Claude API, Open Agent Skills | FAIL | | R7 | Edge cases / failure handling | Claude API, Perplexity | FAIL | | R8 | Test cases (should/shouldn't activate) | Perplexity, agentskills.io | WARN | | R9 | "When to use" section in body | Claude API | WARN | | R10 | Clean directory (no loose files) | AgentSkills spec | WARN |

Run Validator

# Validate all skills in default directory
bash references/validate.sh

# Validate specific directory
bash references/validate.sh /path/to/skills

# Validate single skill
bash references/validate.sh /path/to/skills/my-skill

Output: per-skill checks with ✅/❌/⚠️ and summary with pass/fail/warn counts.

Fix Common Issues

R3: Description too short / no keywords

Bad: description: Helps with PDFs. Good: description: Extracts text and tables from PDF files, fills PDF forms, and merges multiple PDFs. Triggers when: user says "extract PDF", "fill PDF form", "merge PDFs", or mentions PDF documents.

R4: SKILL.md > 500 lines

Move detailed content to references/:

my-skill/
├── SKILL.md              # < 500 lines (core workflow)
└── references/
    ├── detailed-guide.md  # Heavy docs loaded on demand
    └── recipes.md         # Concrete examples

R6: No examples

Add section with input → output:

## Examples

### Example 1: Extract text from PDF

**Input:** "Extract text from report.pdf"

**Steps:**
1. Run: `python scripts/extract_text.py report.pdf`
2. Return extracted text

**Output:** "Extracted 3 pages, 2,847 characters"

R7: No edge cases

Add section with failure modes:

## Edge Cases

### File doesn't exist
- Check if file path is absolute or relative
- Ask user to verify path

### PDF is password-protected
- Ask user for password
- Run: `python scripts/extract_text.py --password <pwd> file.pdf`

R8: No test cases

Add section with activation prompts:

## Test Cases

### Should activate
- "Extract text from report.pdf"
- "Fill out the tax form"
- "Merge invoice1.pdf and invoice2.pdf"

### Should NOT activate
- "Convert PDF to Word" (different skill)
- "Read this text file" (not a PDF)

Test Cases

Should activate

• "Validate my skills"
• "Check if my skills are good"
• "Run skill audit"
• "Are my skills following best practices?"

Should NOT activate

• "Create a skill" (use skill-creator)
• "Edit SKILL.md" (use edit tool)
• "What is a skill?" (general question)