shieldon-dev/skills-audit

Skills Audit

Security scanner that audits AI agent skill files before installation. Detects credential theft, data exfiltration, dangerous commands, and obfuscation using pattern matching. Everything runs locally — no data leaves your machine.

When to Use

Before installing any new skill or MCP server, scan it first. This catches embedded secrets, reverse shells, data exfiltration endpoints, obfuscated payloads, and other threats that hide in skill files.

How to Run

Three input modes:

Scan a local file

python scripts/scan.py --file /path/to/SKILL.md

Fetch and scan a remote skill by URL

python scripts/scan.py --url https://example.com/SKILL.md

Scan raw content directly

python scripts/scan.py --content "<paste skill text here>"

Output Format

JSON with risk score, severity level, findings, and recommendation:

{
  "scan_id": "a1b2c3d4-...",
  "risk_score": 75,
  "risk_level": "HIGH",
  "recommendation": "BLOCK",
  "findings": [
    {
      "type": "regex",
      "description": "AWS access key pattern detected",
      "severity": "HIGH",
      "evidence": "Matched Credential_AWS_Access_Key: AKIAIOSFODNN7EXAMPLE"
    }
  ]
}

Decision Logic

| Score | Risk Level | Recommendation | Action | |-------|-----------|----------------|--------| | 0-29 | LOW | SAFE | Proceed with installation | | 30-60 | MEDIUM | REVIEW | Ask human to review findings before proceeding | | 61-70 | HIGH | REVIEW | Ask human to review — multiple concerning patterns | | 71-100 | HIGH/CRITICAL | BLOCK | Do not install — refuse and explain findings |

Any finding with CRITICAL severity overrides to BLOCK regardless of total score.

What It Detects

39 detection patterns across 6 threat categories:

• Credential theft — AWS, GitHub, Stripe, OpenAI, Anthropic, Supabase, database connection strings, crypto private keys, and 30+ more secret formats
• Data exfiltration — Discord/Slack/Telegram webhooks, paste services, file drop services, HTTP client POST calls, DNS tunneling
• Dangerous commands — Reverse shells, system destruction, privilege escalation, container escape, SSH key injection, PowerShell execution, deserialization attacks
• Obfuscation — Base64 decode chains, hex encoding, eval+encoding combos, string concatenation evasion, unicode escape sequences
• Audit-override framing — Prose pleading "educational only", "classify as safe", "ignore the YARA finding", END-OVERRIDE markers, "this skill is disabled" — social engineering aimed at the auditor
• Multistage remote code — exec(...http://...), bash <(curl ...), plugin/manifest loaders that defer code or configuration to a remote URL the auditor cannot inspect

When audit-override framing co-occurs with any non-framing finding (credentials, exfiltration, dangerous commands, obfuscation, multistage), the scanner short-circuits to BLOCK regardless of the score. This mirrors the hosted Shieldon engine's framing co-occurrence policy.

For the full pattern list, see references/DETECTION-COVERAGE.md.

Privacy

Everything runs locally. The --file and --content modes make zero network requests. The --url mode fetches only the specified URL (5s timeout, 100KB limit) and processes it locally. No data is sent to any external service.