admin-baked/agent-evaluation
.agent/skills/skills/agent-evaluation/SKILL.md
Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent.
$ install --global
skillsauthnpx skillsauth add admin-baked/bakedbot-for-brands agent-evaluationInstall this skill globally with one command. Works with Claude Code, Cursor, and Windsurf.
Security Scan Results
3 of 9 scanners reported clean
Some scanners were skipped, did not run, or reported a non-clean status. Review each row below.
3
Scanners Passed
9
Scanners in report
Clean
TrivyContainer and dependency vulnerability scanner
95%
Clean
SemgrepStatic code analysis for vulnerabilities
95%
Clean
mcp-scan (Snyk)Model Context Protocol security validation
95%
Skipped
Snyk (dep)Open source security scanning
50%
Skipped
Socket.devSupply chain security analysis
50%
Skipped
VirusTotalMulti-engine malware detection
50%
Skipped
CrowdStrikeAdvanced threat intelligence
50%
Skipped
OSV-ScannerOpen Source Vulnerability database check
50%
Skipped
OWASP Dep-Check
50%
Last scanned: Mar 29, 2026, 12:26 PM36.9s1 file scanned
SKILL.md
- name:
- agent-evaluation
- description:
- Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent.
- source:
- vibeship-spawner-skills (Apache 2.0)
Agent Evaluation
You're a quality engineer who has seen agents that aced benchmarks fail spectacularly in production. You've learned that evaluating LLM agents is fundamentally different from testing traditional software—the same input can produce different outputs, and "correct" often has no single answer.
You've built evaluation frameworks that catch issues before production: behavioral regression tests, capability assessments, and reliability metrics. You understand that the goal isn't 100% test pass rate—it
Capabilities
- agent-testing
- benchmark-design
- capability-assessment
- reliability-metrics
- regression-testing
Requirements
- testing-fundamentals
- llm-fundamentals
Patterns
Statistical Test Evaluation
Run tests multiple times and analyze result distributions
Behavioral Contract Testing
Define and test agent behavioral invariants
Adversarial Testing
Actively try to break agent behavior
Anti-Patterns
❌ Single-Run Testing
❌ Only Happy Path Tests
❌ Output String Matching
⚠️ Sharp Edges
| Issue | Severity | Solution | |-------|----------|----------| | Agent scores well on benchmarks but fails in production | high | // Bridge benchmark and production evaluation | | Same test passes sometimes, fails other times | high | // Handle flaky tests in LLM agent evaluation | | Agent optimized for metric, not actual task | medium | // Multi-dimensional evaluation to prevent gaming | | Test data accidentally used in training or prompts | critical | // Prevent data leakage in agent evaluation |
Related Skills
Works well with: multi-agent-orchestration, agent-communication, autonomous-agents
Related Skills
admin-baked/skills/shared/executive-brief
testing
VerifiedTrustedCommunity
--- name: executive-brief description: Produce a concise executive brief or portfolio digest for a super user or operator — use when summarizing multi-account performance, cross-org anomalies, top actions needed, or weekly business status for leadership review. Trigger phrases: "executive summary", "weekly brief", "portfolio digest", "top actions this week", "what needs my attention", "board update", "cross-account summary". version: 0.1.0 owner: platform agent_owner: pops allowed_roles: - sup
2SKILL.mdUpdated Apr 1, 2026
admin-baked/skills/shared/anomaly-to-action-memo
development
VerifiedTrustedCommunity
--- name: anomaly-to-action-memo description: Interpret a detected anomaly or signal and produce a decision-ready action memo — use when an alert, metric deviation, or operational signal needs to be turned into a prioritized recommendation with evidence, owner, and next step. Trigger phrases: "what does this anomaly mean", "something looks off", "explain this alert", "revenue is down", "traffic dropped", "flag this for review", "what should we do about this". version: 0.1.0 owner: ops-intelligen
2SKILL.mdUpdated Apr 1, 2026
admin-baked/skills/org/brand-voice
testing
VerifiedTrustedCommunity
--- name: brand-voice description: Apply BakedBot brand voice standards to any customer-facing content — use when generating or reviewing copy that must match a dispensary or brand's approved tone, language patterns, and messaging constraints. Trigger phrases: "does this match our voice", "write in our brand voice", "on-brand copy", "brand guidelines", "tone check". version: 0.1.0 owner: platform agent_owner: craig allowed_roles: - super_user - dispensary_operator - brand_operator outputs:
2SKILL.mdUpdated Apr 1, 2026
admin-baked/skills/grower/sell-through-partner-analysis
testing
VerifiedTrustedCommunity
--- name: sell-through-partner-analysis description: Analyze which retail dispensary partners are selling through a grower's products effectively, identify top performers and laggards, and produce a prioritized partner action plan. Use when a grower wants to know where their products move fastest, which partners need attention, and where to focus wholesale sales effort. Trigger phrases: "which partners are selling our product", "sell-through analysis", "partner performance", "where is inventory
2SKILL.mdUpdated Apr 1, 2026