foryourhealth111-pixel/evals-context
bundled/skills/evals-context/SKILL.md
Provides context about the Roo Code evals system structure in this monorepo. Use when tasks mention "evals", "evaluation", "eval runs", "eval exercises", or working with the evals infrastructure. Helps distinguish between the evals execution system (packages/evals, apps/web-evals) and the public website evals display page (apps/web-roo-code/src/app/evals).
$ install --global
skillsauthnpx skillsauth add foryourhealth111-pixel/vco-skills-codex evals-contextInstall this skill globally with one command. Works with Claude Code, Cursor, and Windsurf.
Security Scan Results
3 of 9 scanners reported clean
Some scanners were skipped, did not run, or reported a non-clean status. Review each row below.
3
Scanners Passed
9
Scanners in report
Clean
TrivyContainer and dependency vulnerability scanner
95%
Clean
SemgrepStatic code analysis for vulnerabilities
95%
Clean
mcp-scan (Snyk)Model Context Protocol security validation
95%
Skipped
Snyk (dep)Open source security scanning
50%
Skipped
Socket.devSupply chain security analysis
50%
Skipped
VirusTotalMulti-engine malware detection
50%
Skipped
CrowdStrikeAdvanced threat intelligence
50%
Skipped
OSV-ScannerOpen Source Vulnerability database check
50%
Skipped
OWASP Dep-Check
50%
Last scanned: Apr 4, 2026, 11:17 PM30.2s1 file scanned
SKILL.md
- name:
- evals-context
- description:
- Provides context about the Roo Code evals system structure in this monorepo. Use when tasks mention "evals", "evaluation", "eval runs", "eval exercises", or working with the evals infrastructure. Helps distinguish between the evals execution system (packages/evals, apps/web-evals) and the public website evals display page (apps/web-roo-code/src/app/evals).
Evals Codebase Context
When to Use This Skill
Use this skill when the task involves:
- Modifying or debugging the evals execution infrastructure
- Adding new eval exercises or languages
- Working with the evals web interface (apps/web-evals)
- Modifying the public evals display page on roocode.com
- Understanding where evals code lives in this monorepo
When NOT to Use This Skill
Do NOT use this skill when:
- Working on unrelated parts of the codebase (extension, webview-ui, etc.)
- The task is purely about the VS Code extension's core functionality
- Working on the main website pages that don't involve evals
Key Disambiguation: Two "Evals" Locations
This monorepo has two distinct evals-related locations that can cause confusion:
| Component | Path | Purpose |
| --------------------------- | -------------------------------------------------------------- | -------------------------------------------------------------- |
| Evals Execution System | packages/evals/ | Core eval infrastructure: CLI, DB schema, Docker configs |
| Evals Management UI | apps/web-evals/ | Next.js app for creating/monitoring eval runs (localhost:3446) |
| Website Evals Page | apps/web-roo-code/src/app/evals/ | Public roocode.com page displaying eval results |
| External Exercises Repo | Roo-Code-Evals | Actual coding exercises (NOT in this monorepo) |
Directory Structure Reference
packages/evals/ - Core Evals Package
packages/evals/
├── ARCHITECTURE.md # Detailed architecture documentation
├── ADDING-EVALS.md # Guide for adding new exercises/languages
├── README.md # Setup and running instructions
├── docker-compose.yml # Container orchestration
├── Dockerfile.runner # Runner container definition
├── Dockerfile.web # Web app container
├── drizzle.config.ts # Database ORM config
├── src/
│ ├── index.ts # Package exports
│ ├── cli/ # CLI commands for running evals
│ │ ├── runEvals.ts # Orchestrates complete eval runs
│ │ ├── runTask.ts # Executes individual tasks in containers
│ │ ├── runUnitTest.ts # Validates task completion via tests
│ │ └── redis.ts # Redis pub/sub integration
│ ├── db/
│ │ ├── schema.ts # Database schema (runs, tasks)
│ │ ├── queries/ # Database query functions
│ │ └── migrations/ # SQL migrations
│ └── exercises/
│ └── index.ts # Exercise loading utilities
└── scripts/
└── setup.sh # Local macOS setup script
apps/web-evals/ - Evals Management Web App
apps/web-evals/
├── src/
│ ├── app/
│ │ ├── page.tsx # Home page (runs list)
│ │ ├── runs/
│ │ │ ├── new/ # Create new eval run
│ │ │ └── [id]/ # View specific run status
│ │ └── api/runs/ # SSE streaming endpoint
│ ├── actions/ # Server actions
│ │ ├── runs.ts # Run CRUD operations
│ │ ├── tasks.ts # Task queries
│ │ ├── exercises.ts # Exercise listing
│ │ └── heartbeat.ts # Controller health checks
│ ├── hooks/ # React hooks (SSE, models, etc.)
│ └── lib/ # Utilities and schemas
apps/web-roo-code/src/app/evals/ - Public Website Evals Page
apps/web-roo-code/src/app/evals/
├── page.tsx # Fetches and displays public eval results
├── evals.tsx # Main evals display component
├── plot.tsx # Visualization component
└── types.ts # EvalRun type (extends packages/evals types)
This page displays eval results on the public roocode.com website. It imports types from @roo-code/evals but does NOT run evals.
Architecture Overview
The evals system is a distributed evaluation platform that runs AI coding tasks in isolated VS Code environments:
┌─────────────────────────────────────────────────────────────┐
│ Web App (apps/web-evals) ──────────────────────────────── │
│ │ │
│ ▼ │
│ PostgreSQL ◄────► Controller Container │
│ │ │ │
│ ▼ ▼ │
│ Redis ◄───► Runner Containers (1-25 parallel) │
└─────────────────────────────────────────────────────────────┘
Key components:
- Controller: Orchestrates eval runs, spawns runners, manages task queue (p-queue)
- Runner: Isolated Docker container with VS Code + Roo Code extension + language runtimes
- Redis: Pub/sub for real-time events (NOT task queuing)
- PostgreSQL: Stores runs, tasks, metrics
Common Tasks Quick Reference
Adding a New Eval Exercise
- Add exercise to Roo-Code-Evals repo (external)
- See
packages/evals/ADDING-EVALS.mdfor structure
Modifying Eval CLI Behavior
Edit files in packages/evals/src/cli/:
runEvals.ts- Run orchestrationrunTask.ts- Task executionrunUnitTest.ts- Test validation
Modifying the Evals Web Interface
Edit files in apps/web-evals/src/:
app/runs/new/new-run.tsx- New run formactions/runs.ts- Run server actions
Modifying the Public Evals Display Page
Edit files in apps/web-roo-code/src/app/evals/:
evals.tsx- Display componentplot.tsx- Charts
Database Schema Changes
- Edit
packages/evals/src/db/schema.ts - Generate migration:
cd packages/evals && pnpm drizzle-kit generate - Apply migration:
pnpm drizzle-kit migrate
Running Evals Locally
# From repo root
pnpm evals
# Opens web UI at http://localhost:3446
Ports (defaults):
- PostgreSQL: 5433
- Redis: 6380
- Web: 3446
Testing
# packages/evals tests
cd packages/evals && npx vitest run
# apps/web-evals tests
cd apps/web-evals && npx vitest run
Key Types/Exports from @roo-code/evals
The package exports are defined in packages/evals/src/index.ts:
- Database queries:
getRuns,getTasks,getTaskMetrics, etc. - Schema types:
Run,Task,TaskMetrics - Used by both
apps/web-evalsandapps/web-roo-code
Related Skills
foryourhealth111-pixel/shap
development
VerifiedTrustedCommunity
Model interpretability and explainability using SHAP (SHapley Additive exPlanations). Use this skill when explaining machine learning model predictions, computing feature importance, generating SHAP plots (waterfall, beeswarm, bar, scatter, force, heatmap), debugging models, analyzing model bias or fairness, comparing models, or implementing explainable AI. Works with tree-based models (XGBoost, LightGBM, Random Forest), deep learning (TensorFlow, PyTorch), linear models, and any black-box model.
1,760SKILL.mdUpdated Apr 27, 2026
foryourhealth111-pixel/sentry
development
VerifiedTrustedCommunity
Use when the user asks to inspect Sentry issues or events, summarize recent production errors, or pull basic Sentry health data via the Sentry API; perform read-only queries with the bundled script and require `SENTRY_AUTH_TOKEN`.
1,760SKILL.mdUpdated Apr 27, 2026
foryourhealth111-pixel/senior-prompt-engineer
development
VerifiedTrustedCommunity
World-class prompt engineering skill for LLM optimization, prompt patterns, structured outputs, and AI product development. Expertise in Claude, GPT-4, prompt design patterns, few-shot learning, chain-of-thought, and AI evaluation. Includes RAG optimization, agent design, and LLM system architecture. Use when building AI products, optimizing LLM performance, designing agentic systems, or implementing advanced prompting techniques.
1,760SKILL.mdUpdated Apr 27, 2026
foryourhealth111-pixel/senior-ml-engineer
development
VerifiedTrustedCommunity
World-class ML engineering skill for productionizing ML models, MLOps, and building scalable ML systems. Expertise in PyTorch, TensorFlow, model deployment, feature stores, model monitoring, and ML infrastructure. Includes LLM integration, fine-tuning, RAG systems, and agentic AI. Use when deploying ML models, building ML platforms, implementing MLOps, or integrating LLMs into production systems.
1,760SKILL.mdUpdated Apr 27, 2026