googlecloudplatform/gcp-agent-eval-engine-runner
ai-ml/dev-signal/.agent/skills/gcp-agent-eval-engine-runner/SKILL.md
Boilerplate for a production evaluation runner that performs parallel inference, captures reasoning traces via SSE, and integrates with the Vertex AI Gen AI Evaluation service.
$ install --global
skillsauthnpx skillsauth add googlecloudplatform/devrel-demos gcp-agent-eval-engine-runnerInstall this skill globally with one command. Works with Claude Code, Cursor, and Windsurf.
Security Scan Results
3 of 9 scanners reported clean
Some scanners were skipped, did not run, or reported a non-clean status. Review each row below.
3
Scanners Passed
9
Scanners in report
Clean
TrivyContainer and dependency vulnerability scanner
95%
Clean
SemgrepStatic code analysis for vulnerabilities
95%
Clean
mcp-scan (Snyk)Model Context Protocol security validation
95%
Skipped
Snyk (dep)Open source security scanning
50%
Skipped
Socket.devSupply chain security analysis
50%
Skipped
VirusTotalMulti-engine malware detection
50%
Skipped
CrowdStrikeAdvanced threat intelligence
50%
Skipped
OSV-ScannerOpen Source Vulnerability database check
50%
Skipped
OWASP Dep-Check
50%
Last scanned: Apr 5, 2026, 1:59 PM6.8s2 files scanned
SKILL.md
- name:
- gcp-agent-eval-engine-runner
- description:
- Boilerplate for a production evaluation runner that performs parallel inference, captures reasoning traces via SSE, and integrates with the Vertex AI Gen AI Evaluation service.
gcp-agent-eval-engine-runner
This skill provides the "engine" for your automated evaluation pipeline. Grounded in evaluation_blog.md, it handles the complexity of running hundreds of parallel requests against a shadow revision while capturing the full "Thinking Process" (Reasoning Trace).
Usage
Ask Antigravity to:
- "Create an evaluation runner script for my agent"
- "Implement parallel inference for my golden dataset"
- "Capture SSE traces for tool trajectory evaluation"
Engine Pattern
- Parallel Inference: Uses
asyncio.Semaphoreto throttle requests (preventing DDOS of the shadow service). - SSE Capture: Connects to the ADK
POST /run_sseendpoint to stream intermediate events. - Dataset Enrichment: Appends
responseandintermediate_eventsto the input dataset. - Vertex AI Integration: Submits the enriched dataset to the
create_evaluation_runAPI.
Python Boilerplate
Refer to scripts/evaluate_agent_boilerplate.py for the core implementation.
Related Skills
googlecloudplatform/gcp-agent-sdp-template-factory
devops
VerifiedTrustedCommunity
Standardizes the creation of Sensitive Data Protection (DLP) templates for PII and credential redaction.
260SKILL.mdUpdated Apr 5, 2026
googlecloudplatform/gcp-agent-safety-gatekeeper
development
VerifiedTrustedCommunity
Implements the "Defense-in-Depth" integration pattern in Python (intercepting prompts, parsing filter results).
260SKILL.mdUpdated Apr 5, 2026
googlecloudplatform/gcp-agent-model-armor-shield
testing
VerifiedTrustedCommunity
Configures Model Armor security policies (Prompt Injection, Jailbreak, RAI filters).
260SKILL.mdUpdated Apr 5, 2026
googlecloudplatform/gcp-agent-golden-dataset-builder
tools
VerifiedTrustedCommunity
Assists developers in collecting and structuring a library of diverse examples ("Golden Dataset") required for data-driven evaluation, including tool trajectories.
260SKILL.mdUpdated Apr 5, 2026