BuilderIO/grafana

Grafana Integration

Connection

• Base URL: $GRAFANA_URL (e.g. https://your-org.grafana.net)
• Auth: Authorization: Bearer $GRAFANA_API_TOKEN (service account token)
• Env vars: GRAFANA_API_TOKEN
• Caching: 10-minute cache for metadata; query results are NOT cached (time-sensitive)
• Key datasource: Prometheus UID grafanacloud-prom

Server Lib & API Routes

• File: server/lib/grafana.ts

Exported Functions

| Function | Description | | --------------------------------------------- | --------------------------------------- | | listDashboards(query?) | Search dashboards by query | | getDashboard(uid) | Full dashboard JSON with panels | | getDatasources() | List all datasources | | getAlertRules() | All alert rules (flattened from groups) | | getAlertInstances() | Currently firing alert instances | | queryDatasource(uid, queries[], from?, to?) | Proxy to Grafana's /api/ds/query |

API Routes

| Route | Description | | ------------------------------------ | ----------------------------------------- | | GET /api/grafana/dashboards | Search dashboards | | GET /api/grafana/dashboard?uid=... | Full dashboard JSON | | GET /api/grafana/datasources | List datasources | | GET /api/grafana/alerts | Alert rules and firing instances | | POST /api/grafana/query | Query datasource (Prometheus, Loki, etc.) |

Dashboard

• /adhoc/engineering — Engineering dashboard (mirrors a Grafana dashboard by UID)

Grafana Response Frame Format

{ results: { [refId]: { frames: [{ schema: { fields }, data: { values } }] } } }

• values[0] = timestamps (epoch ms), values[1+] = metric values
• Series labels from schema.fields[i].labels (e.g. { model: "claude-3.5" })
• Transform to Recharts: { time: number, [seriesName]: number }[]

Template Variables

• $CodegenMode → default quality-v4
• $AIModel → regex pattern, default .* (all)
• $environment → cloud or cloud-v2, default cloud
• Variables interpolated via string replace before querying

Key Prometheus Metrics

• Codegen: vcpcodegen_completion_total, vcpcodegen_completion_latency_bucket, vcpcodegen_feedback_total, vcpcodegen_error_total
• LLM: llm_completion_cost_total, llm_input_tokens_total, llm_output_tokens_total, llm_latency_bucket, llm_failures_total, llm_completions_total
• Projects: projects_proposed_config_total, projects_status_total, projects_remote_machine_*, projects_start_duration_bucket
• API/Runtime: api_request_total, with_span_duration_bucket, memory_heap_usage_percent_bucket
• Fly.io: fly_endpoint_total, fly_machine_*
• GitHub: builderbot_pr_created_total, builderbot_pr_closed_total

Key Patterns & Gotchas

• queryDatasource constructs body with datasource UID in each query target; timestamps as string ms
• getAlertRules flattens nested rule groups from unified alerting API
• getAlertInstances handles both array and wrapped object response shapes
• POST helper caches by default (fine for idempotent endpoints) but queryDatasource skips cache
• Heatmap panels render as multi-series line/area (Recharts has no native heatmap; PromQL histogram_quantile() computes quantiles)

Incident Investigation Pattern

For production issues, query Grafana/Prometheus FIRST:

1. LLM latency by model (llm_latency_bucket)
2. Request rates (api_request_total)
3. Error rates (llm_failures_total)
4. Instance counts (via Cloud Monitoring)

Then check Sentry for application errors, Cloud Logging for raw logs.