beam-ai-team/langfuse-connect

Langfuse Connect

User-facing entry point for Langfuse integration. Routes to consolidated operation skills.

Safety Contract

Before creating, updating, deleting, ingesting, scoring, or exporting Langfuse data, show the host/project, routed operation, target IDs or filters, payload summary, output path when used, and expected side effects. Require explicit user approval in the current turn before mutating Langfuse or writing files. Read-only trace, prompt, dataset, queue, model, score, status, and config checks with an approved scope do not require additional approval.

Context reference

For score config IDs and API patterns, see Patterns & score config reference below. For shared client, references, and error handling, load langfuse-master/.

---

Pre-Flight Check (ALWAYS FIRST)

Before any operation, run config check:

uv run python skills/integrations/langfuse/langfuse-master/scripts/check_langfuse_config.py --json

If ai_action is:

• proceed_with_operation → Continue with requested operation
• prompt_for_api_key → Ask user for credentials, guide to setup

---

Routing Table

All Langfuse operations are consolidated into 11 skills. Each uses --resource and --action arguments for dispatch.

Traces, Observations & Sessions → langfuse-traces

| User Says | Resource | Action | |-----------|----------|--------| | "list traces", "show traces" | traces | list | | "get trace {id}", "trace details" | traces | get | | "delete trace", "bulk delete traces" | traces | delete | | "list observations", "show spans" | observations | list | | "get observation {id}" | observations | get | | "list sessions", "show sessions" | sessions | list | | "get session {id}" | sessions | get |

Datasets, Items, Runs → langfuse-datasets

| User Says | Resource | Action | |-----------|----------|--------| | "list datasets", "show datasets" | datasets | list | | "create dataset", "new dataset" | datasets | create | | "get dataset {name}" | datasets | get | | "list dataset items", "show items" | items | list | | "create dataset item", "add item" | items | create | | "get dataset item {id}" | items | get | | "delete dataset item" | items | delete | | "list runs", "show runs" | runs | list | | "get run" | runs | get | | "delete run" | runs | delete | | "list run items", "show results" | run-items | list | | "create run item", "log evaluation" | run-items | create |

Scores & Score Configs → langfuse-scores

| User Says | Resource | Action | |-----------|----------|--------| | "list scores", "show evaluations" | scores | list | | "get score {id}" | scores | get | | "create score", "add score" | scores | create | | "delete score", "remove score" | scores | delete | | "list score configs", "show config" | configs | list | | "get score config {id}" | configs | get | | "create score config", "new config" | configs | create | | "update score config", "archive config" | configs | update |

Annotation Queues → langfuse-queues

| User Says | Resource | Action | |-----------|----------|--------| | "list queues", "annotation queues" | queues | list | | "create queue", "new queue" | queues | create | | "get queue {id}" | queues | get | | "list queue items" | items | list | | "add to queue" | items | create | | "get queue item" | items | get | | "update queue item", "annotate" | items | update | | "remove from queue" | items | delete | | "assign reviewer" | assignments | create | | "unassign reviewer" | assignments | delete |

Prompts → langfuse-prompts

| User Says | Action | |-----------|--------| | "list prompts", "show prompts" | list | | "get prompt {name}" | get | | "create prompt", "new prompt" | create | | "delete prompt" | delete | | "update prompt version", "set labels" | update-version |

Models → langfuse-models

| User Says | Action | |-----------|--------| | "list models", "model costs" | list | | "get model {id}" | get | | "create model" | create | | "delete model" | delete |

Projects & Org Admin → langfuse-admin

| User Says | Resource | Action | |-----------|----------|--------| | "current project", "get project" | projects | get | | "create project" | projects | create | | "update project", "rename project" | projects | update | | "delete project" | projects | delete | | "list api keys" | api-keys | list | | "create api key" | api-keys | create | | "delete api key" | api-keys | delete | | "list members", "org members" | org | list-memberships | | "change role", "update member" | org | update-membership | | "remove member" | org | delete-membership | | "list org projects" | org | list-projects | | "org api keys" | org | list-api-keys |

Ingestion, Comments & Media → langfuse-ingestion

| User Says | Resource | Action | |-----------|----------|--------| | "batch ingest", "bulk import" | batch | ingest | | "otel ingest", "opentelemetry" | otel | ingest | | "list comments" | comments | list | | "get comment" | comments | get | | "create comment", "add comment" | comments | create | | "get media" | media | get | | "update media" | media | update | | "upload url", "media upload" | media | upload-url |

Health & Metrics → langfuse-status

| User Says | Action | |-----------|--------| | "health check", "is langfuse up" | health | | "metrics", "usage stats" | metrics |

---

Quick Start

# Check config
uv run python skills/integrations/langfuse/langfuse-master/scripts/check_langfuse_config.py --test

# List recent traces
uv run python scripts/traces.py --resource traces --action list --limit 10

# Get specific trace
uv run python scripts/traces.py --resource traces --action get --id abc123

# Create a score
uv run python scripts/scores.py --resource scores --action create --trace {id} --name tool_efficiency --value 0.85

---

Patterns & score config reference

Critical patterns

Observations require individual fetch

# GET /sessions/{id} does NOT include observations
# Must call GET /traces/{id} for each trace
for trace in session["traces"]:
    full = client.get(f"/traces/{trace['id']}")
    obs = full.get("observations", [])

CATEGORICAL scores use string value

# CORRECT
{"value": "archive", "configId": "..."}

# WRONG (400 error)
{"value": 2, "stringValue": "archive"}

Score config IDs

CONFIG_IDS = {
    # Quality Dimensions (NUMERIC 0-1 unless noted)
    "goal_achievement": "68cfd90c-8c9e-4907-808d-869ccd9a4c07",      # CATEGORICAL
    "tool_efficiency": "84965473-0f54-4248-999e-7b8627fc9c29",
    "process_adherence": "651fc213-4750-4d4e-8155-270235c7cad8",
    "context_efficiency": "ae22abed-bd4a-4926-af74-8d71edb1925d",
    "error_handling": "96c290b7-e3a6-4caa-bace-93cf55f70f1c",        # CATEGORICAL
    "output_quality": "d33b1fbf-d3c6-458c-90ca-0b515fe09aed",
    "overall_quality": "793f09d9-0053-4310-ad32-00dc06c69a71",
    # Meta Scores
    "root_cause_issues": "669bead7-1936-4fc4-bae8-e7814c9eab04",     # CATEGORICAL
    "session_improvements": "2e87193b-c853-4955-b2f0-9fa572531681",  # CATEGORICAL
    "session_notes": "67640329-0c03-4be6-bc9f-49765a0462b5",         # NUMERIC (value=1 + comment/metadata)
}

CATEGORICAL labels

| Score | Labels | |-------|--------| | goal_achievement | failed, partial, complete, exceeded | | error_handling | poor, struggled, recovered, prevented | | root_cause_issues | none, tool_misuse, process_violation, context_waste, error_cascade, output_quality, multiple | | session_improvements | none, minor, moderate, significant, critical |

---

Error Handling

On error, load: langfuse-master/references/error-handling.md

Common issues:

• 401: Check API keys
• 404: Resource not found
• 429: Rate limited, wait and retry

---

References

• Master skill: langfuse-master/
• Setup guide: langfuse-master/references/setup-guide.md
• API reference: langfuse-master/references/api-reference.md