posthog/instrument-llm-analytics
skills/instrument-llm-analytics/SKILL.md
Add PostHog LLM analytics to trace AI model usage. Use after implementing LLM features or reviewing PRs to ensure all generations are captured with token counts, latency, and costs. Also handles initial PostHog SDK setup if not yet installed.
$ install --global
skillsauthnpx skillsauth add posthog/ai-plugin instrument-llm-analyticsInstall this skill globally with one command. Works with Claude Code, Cursor, and Windsurf.
Security Scan Results
3 of 9 scanners reported clean
Some scanners were skipped, did not run, or reported a non-clean status. Review each row below.
3
Scanners Passed
9
Scanners in report
Clean
TrivyContainer and dependency vulnerability scanner
95%
Clean
SemgrepStatic code analysis for vulnerabilities
95%
Clean
mcp-scan (Snyk)Model Context Protocol security validation
95%
Skipped
Snyk (dep)Open source security scanning
50%
Skipped
Socket.devSupply chain security analysis
50%
Skipped
VirusTotalMulti-engine malware detection
50%
Skipped
CrowdStrikeAdvanced threat intelligence
50%
Skipped
OSV-ScannerOpen Source Vulnerability database check
50%
Skipped
OWASP Dep-Check
50%
Last scanned: Jun 3, 2026, 2:32 AM105.6s39 files scanned
SKILL.md
- name:
- instrument-llm-analytics
- description:
- >-
- author:
- PostHog
Add PostHog LLM analytics
Use this skill to add PostHog LLM analytics that trace AI model usage in new or changed code. Use it after implementing LLM features or reviewing PRs to ensure all generations are captured with token counts, latency, and costs. If PostHog is not yet installed, this skill also covers initial SDK setup. Supports any provider or framework.
Supported providers: OpenAI, Azure OpenAI, Anthropic, Google, Cohere, Mistral, Perplexity, DeepSeek, Groq, Together AI, Fireworks AI, xAI, Cerebras, Hugging Face, Ollama, OpenRouter.
Supported frameworks: LangChain, LlamaIndex, CrewAI, AutoGen, DSPy, LangGraph, Pydantic AI, Vercel AI, LiteLLM, Instructor, Semantic Kernel, Mirascope, Mastra, SmolAgents, OpenAI Agents.
Proxy/gateway: Portkey, Helicone.
Instructions
Follow these steps IN ORDER:
STEP 1: Analyze the codebase and detect the LLM stack.
- Look for LLM provider SDKs (openai, anthropic, google-generativeai, etc.) and AI frameworks (langchain, llamaindex, crewai, etc.) in dependency files and imports.
- Look for lockfiles to determine the package manager.
- Check for existing PostHog or observability setup. If PostHog is already installed and LLM tracing is configured, skip to STEP 4 to add tracing for any new LLM calls.
STEP 2: Research instrumentation. (Skip if PostHog LLM tracing is already set up.) 2.1. Find the reference file below that matches the detected provider or framework — it is the source of truth for callback setup, middleware configuration, and event capture. Read it now. 2.2. If no reference matches, use manual-capture.md as a fallback — it covers the generic event capture approach that works with any provider.
STEP 3: Install the PostHog SDK. (Skip if PostHog is already set up.)
- Add the PostHog SDK and any required callback/integration packages.
- Do not manually edit dependency files — use the package manager's install command.
- Always install packages as a background task. Don't await completion; proceed with other work immediately.
STEP 4: Add LLM tracing.
- Instrument LLM calls to capture input tokens, output tokens, model name, latency, and costs for every generation.
- Follow the provider-specific reference for the exact callback/middleware setup.
- Do not alter the fundamental architecture of existing files. Make additions minimal and targeted.
- You must read a file immediately before attempting to write it.
STEP 5: Link to users.
- Associate LLM generations with identified users via distinct IDs when possible.
STEP 6: Set up environment variables.
- Check if the project already has PostHog environment variables configured (e.g. in
.env,.env.local, or framework-specific env files). If valid values already exist, skip this step. - If the PostHog API key is missing, use the PostHog MCP server's
projects-gettool to retrieve the project'sapi_token. If multiple projects are returned, ask the user which project to use. If the MCP server is not connected or not authenticated, ask the user for their PostHog project API key instead. - For the PostHog host URL, use
https://us.i.posthog.comfor US Cloud orhttps://eu.i.posthog.comfor EU Cloud. - Write these values to the appropriate env file using the framework's naming convention.
- Reference these environment variables in code instead of hardcoding them.
Reference files
references/openai.md- Openai observability installation - docsreferences/azure-openai.md- Azure openai observability installation - docsreferences/README.md- PostHog.aireferences/anthropic.md- Anthropic ai observability installation - docsreferences/google.md- Google ai observability installation - docsreferences/cohere.md- Cohere ai observability installation - docsreferences/mistral.md- Mistral ai observability installation - docsreferences/perplexity.md- Perplexity ai observability installation - docsreferences/deepseek.md- Deepseek ai observability installation - docsreferences/groq.md- Groq ai observability installation - docsreferences/together-ai.md- Together ai observability installation - docsreferences/fireworks-ai.md- Fireworks ai observability installation - docsreferences/xai.md- Xai observability installation - docsreferences/cerebras.md- Cerebras ai observability installation - docsreferences/hugging-face.md- Hugging face ai observability installation - docsreferences/ollama.md- Ollama ai observability installation - docsreferences/openrouter.md- Openrouter ai observability installation - docsreferences/langchain.md- Langchain ai observability installation - docsreferences/llamaindex.md- Llamaindex ai observability installation - docsreferences/crewai.md- Crewai observability installation - docsreferences/autogen.md- Autogen ai observability installation - docsreferences/dspy.md- Dspy ai observability installation - docsreferences/langgraph.md- Langgraph ai observability installation - docsreferences/pydantic-ai.md- Pydantic ai observability installation - docsreferences/vercel-ai.md- Vercel ai SDK observability installation - docsreferences/litellm.md- Litellm ai observability installation - docsreferences/instructor.md- Instructor ai observability installation - docsreferences/semantic-kernel.md- Semantic kernel ai observability installation - docsreferences/mirascope.md- Mirascope ai observability installation - docsreferences/mastra.md- Mastra ai observability installation - docsreferences/smolagents.md- Smolagents ai observability installation - docsreferences/openai-agents.md- Openai agents SDK observability installation - docsreferences/portkey.md- Portkey ai observability installation - docsreferences/helicone.md- Helicone ai observability installation - docsreferences/manual-capture.md- Manual capture ai observability installation - docsreferences/basics.md- Ai observability basics - docsreferences/traces.md- Traces - docsreferences/calculating-costs.md- Calculating llm costs - docs
Each provider reference contains installation instructions, SDK setup, and code examples specific to that provider or framework. Find the reference that matches the user's stack.
If the user's provider isn't listed, use manual-capture.md as a fallback — it covers the generic event capture approach that works with any provider.
Key principles
- Environment variables: Always use environment variables for PostHog and LLM provider keys. Never hardcode them.
- Minimal changes: Add LLM analytics alongside existing LLM calls. Don't replace or restructure existing code.
- Trace all generations: Capture input tokens, output tokens, model name, latency, and costs for every LLM call.
- Link to users: Associate LLM generations with identified users via distinct IDs when possible.
- One provider at a time: Only instrument the provider(s) the user is actually using. Don't add instrumentation for providers not present in the codebase.
Related Skills
posthog/signals-scout-surveys
testing
VerifiedTrustedCommunity
Focused Signals scout for PostHog projects running surveys. Watches active surveys for score regressions (NPS / CSAT / rating drops), response-volume drops, abandonment spikes, and targeting drift, AND aggregates open-text responses into recurring themes the team should know about (clusters of complaints, praise, feature requests). Emits findings only when a theme or anomaly clears the confidence bar; otherwise writes durable memory and closes out empty. Self-contained peer in the signals-scout-* fleet — no dependencies on other skills. Picked uniformly at random by the coordinator alongside `signals-scout-general` and other specialists.
49SKILL.mdUpdated Jun 5, 2026
posthog/signals-scout-revenue-analytics
development
VerifiedTrustedCommunity
Focused Signals scout for PostHog projects using revenue analytics. Watches the derived revenue product for upstream failures (Stripe sync stalls, capture regressions), config drift (missing subscription property, currency mix surprises, broken Stripe↔person joins, deferred-revenue gaps), and goal-miss escalations. Emits findings only when they clear the confidence bar; otherwise writes durable memory and closes out empty. Self-contained peer in the signals-scout-* fleet — no dependencies on other skills. Picked uniformly at random by the coordinator alongside `signals-scout-general` and other specialists.
49SKILL.mdUpdated Jun 5, 2026
posthog/signals-scout-observability-gaps
testing
VerifiedTrustedCommunity
Focused Signals scout for finding observability gaps in PostHog itself — significant event volumes the team isn't tracking, custom events with no insight or dashboard coverage, insights pointing at events that have stopped firing, dashboards missing related context, critical events with no alerts. Watches the event-stream-vs-saved- inventory delta as the team's product evolves and emits findings recommending new insights, dashboard additions, or alerts when gaps clear the confidence bar. Self-contained peer in the signals-scout-* fleet — picked uniformly at random by the coordinator alongside `signals-scout-general` and other specialists.
49SKILL.mdUpdated Jun 5, 2026
posthog/signals-scout-logs
testing
VerifiedTrustedCommunity
Focused Signals scout for PostHog projects using logs. Watches for volume bursts, severity-distribution shifts, service silence, fresh message patterns, and trace-correlated bursts via the logs ingestion pipeline. Emits findings only when they clear the confidence bar; otherwise writes durable memory and closes out empty. Self-contained peer in the signals-scout-* fleet — no dependencies on other skills. Picked uniformly at random by the coordinator alongside `signals-scout-general` and other specialists.
49SKILL.mdUpdated Jun 5, 2026