mongo-ai/posthog-feature-flag-diagnosis

Feature flag / experiment diagnosis workflow

Before you diagnose: fetch live docs

1. Run docs-search("feature flags troubleshooting {specific symptom}")
2. If SDK-specific: run Context7 query-docs for that SDK's flag API reference
3. If potential known bug: run gh search issues on the relevant SDK repo
4. Use the fetched docs as your primary reference — NOT the steps below

The steps below are a diagnostic FRAMEWORK (what questions to ask, in what order). The docs responses are the CONTENT (current answers, version requirements, config details).

Goal

Determine whether the problem is caused by:

• disabled / misordered rollout conditions
• user not matching conditions
• identify timing
• stale person/group properties
• exposure deduplication behavior
• flag loading latency / bootstrap gap
• server-side remote evaluation latency
• recent changes to flag config
• known product bug

Step 1: Inspect the flag definition

Use the flag key to inspect:

• enabled/disabled state
• rollout conditions
• variants
• percentage rollout
• dependencies / targeting logic

Step 2: Inspect evaluation evidence

Query recent $feature_flag_called events for the user.

SELECT properties.$feature_flag,
       properties.$feature_flag_response,
       timestamp
FROM events
WHERE event = '$feature_flag_called'
  AND distinct_id = '{distinct_id}'
ORDER BY timestamp DESC
LIMIT 50;

Interpret:

• false -> evaluation completed but user didn't match
• empty/missing -> flag likely failed to load or fetch

Step 3: Check identify timing

For frontend flows:

• evaluate whether flags are checked before identify
• evaluate whether anonymous vs identified state could explain different values

Fetch SDK-specific identify/flag timing guidance from docs if needed: docs-search("{sdk} feature flags identify timing")

Step 4: Check property ingestion race conditions

If the flag depends on properties set immediately before evaluation:

• run docs-search("feature flags property race condition")
• extract current recommended approach for setting properties before flag evaluation
• determine whether they are hitting an ingestion timing race

Step 5: Check flicker / flash-of-original-content

If the complaint is flicker:

• run docs-search("feature flags flicker bootstrap")
• extract current bootstrap / wait-for-flags approaches
• determine whether flags are fetched asynchronously

Step 6: Check server-side latency path

If complaint is slow server-side flag checks:

• run docs-search("feature flags local evaluation server-side")
• extract current local evaluation guidance
• determine whether remote per-request evaluation is the cause

Step 7: Experiment-specific checks

If this is an experiment:

• inspect exposure behavior
• run docs-search("experiment exposure deduplication") for current dedup behavior
• inspect attribution assumptions
• determine whether event/flag data alignment is the real issue

Step 8: Check recent changes

Use activity logs to correlate config changes with issue start time.

Step 9: Search known issues

Search GitHub with at least 2 variants:

• gh search issues --repo PostHog/{sdk-repo} "feature flag {symptom}"
• gh search issues --repo PostHog/{sdk-repo} "experiment {symptom}"