coalesce-labs/metrics-framework

Metrics Framework: Leading vs Lagging Indicators

When to use: Setting up metrics, designing dashboards, or when you need early signal before outcomes materialize

Framework source: Aakash Gupta's "Become an A/B Testing Expert" and metrics guidance

Quick Start

1. Tell me: "Help me build a metrics framework for [product/feature]"
2. I will check thoughts/shared/pm/frameworks/ for your North Star and thoughts/shared/pm/metrics/ for baseline data
3. I will ask about your North Star (lagging metric), product stage, and decision speed needs
4. We build a metric hierarchy: Lagging (quarterly) -> Leading (weekly) -> Input (daily)
5. For each key metric, we define alert thresholds (green/yellow/red) and a dashboard layout
6. Output goes to thoughts/shared/pm/metrics/metrics-framework-[date].md

Key principle: Leading metrics should PREDICT lagging metrics. If they do not, you are tracking the wrong thing. We always validate the correlation before committing to a leading indicator.

Context Routing Logic (Internal - for Claude)

Automatic Context Checks: When this skill is invoked, immediately check:

| Source | Files/Folders | Search Terms | What to Extract | | ---------------- | ---------------------------------------------- | ------------------------------------ | -------------------------------------- | | Strategy Context | thoughts/shared/pm/frameworks/*.md | North Star, objective, business goal | Company-wide lagging metric target | | Business Model | thoughts/shared/pm/context/business-info-template.md | metrics, KPIs, growth model | Current revenue/retention/growth focus | | Metrics History | thoughts/shared/pm/metrics/*.md | baseline, trends, benchmarks | Historical leading/lagging metric data | | PRDs | thoughts/shared/pm/prds/*.md | success metrics, measurement | Feature-level metric definitions | | Meetings | thoughts/shared/product/meeting-notes/*.md | "metrics", "dashboard", "KPI" | Stakeholder metric priorities |

Context Priority:

1. Business strategy and North Star FIRST
2. Current product metrics and baselines SECOND
3. Historical trend data THIRD
4. Feature-specific metrics FOURTH

Cross-Skill Links:

• If defining North Star → Link to /define-north-star
• If setting up feature success → Link to /feature-metrics for STEDII framework
• If tracking product health → Link to /retention-analysis and /activation-analysis
• If analyzing test results → Link to /feature-results

---

Step 0: Understanding Your Metric System

Before designing your metrics framework, let me understand what you're optimizing for...

Checking:

• thoughts/shared/pm/context/business-info-template.md for business model and goals
• thoughts/shared/pm/frameworks/*.md for strategic priorities
• thoughts/shared/pm/metrics/ for existing metric baselines
• thoughts/shared/pm/prds/ for current feature success metrics

Based on what I find, I'll show you:

Your Current Metric Health

North Star / Lagging Metric:

• [Current North Star metric from strategy]
• [Baseline value: X]
• [Target: Y by when]

Existing Leading Indicators:

• [What you're already tracking]
• [Historical effectiveness: does this predict the lagging metric?]

Metric Gaps:

• [Metrics you're missing that would help you move faster]
• [Decision-making gaps: where you're flying blind]

PM-Specific Diagnosis Questions

1. Decision Speed: How long do you need to make go/no-go decisions? (Days vs weeks vs months)
2. Product Stage: Are you focused on growth, retention, monetization, or operational efficiency?
3. Team Alignment: Do different teams use different success metrics? (Problem!)
4. Data Quality: Do you have reliable tracking for the metrics you care about?
5. Stakeholder Expectations: Does your executive team care about your current metrics?

---

---

Leading vs Lagging Metrics

Lagging Metrics (Outcome Indicators)

Definition: Metrics that measure final outcomes - what already happened

Characteristics:

• Slow to respond
• High certainty (accurate)
• Ultimate measure of success
• Hard to influence directly

Examples:

• Revenue
• Customer lifetime value (LTV)
• Annual retention rate
• Net Promoter Score (NPS)
• Market share
• Profit margins

When to use:

• Board reporting
• Annual/quarterly goals
• Validating long-term bets
• Measuring ultimate success

---

Leading Metrics (Predictive Indicators)

Definition: Metrics that predict future outcomes - early signals

Characteristics:

• Fast to respond
• Lower certainty (probabilistic)
• Actionable in real-time
• Directly influenceable

Examples:

• Day 7 activation rate (predicts retention)
• Weekly active users (predicts revenue)
• Feature adoption rate (predicts stickiness)
• Time to first value (predicts churn)
• Support ticket volume (predicts satisfaction)

When to use:

• Day-to-day decisions
• Experiment evaluation
• Early problem detection
• Sprint planning

---

The Connection

Leading metrics should PREDICT lagging metrics.

If they don't, you're tracking the wrong leading metrics.

Example 1: E-commerce

Lagging: Monthly revenue Leading:

• Add-to-cart rate (faster signal)
• Checkout start rate
• Payment success rate

Logic: If add-to-cart drops today, revenue drops next week.

---

Example 2: SaaS Product

Lagging: 90-day retention Leading:

• Day 7 activation rate
• Weekly active usage
• Feature adoption in first 14 days

Logic: Users who activate in 7 days have 10x better 90-day retention.

---

Example 3: Marketplace

Lagging: GMV (Gross Merchandise Value) Leading:

• Search-to-view rate
• View-to-cart rate
• Seller response time
• Inventory depth

Logic: More searches + better inventory = more transactions = higher GMV

---

How to Identify Your Leading Metrics

Step 1: Start with Your North Star (Lagging Metric)

What's the ultimate outcome you care about?

Examples:

• Revenue
• User retention
• Engagement
• Customer satisfaction

---

Step 2: Work Backwards to Find Predictors

Use this prompt pattern:

Use /metrics-framework and reference thoughts/shared/pm/context/business-info-template.md

Our North Star / main outcome metric is: [your lagging metric]

Help me identify 3-5 leading indicators that:
1. Happen BEFORE this outcome
2. Predict this outcome with data
3. Are actionable (we can influence them)
4. Give us signal within 1-2 weeks

Our product: [describe product]
User journey: [describe key actions]
Current metrics: [what you track today]

---

Step 3: Validate the Correlation

Run analysis to prove the connection:

1. Cohort analysis:

   • Users with high leading metric → retention/revenue outcome
   • Users with low leading metric → churn outcome

2. Time-series correlation:

   • Leading metric moves up Week 1
   • Lagging metric moves up Week 4-6
   • Correlation coefficient > 0.7

If no correlation exists, it's not a leading indicator.

Minimum Data Quality Bar

Before trusting any correlation between leading and lagging metrics:

Minimum thresholds:

• Sample size: At least 1,000 users (or 100 accounts for B2B) in each comparison group
• Time period: At least 6 weeks of data (2 full product cycles minimum)
• Event frequency: Each user should have 5+ events of the measured behavior (avoid drawing conclusions from single actions)
• Segment stability: The user segments being compared should be stable (not changing composition) during the measurement period

Data quality red flags:

• Correlation based on fewer than 500 users -- mark as "Directional only, not statistically reliable"
• Correlation from a single week of data -- mark as "Preliminary, needs confirmation over 4+ weeks"
• Correlation that only holds for one segment but not others -- note the segment-specific caveat
• Correlation that reverses direction when a confounding variable is controlled for -- do not report as a finding

How to communicate data quality:

• High confidence: "Based on 3,200 users over 8 weeks, X predicts Y with r=0.72"
• Medium confidence: "Directional signal from 800 users over 4 weeks. Needs more data to confirm."
• Low confidence: "Preliminary observation from 200 users. Not yet actionable."

---

Metric Hierarchy

Build a metric tree from lagging → leading:

LAGGING (Outcome)
    ↑
LEADING (Predictive)
    ↑
INPUT METRICS (Drivers)

Example: Slack

Tier 1 - Lagging (Annual):

• Revenue
• Logo retention

Tier 2 - Leading (Monthly):

• Monthly Active Teams (MAT)
• Teams sending 2,000+ messages

Tier 3 - Input Metrics (Weekly):

• Weekly Active Users
• Messages sent per user
• Channels created
• Integrations added

Tier 4 - Activation Metrics (Daily):

• Day 1 setup completion
• First message sent
• Team invites sent

How it works:

• Day 1 activation → predicts Week 1 usage
• Week 1 usage → predicts Month 1 engagement
• Month 1 engagement → predicts annual retention
• Annual retention → drives revenue

---

Dashboard Design

Problem: Most dashboards show only lagging metrics

Solution: Split your dashboard into sections:

1. North Star (Top of Dashboard)

• The ONE lagging metric that matters most
• Monthly/Quarterly trend

2. Leading Indicators (Middle)

• 3-5 metrics that predict North Star
• Weekly trends
• Alert thresholds

3. Input Metrics (Bottom)

• 5-10 levers you can pull
• Daily/Weekly monitoring
• Team-specific metrics

Example Dashboard: SaaS Product

North Star:

• 📊 MRR (Monthly Recurring Revenue): $250K (+12% MoM)

Leading Indicators:

• ⚡ Weekly Active Accounts: 2,500 (+8% WoW)
• ⚡ Day 7 Activation Rate: 45% (+5pp vs last week)
• ⚡ Feature Adoption (core feature): 72% (+3pp)

Input Metrics:

• Signups this week: 320
• Setup completion rate: 75%
• Time to first value: 8 minutes (median)
• Support tickets: 45 (-10 vs last week)
• Invited users per account: 3.2 avg

---

Real-World Examples

Netflix

Lagging: Subscriber growth, Revenue Leading:

• New user activation (% watching 2+ titles in first month)
• Weekly streaming hours per member
• Content completion rates

Why it works: High engagement predicts low churn predicts revenue growth

---

Airbnb

Lagging: Nights booked (GMV) Leading:

• Search-to-booking conversion rate
• Average search results quality
• Host response time
• New listings added

Why it works: More supply + faster responses = more bookings

---

Spotify

Lagging: Premium subscriber growth Leading:

• Free user engagement (hours listened)
• Playlist saves per user
• Weekly active users

Why it works: Highly engaged free users convert to paid

---

Common Patterns

Pattern 1: Activation → Retention → Revenue

Chain:

1. Input: Signup rate
2. Leading: Day 7 activation
3. Leading: Day 30 retention
4. Lagging: LTV

Application: Focus on activation to drive long-term revenue

---

Pattern 2: Engagement → Monetization

Chain:

1. Input: Feature usage
2. Leading: Weekly active usage
3. Leading: Feature depth (# features used)
4. Lagging: Conversion to paid

Application: Drive engagement before asking for money

---

Pattern 3: Quality → Virality → Growth

Chain:

1. Input: User satisfaction score
2. Leading: Invite/referral rate
3. Leading: Viral coefficient (invites per user)
4. Lagging: Organic user growth

Application: Build quality product → users refer friends

---

When Leading Metrics Fail

Red Flag #1: No Correlation with Outcomes

Problem: Your "leading" metric moves, but lagging metric doesn't

Example:

• Leading (claimed): Page views
• Lagging: Revenue
• Reality: Page views don't predict revenue

Fix: Find a different leading metric with proven correlation

---

Red Flag #2: Too Slow to Be Leading

Problem: Your leading metric takes 6 months to show signal

Example:

• Claiming "Q1 signups" is a leading indicator for "Q3 revenue"
• That's still lagging (too slow)

Fix: Find faster signals (weekly, not quarterly)

---

Red Flag #3: Vanity Metrics Disguised as Leading

Problem: Metric looks good but doesn't predict anything

Examples:

• Social media followers (doesn't predict revenue)
• App downloads (doesn't predict active users)
• Website traffic (doesn't predict conversions)

Fix: Validate with cohort analysis

---

Practical Application

For Product Decisions:

Question: Should we build Feature X?

Bad answer: "It might increase revenue" Good answer: "It will increase [leading metric] by X%, which historically drives [lagging metric] by Y%"

---

For Experiments:

Question: Should we run an A/B test for 4 weeks?

Bad answer: "We need to measure revenue impact" Good answer: "We can measure activation rate in Week 1, which predicts retention in Month 1"

---

For Team Goals:

Bad goal: "Increase revenue 20%" Good goal: "Increase activation rate from 45% → 60%, which drives revenue +20%"

Why better: Team can influence activation daily, can't directly influence revenue

---

Metric Selection Checklist

Use this for any metric you're considering:

Leading Metric Criteria:

• [ ] Responds within 1-2 weeks (not months)
• [ ] Predicts lagging outcome (proven with data)
• [ ] Actionable (team can influence it)
• [ ] Measurable (reliable tracking)
• [ ] Understandable (team knows what it means)

Lagging Metric Criteria:

• [ ] Ultimate outcome you care about
• [ ] Aligns with business goals
• [ ] Can't be gamed
• [ ] Measures real value delivery

---

Example Workflow: Building Your Metric System

Step 1: Define North Star (Lagging)

"What's the ultimate outcome?" → Answer: Customer lifetime value (LTV)

Step 2: Identify Leading Indicators

"What predicts LTV?" → Analysis shows: 90-day retention predicts LTV → Leading metric: 90-day retention rate

Step 3: Find Faster Signals

"What predicts 90-day retention?" → Analysis shows: Day 7 activation predicts 90-day retention → Leading metric: Day 7 activation rate

Step 4: Identify Input Drivers

"What drives Day 7 activation?" → Analysis shows:

• Time to first value
• Feature adoption in first session
• Invite completion → Input metrics: TTV, first-session features used, invites sent

Step 5: Build Dashboard

• North Star: LTV
• Leading (slow): 90-day retention
• Leading (fast): Day 7 activation
• Inputs: TTV, feature adoption, invites

---

Common Mistakes

❌ Only tracking lagging metrics

• Problem: No early warning system
• Fix: Add 3-5 leading indicators

❌ Claiming correlation without data

• Problem: "Trust me, this predicts that"
• Fix: Run cohort analysis to prove it

❌ Too many leading metrics

• Problem: Analysis paralysis
• Fix: Pick 3-5 that matter most

❌ Optimizing leading metrics that don't matter

• Problem: Increase page views, revenue doesn't move
• Fix: Validate correlation first

---

Output Template

Metrics Framework: [Feature/Product Name]

North Star: [Metric] Framework Type: [Leading/Lagging Hierarchy or Input/Output Model]

Metric Hierarchy

| Tier | Metric | Current | Target | Cadence | Owner | | ------------------- | ----------------------------------------------- | ---------- | ------ | ---------------- | -------------- | | Lagging (Quarterly) | [e.g., Revenue, LTV, 90-day retention] | [Baseline] | [Goal] | Quarterly review | [Executive/PM] | | Leading (Weekly) | [e.g., Day 7 activation, WAU, feature adoption] | [Baseline] | [Goal] | Weekly review | [PM/Growth] | | Leading (Weekly) | [e.g., Time to first value, invite rate] | [Baseline] | [Goal] | Weekly review | [PM/Growth] | | Input (Daily) | [e.g., Signups, setup completion, first action] | [Baseline] | [Goal] | Daily monitoring | [Team lead] | | Input (Daily) | [e.g., Support tickets, error rate] | [Baseline] | [Goal] | Daily monitoring | [Engineering] |

Correlation Validation Plan

| Hypothesis | Test Method | Data Needed | Timeline | | ---------------------------------------------- | ------------------------------------------------------------ | --------------------------- | ------------------- | | [Leading metric X] predicts [Lagging metric Y] | Cohort analysis: compare high-X vs low-X users on Y outcome | 3 months of user-level data | 2 weeks to validate | | [Input metric A] drives [Leading metric B] | Time-series correlation: does A movement precede B movement? | 6 weeks of daily data | 1 week to validate |

Alert Thresholds

| Metric | Green (On Track) | Yellow (Attention) | Red (Intervention) | Action on Red | | ---------------- | ---------------- | --------------------------------- | --------------------------- | --------------------------------------------------------------------- | | [Lagging metric] | [Above target] | [Within 1 SD or 10% below target] | [2 SD or 20%+ below target] | [Who gets alerted; what investigation to run; what corrective action] | | [Leading metric] | [Above target] | [Within 1 SD or 10% below target] | [2 SD or 20%+ below target] | [Who gets alerted; what investigation to run; what corrective action] | | [Input metric] | [Above target] | [Within 1 SD or 10% below target] | [2 SD or 20%+ below target] | [Who gets alerted; what investigation to run; what corrective action] |

Dashboard Design

Section 1 - North Star (top of dashboard): [The ONE lagging metric, shown as a large number with monthly/quarterly trend line and target overlay]

Section 2 - Leading Indicators (middle): [3-5 weekly metrics shown as sparkline charts with alert thresholds marked. Each metric includes a comparison to prior week and target.]

Section 3 - Input Metrics (bottom): [5-10 daily levers shown as a table with current value, WoW change, and status indicator (green/yellow/red). Grouped by team ownership.]

---

Alert Threshold Guidance

For each key metric, define three thresholds:

• Green (On Track): Metric is at or above target. No action needed. Keep monitoring at standard cadence.
• Yellow (Needs Attention): Metric is trending down or below target by a small margin (typically 1 standard deviation below target, or 10% miss). Action: Increase monitoring cadence (e.g., daily instead of weekly), investigate root cause, prepare a response plan.
• Red (Intervention Needed): Metric has dropped significantly (typically 2 standard deviations below target, or missing by more than 20%). Action: Immediate investigation.

For each Red threshold, specify three things:

1. Who gets alerted: Name the person or team (e.g., "PM + Engineering Lead + VP Product")
2. What investigation to run: Specific analysis (e.g., "Run cohort breakdown by acquisition channel and device type to isolate the drop")
3. What action to take: Concrete response (e.g., "If caused by a recent release, initiate rollback decision within 24 hours. If caused by external factor, schedule war room within 48 hours.")

Calibration tip: Start with thresholds based on historical variance. After 4-6 weeks, adjust based on actual alert volume. If you get Yellow alerts every week, your threshold is too sensitive. If you never get Yellow alerts, it is too loose.

---

Metric Retirement

Leading indicators don't last forever. Retire or replace them when:

Signals it's time to retire a metric:

• The correlation to the lagging outcome has weakened for 2+ consecutive quarters
• The metric has flatlined (no movement despite product changes) -- it may have reached a natural ceiling
• The metric is being gamed (teams optimize for the number rather than the underlying behavior)
• The product has evolved and the metric no longer captures the core user behavior

Retirement process:

1. Flag it: Add a "Under Review" tag to the metric in your dashboard and alert stakeholders
2. Run a replacement analysis: What metric better predicts the lagging outcome now? Use the correlation validation plan to test candidates.
3. Overlap period: Run old and new metrics in parallel for 4-6 weeks to validate the replacement
4. Sunset: Remove old metric from primary dashboard. Archive in a "Retired Metrics" section with retirement rationale and replacement metric.
5. Document: Add to the learning log -- "Metric X retired because [reason]. Replaced by Metric Y because [evidence]."

Common retirement patterns:

• Activation metric retires when onboarding flow changes fundamentally
• Feature adoption metric retires when feature reaches saturation (>80% adoption)
• NPS retires when replaced by a more predictive satisfaction measure (e.g., task completion rate)

---

Multi-Product Guidance

For products with multiple value streams (e.g., separate features for creation, analytics, and collaboration), do not try to force everything into one flat metric hierarchy. Instead:

1. Define a shared North Star that captures overall product value (e.g., "Weekly Active Organizations").
2. Build a metric hierarchy for each value stream:
   • Stream A (Creation): Leading metrics specific to creation (e.g., documents created, time to first publish)
   • Stream B (Analytics): Leading metrics specific to analytics (e.g., dashboards viewed, insights exported)
   • Stream C (Collaboration): Leading metrics specific to collaboration (e.g., comments posted, shared workspaces)
3. Show how each stream rolls up to the North Star: Each stream contributes to the shared North Star through a specific mechanism. Make this explicit (e.g., "Users who use both Creation and Collaboration have 2x higher weekly activity than single-stream users").
4. Each stream needs its own leading indicators, but guardrail metrics can be shared across streams (e.g., NPS, support ticket volume).
5. Avoid averaging across streams. If one stream is thriving and another is declining, an average masks the problem. Report each stream independently and then show the aggregate.

---

Output Integration

Where Files Go

Metric framework documentation:

• Active work: thoughts/shared/pm/metrics/metrics-framework-[date].md (living document for your team)
• Dashboard spec: Share with analytics team for instrumentation

Link to Other Work

After building your framework:

• Reference in strategy - "Our metric hierarchy connects [lagging] to [leading] to [input]"
• Use in feature PRDs - "This feature's success metric is [X], a leading indicator for [lagging]"
• Guide experiments - Teams use leading metrics to evaluate experiments faster
• Inform roadmap - Identify which product work drives which leading metrics

Cross-Skill Integration

Feeds into:

• /define-north-star - Your lagging metric becomes (or informs) the North Star
• /feature-metrics - Feature success metrics are leading indicators in this framework
• /retention-analysis - Leading metrics for retention are identified here
• /activation-analysis - Activation metrics are leading indicators for engagement

Pulls from:

• /write-prod-strategy - Strategy determines what you're optimizing for
• thoughts/shared/pm/metrics/ - Historical data validates correlations
• /define-north-star - What is your ultimate success metric?

---

Related Skills

• /define-north-star - Choose your North Star (lagging metric)
• /activation-analysis - Find leading activation indicators
• /feature-metrics - Choose experiment metrics with STEDII
• /retention-analysis - Analyze retention predictors

---

---

Output Quality Self-Check

Before delivering the metrics framework, verify:

• [ ] North Star is anchored: The framework starts from a clearly defined North Star or lagging metric. If none exists, recommend running /define-north-star first.
• [ ] Three tiers defined: The hierarchy includes at least one metric at each tier: Lagging (quarterly), Leading (weekly), and Input (daily). A framework with only lagging metrics is just a scorecard, not a decision tool.
• [ ] Correlation hypotheses stated: For each leading-to-lagging connection, the hypothesis is explicit and a validation method is described. Do not claim "X predicts Y" without explaining how to prove it.
• [ ] Alert thresholds set: At least the top 3-5 metrics have green/yellow/red thresholds defined with specific actions for Red. A metric without an alert threshold is decoration, not instrumentation.
• [ ] Owners assigned: Every metric has a team or person responsible. Unowned metrics do not get watched.
• [ ] Dashboard design described: The output includes a dashboard layout description showing what goes where and at what cadence. The PM should be able to hand this to an analyst and get a working dashboard.
• [ ] No vanity metrics: Every metric in the framework has a clear connection to either a lagging outcome or an actionable input. If a metric "looks good but does not drive decisions," remove it.
• [ ] Multi-stream handled: If the product has multiple value streams, each stream has its own leading indicators with explicit roll-up to the shared North Star. If the product is single-stream, this is not needed.
• [ ] Metric retirement guidance included: For any existing metrics being reviewed, include signals for when to retire them and a replacement process. New frameworks should note that metrics have a shelf life.
• [ ] Data quality bar stated: For any correlation claims between leading and lagging metrics, state the sample size, time period, and confidence level. Do not present low-confidence correlations as validated findings.

---

Framework credit: Adapted from Aakash Gupta's metrics frameworks. Read: https://www.news.aakashg.com/p/become-an-ab-testing-expert-advanced