samarv/pmf-leading-indicator-assessment

The Sean Ellis Test (or "The 40% Test") provides a leading indicator of product-market fit. While retention cohorts are the ultimate "lagging" proof of fit, this survey allows you to identify if you have a "must-have" product day one, without waiting months for data to mature.

1. Segment the Audience

Do not survey everyone. Surveying people who have only seen a demo or just signed up will result in "noise."

• Target: Users who have experienced the "core" of the product.
• Criteria: Must have used the product at least twice.
• Recency: Must have used the product within the last 1–2 weeks (active users, not churned users).
• Sample Size: Aim for at least 30–50 responses for valid initial signal.

2. Deploy the Primary Question

Ask the single most important question to gauge "must-have" status:

"How would you feel if you could no longer use [Product Name]?"

1. Very disappointed (The "Must-Have" group)
2. Somewhat disappointed (The "Nice-to-Have" group)
3. Not disappointed
4. N/A – I no longer use the product

3. Analyze the Threshold

• 40% or higher "Very Disappointed": You have found a "must-have" vein. You are ready to focus on activation and sustainable growth.
• Below 40%: You are likely in a "commodity" state or targeting the wrong segment. Do not scale marketing yet; you will waste capital.

4. Extract the "Must-Have" Benefit

For the users who said they would be Very Disappointed, run a follow-up qualitative analysis to understand why they care.

Step A: Open-Ended Discovery

Ask: "What is the primary benefit that you get from [Product]?" and "Why is that benefit important to you?"

• Look for recurring language or "hooks" (e.g., "I'm drowning in email").

Step B: Benefit Validation (Multiple Choice)

Once you have 4–5 recurring themes, survey a different group of users.

• Question: "Which of these is the primary benefit you receive?" (Force a choice between the 4–5 themes).
• Question: "What would you use if this product were no longer available?" (Identifies the true competition).

5. Move the Score (The "Lookout" Strategy)

If your score is low (e.g., 10-15%), use this workflow to reach the 40% threshold:

1. Isolate the "Very Disappointed" group: Even if it's only 7%, find out who they are and what feature they use.
2. Reposition the Hook: Update marketing and landing pages to focus exclusively on the specific benefit that 7% loved.
3. Streamline Onboarding: Remove any steps that don't lead directly to that specific benefit. Ensure "Speed to Value" (Aha Moment) happens in the first session.
4. Ignore the "Somewhat Disappointed" users: Do not build features for them yet. Trying to please everyone dilutes the product for your "must-have" core.

---

Example 1: Mobile Security (Lookout)

• Context: A mobile app with backup, phone-finding, and antivirus features had only a 7% PMF score.
• Insight: The 7% who loved it cared exclusively about "Antivirus," even though phone viruses were rare at the time.
• Application: The team repositioned all marketing on "Antivirus" and changed onboarding so the first thing a user saw was an "Antivirus Scanning" progress bar.
• Output: The score jumped to 40% in two weeks because the product now attracted people seeking that specific value and delivered it immediately.

Example 2: Productivity Tool (Xobni)

• Context: Testing the primary benefit of an email plugin.
• Input: Users were asked "Why is finding things faster important to you?"
• Application: Qualitative responses repeatedly used the phrase "I'm drowning in email."
• Output: The marketing was changed to lead with the "Drowning in email?" hook, which significantly increased the conversion of users who eventually became "Very Disappointed" must-have users.

---

Common Pitfalls

• Surveying Demos: Asking "would you use this?" is useless. Only survey users who have actually performed the core action.
• The "Somewhat Disappointed" Trap: Trying to convert "Somewhat Disappointed" users by adding their requested features. This often results in a mediocre product that is "good for everyone but great for no one."
• Ignoring Context: Not asking why a benefit is important. The "why" provides the emotional context needed for high-performing acquisition copy.
• Premature Scaling: Stepping on the gas (paid ads) when the score is at 15%. This results in high churn and "leaky bucket" syndrome.