mohitagw15856/ab-test-readout
plugins/pm-data/skills/ab-test-readout/SKILL.md
Analyse a finished A/B test and write the readout — the result, whether it's statistically and practically significant, what it means, and the ship/no-ship call. Use when asked to analyse experiment results, write an A/B test readout, interpret test data, or decide whether to ship a variant. Produces a clear verdict with the lift and confidence, segment cuts, the risks (peeking, novelty, sample), and a recommendation. Distinct from planning a test — this reads results.
$ install --global
skillsauthnpx skillsauth add mohitagw15856/pm-claude-skills ab-test-readoutInstall this skill globally with one command. Works with Claude Code, Cursor, and Windsurf.
Security Scan Results
3 of 9 scanners reported clean
Some scanners were skipped, did not run, or reported a non-clean status. Review each row below.
3
Scanners Passed
9
Scanners in report
Clean
TrivyContainer and dependency vulnerability scanner
95%
Clean
SemgrepStatic code analysis for vulnerabilities
95%
Clean
mcp-scan (Snyk)Model Context Protocol security validation
95%
Skipped
Snyk (dep)Open source security scanning
50%
Skipped
Socket.devSupply chain security analysis
50%
Skipped
VirusTotalMulti-engine malware detection
50%
Skipped
CrowdStrikeAdvanced threat intelligence
50%
Skipped
OSV-ScannerOpen Source Vulnerability database check
50%
Skipped
OWASP Dep-Check
50%
Last scanned: Jun 22, 2026, 6:56 AM102.8s1 file scanned
SKILL.md
- name:
- ab-test-readout
- description:
- Analyse a finished A/B test and write the readout — the result, whether it's statistically and practically significant, what it means, and the ship/no-ship call. Use when asked to analyse experiment results, write an A/B test readout, interpret test data, or decide whether to ship a variant. Produces a clear verdict with the lift and confidence, segment cuts, the risks (peeking, novelty, sample), and a recommendation. Distinct from planning a test — this reads results.
A/B Test Readout Skill
The hard part of an experiment is the readout: not "B won" but "is this real, is it big enough to matter, and should we ship?" This skill turns results into an honest decision — and flags the ways A/B results lie.
Working from a brief
Given results (even partial), write the full readout anyway. If significance isn't provided, reason about it from the numbers and flag what's needed to confirm. Mark assumed figures. Never declare a winner without addressing significance and sample.
Required Inputs
Ask for (if not already provided):
- The hypothesis and the primary metric
- Results — control vs variant: conversions/rate, sample size per arm, duration
- Guardrail metrics (revenue, retention, latency, complaints) that mustn't regress
- Pre-registered decision rule (what would count as a win) if one exists
Output Format
1. Verdict (one line)
Ship / Don't ship / Inconclusive — keep running — with the headline number.
2. The result
| Metric | Control | Variant | Relative lift | Significant? | |---|---|---|---|---| | Primary | | | | p / CI | | Guardrail(s) | | | | |
State statistical significance (p-value / confidence interval) and practical significance (is the lift big enough to matter given the cost?).
3. Did it really win?
Address the ways A/B tests mislead:
- Sample / power — was the test adequately powered, or under-sampled?
- Peeking — was the call made early, inflating false positives?
- Novelty / primacy — could the effect fade?
- Segments — does the win hold across key segments, or is it driven by one?
4. Segment cuts
Where the effect is strong vs flat vs negative (new vs returning, platform, geography).
5. Recommendation & next step
Ship / iterate / re-run, plus what to monitor post-launch or what the follow-up test should isolate.
Quality Checks
- [ ] Distinguishes statistical from practical significance
- [ ] Checks guardrail metrics, not just the primary
- [ ] Flags peeking, power, novelty, and segment-driven wins
- [ ] Recommendation follows from the evidence, with a monitoring/next-test step
- [ ] Doesn't declare a winner on an underpowered or peeked result
Anti-Patterns
- "B won by 8%!" with no significance or sample size
- Calling a result early (peeking) and shipping
- Ignoring a guardrail regression because the primary went up
- A statistically significant but practically meaningless lift treated as a win
Related Skills
mohitagw15856/win-loss-analysis
business
VerifiedTrustedCommunity
Analyze why deals are won and lost and turn it into an action plan. Use when asked to run a win/loss analysis, review closed-won and closed-lost deals, understand why the team is losing to a competitor, or summarize sales feedback into patterns. Produces a structured win/loss report with themes, win/loss rates by segment and competitor, representative quotes, and prioritized actions for product, marketing, and sales.
1,117SKILL.mdUpdated Jul 2, 2026
mohitagw15856/which-skill
development
VerifiedTrustedCommunity
Route a fuzzy request to the right skill in this library. Use when the user is unsure which skill fits, asks 'which skill should I use for X', describes a task without naming a skill, or when a request could plausibly match several skills. Produces a best-fit recommendation with the inputs to gather, a runner-up with the tie-breaker, and a workflow recipe when the job spans multiple skills.
1,117SKILL.mdUpdated Jul 2, 2026
mohitagw15856/vuln-triage
testing
VerifiedTrustedCommunity
Triage a vulnerability or scanner finding — assess real severity, exploitability, and how urgently to fix. Use when asked to triage a CVE, prioritize scanner/pentest findings, assess a vuln's risk, or decide what to patch first. Produces a triage verdict: CVSS-informed severity adjusted for your context, exploitability, real risk, a fix/mitigation, and an SLA — so you fix what matters, not just what's red.
1,117SKILL.mdUpdated Jul 2, 2026
mohitagw15856/voice-of-customer-program
development
VerifiedTrustedCommunity
Stand up a Voice of Customer (VoC) program that turns feedback into action. Use when asked to build a VoC program, design a customer feedback loop, consolidate feedback sources, or set up a closed-loop feedback process. Produces a VoC program design — objectives, feedback sources and channels, a taxonomy, collection and analysis cadence, closed-loop routing, ownership, and success metrics.
1,117SKILL.mdUpdated Jul 2, 2026