praveenmaiya/analyze-ctr
.claude/skills/analyze-ctr/SKILL.md
Run Thompson Sampling CTR analysis on email treatments. Use when analyzing treatment performance or optimizing email campaigns.
data-ai
Updated Apr 17, 2026
$ install --global
skillsauthnpx skillsauth add praveenmaiya/holley-rec analyze-ctrInstall this skill globally with one command. Works with Claude Code, Cursor, and Windsurf.
Security Scan Results
3 of 9 scanners reported clean
Some scanners were skipped, did not run, or reported a non-clean status. Review each row below.
3
Scanners Passed
9
Scanners in report
Clean
TrivyContainer and dependency vulnerability scanner
95%
Clean
SemgrepStatic code analysis for vulnerabilities
95%
Clean
mcp-scan (Snyk)Model Context Protocol security validation
95%
Skipped
Snyk (dep)Open source security scanning
50%
Skipped
Socket.devSupply chain security analysis
50%
Skipped
VirusTotalMulti-engine malware detection
50%
Skipped
CrowdStrikeAdvanced threat intelligence
50%
Skipped
OSV-ScannerOpen Source Vulnerability database check
50%
Skipped
OWASP Dep-Check
50%
Last scanned: Apr 24, 2026, 9:01 PM1.9s1 file scanned
SKILL.md
- name:
- analyze-ctr
- description:
- Run Thompson Sampling CTR analysis on email treatments. Use when analyzing treatment performance or optimizing email campaigns.
- allowed-tools:
- Bash, Read, Glob, Grep
CTR Analysis Skill
Analyzes email treatment click-through rates using Bayesian Thompson Sampling.
When to Use
- Analyzing which treatments perform best
- Checking if bandit model is learning correctly
- Comparing CTR across treatment groups
- Preparing treatment optimization recommendations
Process
Step 1: Run the Bandit Analysis
python src/bandit_click_holley.py
This computes:
- CTR per treatment (clicks / views)
- Beta posterior parameters (α = 1 + clicks, β = 1 + views - clicks)
- Thompson Sampling selection probabilities
Step 2: Interpret Results
Key metrics to report:
| Metric | Meaning |
|--------|---------|
| posterior_mean | Bayesian CTR estimate |
| posterior_std | Uncertainty (higher = less data) |
| selection_pct | % of time bandit would pick this treatment |
Step 3: Quick SQL Analysis (if needed)
For raw funnel data:
bq query --use_legacy_sql=false "
SELECT
treatment_id,
COUNT(DISTINCT CASE WHEN interaction_type = 'VIEWED' THEN user_id END) as views,
COUNT(DISTINCT CASE WHEN interaction_type = 'CLICKED' THEN user_id END) as clicks,
SAFE_DIVIDE(
COUNT(DISTINCT CASE WHEN interaction_type = 'CLICKED' THEN user_id END),
COUNT(DISTINCT CASE WHEN interaction_type = 'VIEWED' THEN user_id END)
) as ctr
FROM \`auxia-gcp.company_1950.treatment_interaction\`
WHERE DATE(TIMESTAMP_MICROS(interaction_timestamp_micros)) >= DATE_SUB(CURRENT_DATE(), INTERVAL 60 DAY)
GROUP BY treatment_id
ORDER BY ctr DESC
"
Treatment Categories
Reference files:
- Personalized:
configs/personalized_treatments.csv(10 treatments) - Static:
configs/static_treatments.csv(22 treatments)
Output Format
Report should include:
- Top 5 treatments by posterior mean CTR
- Bottom 5 treatments (candidates for pausing)
- High uncertainty treatments (need more data)
- Recommendation on treatment optimization
Related Files
src/bandit_click_holley.py- Core analysis codesql/reporting/campaign_performance.sql- Funnel queriesdocs/analysis/model_ctr_comparison_2025_12_17.md- Analysis methodology
Related Skills
praveenmaiya/weekly-update
testing
VerifiedTrustedCommunity
Generate a team-facing weekly status update from STATUS_LOG.md and git history.
SKILL.mdUpdated Apr 17, 2026
praveenmaiya/validate
testing
VerifiedTrustedCommunity
Run QA validation checks on the recommendation pipeline output. Use after pipeline runs to verify data quality.
SKILL.mdUpdated Apr 17, 2026
praveenmaiya/uplift
research
VerifiedTrustedCommunity
Compare Personalized vs Static treatment performance with unbiased methodology. Use for A/B analysis and treatment comparison.
SKILL.mdUpdated Apr 17, 2026
praveenmaiya/status
testing
VerifiedTrustedCommunity
Show current pipeline and deployment status. Use for quick health check.
SKILL.mdUpdated Apr 17, 2026