Abhi5415/gtm-analytics

GTM Analytics

Cross-channel performance report for Clarido's go-to-market: Meta Ads, Instagram, Pinterest, Blog, GA4, and Search Console. Invoked with /gtm-analytics.

Data Sources

| Channel | Method | Script / Tool | Output | |---------|--------|---------------|--------| | Meta Ads | Python API script | scripts/analytics/pull_meta_ads.py | Campaigns, ads, boosted posts, demographics | | Instagram | Python API script | scripts/analytics/pull_instagram_insights.py | Media items with engagement rates, boost flags | | GA4 | Python API script | scripts/analytics/pull_ga4.py | Sessions, pageviews, traffic sources | | Search Console | Python API script | scripts/analytics/pull_search_console.py | Clicks, impressions, queries, page performance | | Pinterest | Chrome + JavaScript | Pinterest Analytics (browser automation) | Account metrics, pin performance | | Blog | Inline (curl + file read) | Health check + GSC cross-reference | Article status, SEO performance per article | | Report | Python aggregator | scripts/analytics/generate_report.py | Unified markdown report |

All intermediate data goes to tmp/analytics/*.json. Final report: reports/gtm/YYYY-MM-DD.md (one file per day, never overwritten).

Workflow

Phase 1: API Data Pull (automated)

Run all 5 scripts. If any fail (token issues, API errors), note the error and continue.

1. mkdir -p tmp/analytics
2. Meta Ads: pipenv run python3 scripts/analytics/pull_meta_ads.py --days 30
   • Pulls campaigns, ad-level data with creative linking, boosted post detection, daily trends, demographics
3. Instagram: pipenv run python3 scripts/analytics/pull_instagram_insights.py --days 30
   • Pulls media items with computed engagement_rate and boosted flag
4. Search Console: pipenv run python3 scripts/analytics/pull_search_console.py --days 30
   • Pulls clicks, impressions, top queries, top pages, device/country breakdowns
5. GA4: pipenv run python3 scripts/analytics/pull_ga4.py --days 30
   • Property 507996912 (owned by [email protected]). Service account has Viewer access.

Phase 2: Pinterest Analytics Scrape (Chrome + JS)

1. Create a new tab and navigate to https://analytics.pinterest.com/

   • If redirected to login, try https://www.pinterest.com/business/hub/ instead
   • The user should already be logged in via their Chrome session

2. Wait for the analytics dashboard to load (3-5 seconds)

3. Run JavaScript to extract account-level metrics:

// Extract Pinterest Analytics dashboard data
(() => {
  const metrics = {};

  // Look for metric cards on the analytics dashboard
  const cards = document.querySelectorAll('[class*="metric"], [class*="Metric"], [class*="stat"], [class*="Stat"], [data-test*="metric"]');
  cards.forEach(card => {
    const label = card.querySelector('[class*="label"], [class*="Label"], [class*="title"]')?.textContent?.trim()?.toLowerCase();
    const value = card.querySelector('[class*="value"], [class*="Value"], [class*="number"]')?.textContent?.trim();
    if (label && value) metrics[label] = value;
  });

  // Fallback: parse visible text for known Pinterest metrics
  if (Object.keys(metrics).length === 0) {
    const text = document.body.innerText;
    const patterns = {
      impressions: /impressions[:\s]*([0-9,]+)/i,
      engagements: /(?:engagements|total engagement)[:\s]*([0-9,]+)/i,
      pin_clicks: /(?:pin clicks|closeups)[:\s]*([0-9,]+)/i,
      outbound_clicks: /(?:outbound clicks|link clicks)[:\s]*([0-9,]+)/i,
      saves: /(?:saves)[:\s]*([0-9,]+)/i,
    };
    for (const [key, pattern] of Object.entries(patterns)) {
      const match = text.match(pattern);
      if (match) metrics[key] = match[1];
    }
  }

  // Try to find top pins table
  const pins = [];
  const pinRows = document.querySelectorAll('[class*="pin-row"], [class*="PinRow"], table tbody tr, [class*="top-pin"]');
  pinRows.forEach(row => {
    const title = row.querySelector('[class*="title"], [class*="pin-name"], img')?.getAttribute('alt') || row.querySelector('[class*="title"]')?.textContent?.trim();
    const impressions = row.querySelectorAll('[class*="value"], td')[0]?.textContent?.trim();
    const saves = row.querySelectorAll('[class*="value"], td')[1]?.textContent?.trim();
    if (title) pins.push({ title, impressions, saves });
  });

  return JSON.stringify({ metrics, pins, url: window.location.href });
})()

4. If account-level data is available, navigate to individual pin analytics pages if needed.

5. Write results to tmp/analytics/pinterest.json:

{
  "source": "pinterest",
  "pulled_at": "2026-02-23T...",
  "account": {
    "impressions": 150,
    "saves": 3,
    "pin_clicks": 5,
    "outbound_clicks": 2
  },
  "pins": [
    {
      "title": "Brain Dump Before Bed",
      "metrics": { "impressions": 50, "saves": 1, "pin_clicks": 2, "outbound_clicks": 1 }
    }
  ]
}

Phase 3: Blog Health Check + SEO Cross-Reference (inline)

No script needed. Do this inline:

1. Read all ~/dev/clarido-landing-page/content/blog/*.md files — extract frontmatter (title, slug, date, tags)
2. For each article: curl -s -o /dev/null -w "%{http_code}" https://clarido.app/blog/{slug}
3. Cross-reference with Search Console top_pages data from tmp/analytics/search_console.json:
   • For each blog article URL, find matching GSC entries
   • Enrich blog.json with GSC clicks/impressions per article
4. Write results to tmp/analytics/blog.json:

{
  "source": "blog",
  "pulled_at": "2026-03-02T...",
  "articles": [
    {
      "title": "Why Brain Dumping Works",
      "slug": "brain-dumping-article",
      "date": "2026-02-19",
      "url": "https://clarido.app/blog/brain-dumping-article",
      "live": true,
      "gsc_clicks": 5,
      "gsc_impressions": 40
    }
  ],
  "total_articles": 4
}

Phase 4: Generate Report

1. Run: pipenv run python3 scripts/analytics/generate_report.py
2. This reads all JSON from tmp/analytics/ and produces reports/gtm/YYYY-MM-DD.md
3. Report structure:
   • Executive Summary (channel snapshot table)
   • What To Do Today (cadence checks, specific action items)
   • Meta Ads (active campaigns, boosted posts with ROI, trends, demographics)
   • Instagram Reels (engagement rates, retention analysis, boost flags)
   • Pinterest (account overview, pin performance)
   • Blog & SEO (article health, Search Console data, GA4 data or "not configured" notice)
   • Cross-Channel Insights (topic matrix, channel summary)
   • Recommendations (budget, content, infrastructure)
   • Data Gaps (status of each data source with fix instructions)

Phase 5: AI Analysis

After the report is generated, read both reports/gtm/YYYY-MM-DD.md and docs/content-strategy.md, then provide opinionated analysis:

The report's "What To Do Today" section now includes a Current Status table (followers, posts this week, channel phase, target frequency) and automated action items based on the content strategy rules. Use this as a starting point, then layer on:

• What to do today: Validate the automated recommendations. Add context — e.g., if the strategy says "post 4 more Reels by Sunday" but engagement rate is dropping, prioritize quality over hitting the number.
• Phase assessment: Are we in the right phase for each channel? Should any phase transitions happen based on the data?
• Scaling signals: Call out any scale-up or scale-down triggers from the strategy doc that are firing (e.g., "engagement rate > 5% for 2 cycles → increase frequency").
• High-ROI ideas: Topics or formats to double down on, content that performed well to repurpose cross-platform.
• Direction changes: Anything the data suggests we should pivot on (e.g., campaign not working, channel underperforming).
• Boosted post ROI: Were the boosts worth the spend? Would the money have been better spent on targeted campaigns?
• SEO opportunities: High-impression low-CTR queries from GSC that we should optimize for. Blog topics to write based on search demand.
• Hypothesis check: What content approaches are working vs not? Is the psychology explainer format resonating? Are Reels driving meaningful engagement?
• Repurposing gaps: Are we cross-posting enough? Does every blog post have 2-3 Pinterest pins? Are high-performing Reels being turned into blog posts?

Phase 6: Email Report (automatic)

After the AI analysis, email the update to both [email protected] and [email protected]. This is NOT optional — always send the email at the end of every run.

• To: [email protected] (use as user_google_email)
• CC: [email protected]
• Subject: Clarido GTM Update — {YYYY-MM-DD}
• Body format: Plain text. Write it like a quick update from a teammate, not a formal report. Include:
  1. One-line summary — what's working, what's not, in one sentence
  2. Numbers that matter — only the 3-5 metrics that actually changed or matter this week (don't dump every number from the report)
  3. What we did today — content published, campaigns adjusted, etc.
  4. What to do next — the 2-3 highest-priority action items with specific next steps
  5. Link to full report — reports/gtm/{YYYY-MM-DD}.md in the repo

Keep it short enough to read in 30 seconds on a phone. No markdown tables in the email — use plain text with dashes or line breaks. No attachments.

Adapting to DOM Changes

Pinterest Analytics changes its DOM structure frequently. The JavaScript snippets above use multiple selector strategies (class patterns, fallback regex on page text) to be resilient. If they return empty data:

1. Use read_page to see the current page structure
2. Adapt selectors in the JavaScript based on what you see
3. Use read_page with filter: "text" as a last resort to get all visible text and parse it manually
4. Document working selectors in the performance.md notes so future runs can reuse them

Critical Details

• Meta token is non-expiring — no refresh needed
• Instagram insights require 24-48h after posting before metrics are available
• Pinterest token expires every 30 days — if 401, run pipenv run python3 scripts/pinterest/refresh_token.py
• GA4 uses service account auth — credentials at GOOGLE_APPLICATION_CREDENTIALS, property ID at GA4_PROPERTY_ID (507996912)
• Search Console has 2-3 day data lag — script auto-adjusts end date to account for this
• GSC uses service account auth — same credentials file, site URL at GSC_SITE_URL
• All JSON files are temporary — they live in tmp/analytics/ and can be regenerated anytime
• Reports are date-stamped — one file per day at reports/gtm/YYYY-MM-DD.md, never overwritten
• Boosted posts are detected by campaign name pattern "Instagram post: ..." — Meta's naming convention for boosts

Key Files

| File | Purpose | |------|---------| | scripts/analytics/pull_meta_ads.py | Meta Ads data → JSON (campaigns, ads, boosted posts) | | scripts/analytics/pull_instagram_insights.py | Instagram insights → JSON (with engagement rate) | | scripts/analytics/pull_ga4.py | GA4 site analytics → JSON | | scripts/analytics/pull_search_console.py | Search Console SEO data → JSON | | scripts/analytics/generate_report.py | Assemble all JSON → markdown report | | reports/gtm/YYYY-MM-DD.md | Unified output report | | docs/content-strategy.md | Posting frequency rules, scaling phases, and triggers | | tmp/analytics/*.json | Intermediate data files |