alirezarezvani/universal-scraping-architect
engineering/universal-scraping-architect/skills/universal-scraping-architect/SKILL.md
Use for web scraping, crawling, document extraction, API parsing, or building validation-heavy data pipelines using Firecrawl or local Python scripts.
$ install --global
skillsauthnpx skillsauth add alirezarezvani/claude-skills universal-scraping-architectInstall this skill globally with one command. Works with Claude Code, Cursor, and Windsurf.
Security Scan Results
3 of 9 scanners reported clean
Some scanners were skipped, did not run, or reported a non-clean status. Review each row below.
3
Scanners Passed
9
Scanners in report
Clean
TrivyContainer and dependency vulnerability scanner
95%
Clean
SemgrepStatic code analysis for vulnerabilities
95%
Clean
mcp-scan (Snyk)Model Context Protocol security validation
95%
Skipped
Snyk (dep)Open source security scanning
50%
Skipped
Socket.devSupply chain security analysis
50%
Skipped
VirusTotalMulti-engine malware detection
50%
Skipped
CrowdStrikeAdvanced threat intelligence
50%
Skipped
OSV-ScannerOpen Source Vulnerability database check
50%
Skipped
OWASP Dep-Check
50%
Last scanned: Jul 2, 2026, 4:28 AM133.5s8 files scanned
SKILL.md
- name:
- universal-scraping-architect
- description:
- Use for web scraping, crawling, document extraction, API parsing, or building validation-heavy data pipelines using Firecrawl or local Python scripts.
Universal Scraping Architect
Design complete, robust data-extraction pipelines with intelligent routing, validation, and token-budget tracking — not brittle one-off scripts.
Dependency Notice: BYOK (Bring Your Own Key) pattern for Firecrawl; API keys must only be loaded via environment variables. Per-script dependencies:
| Script | Dependencies | Exact CLI |
|---|---|---|
| scripts/validate_extraction.py | stdlib only | python3 scripts/validate_extraction.py output.json --json |
| scripts/firecrawl_example.py | firecrawl, requests (template; --sample runs offline) | python3 scripts/firecrawl_example.py --sample |
| scripts/local_bs4_example.py | beautifulsoup4, pandas (template; --sample runs offline) | python3 scripts/local_bs4_example.py --sample |
Before Starting
Check for context first:
If project-context.md exists, read it before asking questions. Determine the target data format, scale of extraction, and deployment environment before writing any code.
How This Skill Works
This skill supports 3 extraction modes based on intelligent routing:
Mode 1: API-Driven (Firecrawl)
Use when the source is a public URL, heavily dynamic (JS/SPA), requires search-first discovery, or involves bulk crawling across a domain.
Mode 2: Local Python (Traditional)
Use when extracting from local files (PDF, Excel, CSV), the data is private/sensitive, or the target is a simple static HTML page where Firecrawl is overkill.
Mode 3: Hybrid Pipeline
Use when Firecrawl handles URL discovery/web extraction, but local Python (Pandas) is required to clean, normalize, and structure the output before saving.
The Extraction Pipeline
When executing a scraping task, always follow this sequence:
- Route the Approach: Explicitly state whether Firecrawl or Local Python is being used and why.
- Track Budgets: Estimate Firecrawl API quotas or LLM token context limits before executing large jobs.
- Extract Safely: Implement checkpointing for multi-page jobs. Handle pagination and dynamic layouts gracefully. Start from the editable runner templates —
scripts/firecrawl_example.py(Mode 1) orscripts/local_bs4_example.py(Mode 2); run each with--samplefirst to see the expected summary shape without network access. - Validate & Clean: Run
python3 scripts/validate_extraction.py extracted_output.json --jsonon every extraction result before delivering it. It exits 0 only on{"status": "ok"};warning(empty output) orerror(malformed JSON) exit 1 — fix and re-extract, never ship unvalidated data. Beyond this structural gate, also check required fields and duplicates against the pipeline spec before delivering. - Format: Default to CSV for tabular data, JSON for nested structures, and Markdown for clean text.
Proactive Triggers
Surface these issues WITHOUT being asked when you notice them in context:
- Hardcoded API Keys → Flag immediately and rewrite to use
os.getenv('FIRECRAWL_API_KEY'). - Private Data Leakage → If the user asks to send local, sensitive files to an external API, flag the privacy risk and suggest Mode 2 (Local Python).
- Missing Pagination → If the target implies hundreds of records but no pagination logic is requested, flag it and add checkpointing.
Output Artifacts
| When you ask for... | You get... | |---------------------|------------| | "Scrape this site" | A fully validated Python extraction script with routing logic and error handling. | | "Get data from this table" | A clean CSV/JSON dataset with a summary log of row counts and empty values. | | "Crawl these docs" | A Markdown deliverable chunked for LLM token limits. |
Anti-Patterns
- Brittle Selectors: Never use highly nested CSS selectors (e.g.,
div > span > ul > li:nth-child(3)). Use data attributes or robust structural anchors. - Ignoring Etiquette: Never scrape without checking
robots.txtor implementing sensible rate limits. - No Validation: Never blindly write scraped data to a file without checking if the array is empty or missing critical keys.
Related Skills
- data-cleaning: Use when the scraped data requires complex statistical normalization or deduplication.
- browser-automation: Use for highly interactive scraping requiring user emulation (clicks, logins) where Firecrawl is insufficient.
Related Skills
alirezarezvani/weekly-review
development
VerifiedTrustedCommunity
Use when someone wants to run a weekly review, close open loops, audit stalled projects and commitments, get their system back to trusted, restart a lapsed review habit, or says "/cs:weekly-review". Walks David Allen's three-phase loop — GET CLEAR, GET CURRENT, GET CREATIVE — with deterministic scripts that inventory open loops, gate the checklist with named gaps, and score commitment health 0-100.
22,702SKILL.mdUpdated Jul 18, 2026
alirezarezvani/meetings
development
VerifiedTrustedCommunity
Use when someone wants to decide whether a meeting is worth calling, price a meeting in dollars, build a timeboxed agenda with desired outcomes, or turn messy meeting notes into owned action items — or says "should this be a meeting", "/cs:meeting-prep", or "/cs:meeting-actions". Runs a cost gate (ASYNC / NOT-READY / MEET), builds a decision-first agenda, and extracts an owner + due-date checklist that flags every orphan.
22,702SKILL.mdUpdated Jul 18, 2026
alirezarezvani/fable-goal
development
VerifiedTrustedCommunity
Convert a rambling description of a desired outcome into one polished, autonomous /goal prompt ready to paste into a fresh session. Use when the user says "/fable-goal", "turn this into a goal prompt", "write me a fable prompt", "write the prompt that builds X", or rambles about something they want made and asks for the prompt that makes it happen. The output is a single copy-paste prompt, never the build itself. Do NOT use when the user wants the thing built right now in this session — only when they want the PROMPT that will make it happen in a fresh session.
22,702SKILL.mdUpdated Jul 18, 2026
alirezarezvani/deep-work
development
VerifiedTrustedCommunity
Use when someone wants to plan a deep work day, time-block their calendar or task list, budget or cut shallow work, protect focus hours, track deep-work sessions and streaks, run an end-of-day shutdown ritual, or says "/deep-work" or "/time-block". Classifies tasks deep vs shallow, builds an energy-first time-blocked schedule that refuses deep demand past the 4-hour ceiling, batches shallow work into at most two windows, and logs focus sessions against a weekly target.
22,702SKILL.mdUpdated Jul 18, 2026