nimbleway/nimble-web-expert

Nimble Web Expert

Web extraction, search, and URL discovery using the Nimble CLI. Returns clean structured data from any website.

User request: $ARGUMENTS

Core principles

• Route by intent first. Named site/domain → check for pre-built agent first (announce it out loud). Direct URL → nimble extract. Research/topic → nimble search. Discover/crawl URLs → nimble map or nimble crawl.
• One command → present results → done. Run once with --transform "data.parsing" for agents. Show the data immediately as a table. Do NOT experiment, loop, or write Python to parse output.
• Multiple inputs → always parallel. 2+ URLs/keywords/ASINs → &+wait. 6–20 → xargs -P. 20+ → Python asyncio script. See references/batch-patterns.md.
• Escalate render tiers silently. Tier 1 → 2 → 3 → … without asking. Surface a decision only when all tiers fail and investigation tools are needed.
• Never answer from training data. Live prices, current news, today's listings → always fetch via Nimble. If unavailable, say so.
• AskUserQuestion at every meaningful choice. Header ≤12 chars, 2–4 options, label 1–5 words, recommended option first. Never present choices as numbered prose.
• Save all outputs to .nimble/. Never leave extraction results in memory only.
• Verify the connection BEFORE working — don't fire a data call and react to the error. With bash, nimble --version + NIMBLE_API_KEY confirms the CLI path; otherwise run one read-only mcp__plugin_nimble_nimble__nimble_agents_list probe. Success = connected; an auth/not-connected error or a response containing an OAuth authorization URL = not connected.
• No working CLI and no connected MCP → stop. Do not fall back to WebFetch, WebSearch, curl, or dangerouslyDisableSandbox. If the plugin is installed but the connector isn't connected (typical Cowork / claude.ai), surface the verbatim connect steps from rules/setup.md and stop; if no plugin at all, follow the install flow in rules/setup.md.
• If a tool hands back an OAuth "Authorize" link instead of data, present it exactly as given and stop. Never invent a "paste the URL back" / "I'll complete the connection" step — none exists — and never claim tools "will activate" then call them in the same turn. Wait for the user to authorize, then retry or re-probe.

Skill ecosystem

| Skill | Best for | Key commands | | ---------------------------------- | -------------------------------------------------------------------------- | ------------------------------------------------ | | nimble-web-expert (this skill) | Real-time data — fetch any URL, search, map, crawl, run existing agents | extract, search, map, crawl, agent run | | nimble-agent-builder | Build reusable agents — create, refine, publish named extraction templates | CLI: generate, get-generation, publish |

Hand off to nimble-agent-builder only when all of these are true: the user has signalled a recurring/scheduled need, the pattern is repetitive (same site, same fields), and they've seen and approved the results. Don't ask after every extract — only when language clearly signals a recurring workflow ("I want to do this every day", "build me a pipeline", "make this reusable").

For agent refinement: "Agent updates are handled by nimble-agent-builder — it can refine the existing agent without rebuilding from scratch."

Interactive UX

• Use AskUserQuestion at every meaningful choice — never guess, never ask in prose.
• Ambiguous request (no URL, vague topic): ask before running — "What would you like to do?" → Search / Fetch URL / Discover URLs / Call API
• Before running a search (if task maps to a specific focus mode): offer focus mode — General / News / Coding / Shopping / Academic / Social
• After all tiers fail: check investigation tools (which browser-use, python3 -c "from playwright.sync_api...") and ask whether to investigate with browser-use, Playwright, or skip.
• After presenting results, always close with: "Were these results what you needed?" → Looks great! / Mostly good / Not quite / Skip feedback

Prerequisites

Pick CLI or MCP at session start — same skill, two transports. Once a transport is selected, stick with it for the session and don't re-probe on every command.

nimble --version && echo "${NIMBLE_API_KEY:+API key: set}"        # CLI path
# OR (fallback when shell isn't available)
claude mcp list 2>/dev/null | grep -q "nimble" && echo "MCP: ok"  # plugin MCP

• CLI ready (version + API key both print) → proceed to Step 0, use nimble ... commands.
• MCP connected (no CLI, but plugin is installed) → proceed to Step 0, use mcp__plugin_nimble_nimble__* tools instead.
• Neither → load rules/setup.md for the environment-aware install flow. Any Claude product (Code, Cowork, claude.ai) → /plugin install nimble. Codex or other terminal-only agents → npm i -g @nimble-way/nimble-cli. Cursor / VS Code / generic MCP clients → paste the mcp.json snippet.

If bash is denied: you're in a Cowork-like / MCP-only host. Use mcp__plugin_nimble_nimble__* tools, but verify the connection first with one read-only nimble_agents_list probe. If the probe fails with an auth/not-connected error or returns an OAuth authorization URL, the connector isn't connected — surface the connection steps from Core principles and stop (and never invent an auth-completion flow). Never substitute WebFetch, WebSearch, curl, or any other tool for Nimble operations.

---

Analyze & Route

| User signal | Command | Notes | | ---------------------------------- | --------------------------------------------- | ---------------------------------------------- | | Names a specific site or domain | nimble agent → nimble extract if no agent | Always check for agent first — announce it | | Provides a direct URL | nimble extract | Skip agent check | | Research, topic, or vertical query | nimble search | Use focus modes for news, jobs, shopping, etc. | | "Find URLs / sitemap / all pages" | nimble map | Returns URL list + metadata | | "Crawl / archive a whole section" | nimble crawl | Async bulk extraction |

Step 0 — Agent check (when a domain is named)

Pre-built agents return clean structured data with zero selector work. Always check first.

Always verbalize — never silently:

1. Announce: "Let me check if there's a pre-built Nimble agent for [site]..."
2. Report: "Found <agent_name> — using it now." or "No pre-built agent — falling back to extraction."

Lookup order:

1. ~/.claude/skills/nimble-web-expert/learned/examples.json → agents[] array
2. references/nimble-agents/SKILL.md → baked-in table (50+ sites)
3. nimble agent list --limit 100 --search "<domain or vertical>" → show table, confirm with user
4. No match → proceed to extract/search

Run with --transform "data.parsing" — always:

nimble --transform "data.parsing" agent run --agent <name> --params '{"keyword": "..."}'

Do NOT run without --transform "data.parsing" and then parse raw output. The raw response contains html (useless), headers, and parsing (what you want). The transform flag extracts parsing in one shot.

For the full agent list (50+ sites), see references/nimble-agents/SKILL.md.

⚠️ google_search is for SEO/SERP rank analysis only — not general information retrieval. For finding information, use nimble search.

---

Workflow

| Situation | Command | Reference | | ------------------------------- | -------------------------------------------- | ---------------------------------------------------- | | Site/domain → check agent first | nimble agent list → nimble agent run | references/nimble-agents/SKILL.md | | Direct URL | nimble extract | references/nimble-extract/SKILL.md | | Search the live web | nimble search | references/nimble-search/SKILL.md | | Discover URLs on a site | nimble map | references/nimble-map/SKILL.md | | Bulk crawl a section | nimble crawl run | references/nimble-crawl/SKILL.md | | Batch agents (up to 1,000) | nimble agent run-batch | references/nimble-agents/SKILL.md | | Batch extract (up to 1,000) | nimble extract-batch | references/nimble-extract/SKILL.md | | Poll tasks / batches / results | nimble tasks / nimble batches | references/nimble-tasks/SKILL.md | | Unknown selectors or XHR path | browser-use or Playwright investigation | references/nimble-extract/browser-investigation.md | | Proven site patterns | copy a recipe | references/recipes.md | | 2+ inputs | parallel bash &+wait or generated script | references/batch-patterns.md |

For the full extract waterfall (tiers, flags, browser actions, network capture), see references/nimble-extract/SKILL.md.

---

Response shapes

| Command | Output | | ---------------- | --------------------------------------------------------------------------- | | nimble agent | Structured data in data.parsing — array (SERP/list) or dict (PDP/product) | | nimble extract | HTML, Markdown, or parsed JSON — depends on --format and --parse | | nimble search | Structured results array (title, URL, description) | | nimble map | URL list + metadata | | nimble crawl | Async job — poll with nimble crawl status <job_id> |

Agent runs always need --transform "data.parsing". If the agent name suggests a list (serp, search, plp), expect an array. If it suggests a single item (pdp, product, profile), expect a dict.

Output & Organization

mkdir -p .nimble   # save all outputs here

Naming: .nimble/<site>-<task>.md (e.g. .nimble/amazon-airpods.md, .nimble/yelp-sf-italian.json)

Working with saved files:

wc -l .nimble/page.md && head -100 .nimble/page.md
grep -n "price\|rating" .nimble/page.md | head -30

End every response with: Source: [URL] — fetched live via Nimble CLI

---

Self-Improvement

The skill maintains ~/.claude/skills/nimble-web-expert/learned/examples.json.

• At task start: read the file, scan good[] for url_pattern matches → use documented command/tier as starting point. Scan bad[] → avoid documented pitfalls.
• After presenting results: ask "Were these results what you needed?" → on positive feedback, append to good[] with url_pattern, task, command, tier, notes. On negative feedback, ask "What went wrong?" and append to bad[] with url_pattern, task, issue, avoid, better.
• Keep entries concise — 5–10 per site. Only write on real feedback, never speculatively.

---

Guardrails

• NEVER answer from training data for live prices, current news, or real-time data. If Nimble is unavailable, say so.
• NEVER skip Step 0 silently. Even if certain there's no agent, announce the check before running extract/search/map.
• NEVER retry the same render tier. If a tier returns empty or blocked, escalate — do not re-run.
• NEVER substitute WebFetch, WebSearch, curl, or wget for nimble operations. They're not in allowed-tools — if a Nimble transport isn't available, stop and follow the guidance in the no-transport branch of Core principles. Don't try to work around it.
• NEVER load reference files speculatively. Only read a reference when the current task explicitly needs it.
• Task agents MUST use run_in_background=False. See nimble-agent-builder delegation model for the why.
• Hard retry limit. On error (not empty content): retry at most 2 times with different flags. After 2 errors, report and stop.
• Hard 429 rule. On rate-limit error: stop immediately. Do not retry or switch tiers.

---

Reference files

Load only when needed:

| File | Load when | | ---------------------------------------------------- | ----------------------------------------------------------------------------- | | references/recipes.md | Need a proven command for a common site (Amazon, Yelp, LinkedIn…) | | references/nimble-agents/SKILL.md | Step 0 lookup — full agent table (50+ sites) | | references/nimble-extract/SKILL.md | Extract flags, render tiers, browser actions, network capture, parser schemas | | references/nimble-search/SKILL.md | Search flags, all 8 focus modes | | references/nimble-map/SKILL.md | Map flags, response structure | | references/nimble-crawl/SKILL.md | Full async crawl workflow | | references/nimble-tasks/SKILL.md | Poll tasks/batches, fetch results — for async, batch, and crawl operations | | references/nimble-extract/browser-investigation.md | Tier 6 — CSS selector/XHR discovery with browser-use or Playwright | | references/nimble-extract/parsing-schema.md | Parser types, selectors, extractors, post-processors | | references/nimble-extract/browser-actions.md | Full browser action types and parameters | | references/nimble-extract/network-capture.md | Filter syntax, XHR mode, capture+parse patterns | | references/nimble-search/search-focus-modes.md | Decision tree, mode details, combination strategies | | references/batch-patterns.md | Parallel bash patterns for 2–5, 6–20, and 20+ inputs | | references/error-handling.md | Error codes, known site issues, troubleshooting |