pr-e/web-search-plus

Web Search Plus

Stop choosing search providers. Let the skill do it for you.

This skill connects you to 6 search providers (Serper, Tavily, Exa, Perplexity, You.com, SearXNG) and automatically picks the best one for each query. Shopping question? → Google results. Research question? → Deep research engine. Need a direct answer? → AI-synthesized with citations. Want privacy? → Self-hosted option.

---

✨ What Makes This Different?

• Just search — No need to think about which provider to use
• Smart routing — Analyzes your query and picks the best provider automatically
• 6 providers, 1 interface — Google results, research engines, neural search, AI answers with citations, RAG-optimized, and privacy-first all in one
• Works with just 1 key — Start with any single provider, add more later
• Free options available — SearXNG is completely free (self-hosted)

---

🚀 Quick Start

# Interactive setup (recommended for first run)
python3 scripts/setup.py

# Or manual: copy config and add your keys
cp config.example.json config.json

The wizard explains each provider, collects API keys, and configures defaults.

---

🔑 API Keys

You only need ONE key to get started. Add more providers later for better coverage.

| Provider | Free Tier | Best For | Sign Up | |----------|-----------|----------|---------| | Serper | 2,500/mo | Shopping, prices, local, news | serper.dev | | Tavily | 1,000/mo | Research, explanations, academic | tavily.com | | Exa | 1,000/mo | "Similar to X", startups, papers | exa.ai | | Perplexity | Via Kilo | Direct answers with citations | kilo.ai | | You.com | Limited | Real-time info, AI/RAG context | api.you.com | | SearXNG | FREE ✅ | Privacy, multi-source, $0 cost | Self-hosted |

Setting your keys:

# Option A: .env file (recommended)
export SERPER_API_KEY="your-key"
export TAVILY_API_KEY="your-key"

# Option B: config.json
{ "serper": { "api_key": "your-key" } }

---

🎯 When to Use Which Provider

| I want to... | Provider | Example Query | |--------------|----------|---------------| | Find product prices | Serper | "iPhone 16 Pro Max price" | | Find restaurants/stores nearby | Serper | "best pizza near me" | | Understand how something works | Tavily | "how does HTTPS encryption work" | | Do deep research | Tavily | "climate change research 2024" | | Find companies like X | Exa | "startups similar to Notion" | | Find research papers | Exa | "transformer architecture papers" | | Get a direct answer with sources | Perplexity | "events in Berlin this weekend" | | Know the current status of something | Perplexity | "what is the status of Ethereum upgrades" | | Get real-time info | You.com | "latest AI regulation news" | | Search without being tracked | SearXNG | anything, privately |

Pro tip: Just search normally! Auto-routing handles most queries correctly. Override with -p provider when needed.

---

🧠 How Auto-Routing Works

The skill looks at your query and picks the best provider:

"iPhone 16 price"              → Serper (shopping keywords)
"how does quantum computing work" → Tavily (research question)
"companies like stripe.com"    → Exa (URL detected, similarity)
"events in Graz this weekend"  → Perplexity (local + direct answer)
"latest news on AI"            → You.com (real-time intent)
"search privately"             → SearXNG (privacy keywords)

What if it picks wrong? Override it: python3 scripts/search.py -p tavily -q "your query"

Debug routing: python3 scripts/search.py --explain-routing -q "your query"

---

📖 Usage Examples

Let Auto-Routing Choose (Recommended)

python3 scripts/search.py -q "Tesla Model 3 price"
python3 scripts/search.py -q "explain machine learning"
python3 scripts/search.py -q "startups like Figma"

Force a Specific Provider

python3 scripts/search.py -p serper -q "weather Berlin"
python3 scripts/search.py -p tavily -q "quantum computing" --depth advanced
python3 scripts/search.py -p exa --similar-url "https://stripe.com" --category company
python3 scripts/search.py -p you -q "breaking tech news" --include-news
python3 scripts/search.py -p searxng -q "linux distros" --engines "google,bing"

---

⚙ Configuration

{
  "auto_routing": {
    "enabled": true,
    "fallback_provider": "serper",
    "confidence_threshold": 0.3,
    "disabled_providers": []
  },
  "serper": {"country": "us", "language": "en"},
  "tavily": {"depth": "advanced"},
  "exa": {"type": "neural"},
  "you": {"country": "US", "include_news": true},
  "searxng": {"instance_url": "https://your-instance.example.com"}
}

---

📊 Provider Comparison

| Feature | Serper | Tavily | Exa | Perplexity | You.com | SearXNG | |---------|:------:|:------:|:---:|:----------:|:-------:|:-------:| | Speed | ⚡⚡⚡ | ⚡⚡ | ⚡⚡ | ⚡⚡ | ⚡⚡⚡ | ⚡⚡ | | Direct Answers | ✗ | ✗ | ✗ | ✓✓ | ✗ | ✗ | | Citations | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | | Factual Accuracy | ⭐⭐⭐ | ⭐⭐⭐ | ⭐⭐ | ⭐⭐⭐ | ⭐⭐⭐ | ⭐⭐⭐ | | Semantic Understanding | ⭐ | ⭐⭐ | ⭐⭐⭐ | ⭐⭐⭐ | ⭐⭐ | ⭐ | | Full Page Content | ✗ | ✓ | ✓ | ✓ | ✓ | ✗ | | Shopping/Local | ✓ | ✗ | ✗ | ✗ | ✗ | ✓ | | Find Similar Pages | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | | RAG-Optimized | ✗ | ✓ | ✗ | ✗ | ✓✓ | ✗ | | Privacy-First | ✗ | ✗ | ✗ | ✗ | ✗ | ✓✓ | | API Cost | $$ | $$ | $$ | Via Kilo | $ | FREE |

---

❓ Common Questions

Do I need API keys for all providers?

No. You only need keys for providers you want to use. Start with one (Serper recommended), add more later.

Which provider should I start with?

Serper — fastest, cheapest, largest free tier (2,500 queries/month), and handles most queries well.

What if I run out of free queries?

The skill automatically falls back to your other configured providers. Or switch to SearXNG (unlimited, self-hosted).

How much does this cost?

• Free tiers: 2,500 (Serper) + 1,000 (Tavily) + 1,000 (Exa) = 4,500+ free searches/month
• SearXNG: Completely free (just ~$5/mo if you self-host on a VPS)
• Paid plans: Start around $10-50/month depending on provider

Is SearXNG really private?

Yes, if self-hosted. You control the server, no tracking, no profiling. Public instances depend on the operator's policy.

How do I set up SearXNG?

# Docker (5 minutes)
docker run -d -p 8080:8080 searxng/searxng

Then enable JSON API in settings.yml. See docs.searxng.org.

Why did it route my query to the "wrong" provider?

Sometimes queries are ambiguous. Use --explain-routing to see why, then override with -p provider if needed.

---

🔄 Automatic Fallback

If one provider fails (rate limit, timeout, error), the skill automatically tries the next provider. You'll see routing.fallback_used: true in the response when this happens.

---

📤 Output Format

{
  "provider": "serper",
  "query": "iPhone 16 price",
  "results": [{"title": "...", "url": "...", "snippet": "...", "score": 0.95}],
  "routing": {
    "auto_routed": true,
    "provider": "serper",
    "confidence": 0.78,
    "confidence_level": "high"
  }
}

---

⚠ Important Note

Tavily, Serper, and Exa are NOT core OpenClaw providers.

❌ Don't modify ~/.openclaw/openclaw.json for these
✅ Use this skill's scripts — keys auto-load from .env

---

🔒 Security

SearXNG SSRF Protection: The SearXNG instance URL is validated with defense-in-depth:

• Enforces http/https schemes only
• Blocks cloud metadata endpoints (169.254.169.254, metadata.google.internal)
• Resolves hostnames and blocks private/internal IPs (loopback, RFC1918, link-local, reserved)
• Operators who intentionally self-host on private networks can set SEARXNG_ALLOW_PRIVATE=1

📚 More Documentation

• FAQ.md — Detailed answers to more questions
• TROUBLESHOOTING.md — Fix common errors
• README.md — Full technical reference

---

🔗 Quick Links

• Serper — Google Search API
• Tavily — AI Research Search
• Exa — Neural Search
• Perplexity — AI-Synthesized Answers (via Kilo Gateway)
• You.com — RAG/Real-time Search
• SearXNG — Privacy-First Meta-Search