openclaw/adamnaghs/safe-web

safe-web

Secure web fetch and search with PromptGuard scanning.

Status

✅ Working

Purpose

Protects against prompt injection attacks hidden in web content before returning it to the AI. Wraps web fetching and searching with security scanning.

Installation

Requires PromptGuard and Python dependencies:

# Install PromptGuard first
cd /home/linuxbrew/.openclaw/workspace/skills/prompt-guard
pip3 install --break-system-packages -e .

# Install web dependencies (if not present)
pip3 install --break-system-packages requests beautifulsoup4

Usage

Fetch Command

Fetch a URL and scan the content:

# Basic fetch
safe-web fetch https://example.com/article

# Save to file
safe-web fetch https://example.com --output article.txt

# JSON output for automation
safe-web fetch https://example.com --json

# Strict mode (block on MEDIUM)
safe-web fetch https://example.com --strict

Search Command

Search the web and scan results:

# Basic search
safe-web search "AI safety research"

# More results
safe-web search "stock market news" --count 10

# JSON output
safe-web search "machine learning" --json

Exit Codes

| Code | Meaning | |------|---------| | 0 | Success - content/results are clean | | 1 | Error (network, parsing, etc.) | | 2 | Threat detected - content blocked |

Configuration

Environment Variables

• BRAVE_API_KEY - API key for Brave Search (optional, enables search command)
  • Get one at: https://brave.com/search/api/

Symlink (Recommended)

Create a system-wide symlink so safe-web works from any directory:

sudo ln -s /home/linuxbrew/.openclaw/workspace/skills/safe-web/scripts/safe-web.py /usr/local/bin/safe-web

After creating the symlink, you can use safe-web directly without specifying the full path.

How It Works

Fetch Flow

1. Downloads URL content with requests
2. Extracts text using BeautifulSoup (removes scripts, styles)
3. Scans extracted text with PromptGuard
4. Returns clean content or blocks with SHIELD report

Search Flow

1. Queries Brave Search API (requires API key)
2. Scans each result title and description
3. Filters out suspicious results
4. Returns only clean results

Security Model

Fail-closed: If PromptGuard cannot be loaded or scanning fails, the tool reports an error rather than returning unverified content.

Content sanitization: HTML is parsed and scripts/styles are removed before scanning to reduce false positives.

No execution: This tool only fetches and scans. It never executes JavaScript or runs commands found in web content.

Example Output

Clean Fetch

Fetching: https://site.com/article
Fetched 1523 characters
Scanning with PromptGuard...

Article content here...

Blocked Content

Fetching: https://suspicious-site.com
Fetched 2048 characters
Scanning with PromptGuard...
============================================================
🛡️  SAFE-WEB SECURITY ALERT
============================================================
Source: https://suspicious-site.com
Severity: CRITICAL
Action: BLOCK_NOTIFY
Patterns Matched: 8

Detected Patterns:
  - instruction_override_en
  - role_manipulation_en
  - system_impersonation_en
============================================================

Content from https://suspicious-site.com has been blocked.

Search Results

Searching: AI research
Found 5 results, scanning...

Showing 3 clean results:

1. Latest AI Research Papers
   URL: https://arxiv.org/list/ai/recent
   Recent submissions in artificial intelligence...

2. AI Safety Institute
   URL: https://www.safe.ai/
   Research and development for safe AI systems...

When to Use

Use safe-web when:

• Fetching content from untrusted URLs
• Scraping web pages for analysis
• Searching and processing web results
• Any web content will enter the AI context window

Use standard web_fetch/web_search tools only for:

• Trusted, known-safe domains
• Internal documentation sites
• When you explicitly want to bypass scanning

Comparison with Native Tools

| Feature | Native web_fetch | safe-web fetch | |---------|-------------------|------------------| | Fetches HTML | ✅ | ✅ | | Extracts text | ✅ | ✅ | | Injection scanning | ❌ | ✅ | | JSON output | ✅ | ✅ | | Save to file | ❌ | ✅ | | Exit codes | 0/1 | 0/1/2 (security) |

Dependencies

• Python 3.8+
• PromptGuard 3.1.0+ (installed in workspace)
• requests
• beautifulsoup4
• Brave Search API key (for search command)

Limitations

• Search requires Brave API key (free tier available)
• Fetch does not execute JavaScript (static HTML only)
• Large pages may be truncated during text extraction
• Network timeouts default to 30 seconds