arabold/fetch-url

Fetch URL

Fetch a single URL and convert its content to clean Markdown. This does not add anything to the documentation index; it is a one-shot read operation.

When to use

• You need to read a single web page and get its content as Markdown.
• You want to inspect a documentation page before deciding whether to index it.
• You need the raw text of a URL for summarisation or analysis.

If you need to index the content for repeated searching, use the docs-manage skill (scrape command) instead.

Command

npx @arabold/docs-mcp-server@latest fetch-url <url> [options]

| Flag | Default | Description | |------|---------|-------------| | --follow-redirects | true | Follow HTTP redirects | | --no-follow-redirects | | Disable following redirects | | --scrape-mode auto\|fetch\|playwright | auto | HTML processing strategy | | --header "Name: Value" | | Custom HTTP header (repeatable) | | --quiet | | Suppress non-error diagnostics | | --verbose | | Enable debug logging |

Scrape modes

| Mode | When to use | |------|-------------| | auto | Default. Tries a simple HTTP fetch first, falls back to Playwright for JS-rendered pages. | | fetch | Force a plain HTTP fetch. Fastest, but misses content rendered by JavaScript. | | playwright | Force a headless browser. Use for SPAs or pages that require JavaScript to render. |

Examples

# Fetch a documentation page
npx @arabold/docs-mcp-server@latest fetch-url https://react.dev/reference/react/useEffect

# Fetch with custom auth header
npx @arabold/docs-mcp-server@latest fetch-url https://docs.internal.com/api \
  --header "Authorization: Bearer tok_xxx"

# Force Playwright for a JS-heavy page
npx @arabold/docs-mcp-server@latest fetch-url https://some-spa.dev/docs --scrape-mode playwright

# Disable redirect following
npx @arabold/docs-mcp-server@latest fetch-url https://example.com/old-page --no-follow-redirects

Output

The command writes the converted Markdown text directly to stdout. The global --output flag is accepted but has no effect because the result is already plain text, not structured data.

Diagnostics and errors go to stderr and are suppressed by default in non-interactive sessions. Use --verbose (or set LOG_LEVEL=INFO) to re-enable them. Use --quiet to suppress all non-error diagnostics regardless of session type.

Tips

• Pipe the output to a file if you want to save it: npx @arabold/docs-mcp-server@latest fetch-url <url> > page.md
• Combine with search: fetch a page to read its full content after search returns a relevant URL.
• For pages behind authentication, use --header to pass cookies or tokens.