orcaqubits/nlweb-ask-endpoint

NLWeb /ask Endpoint

Before writing code

Fetch live spec:

1. Fetch https://github.com/nlweb-ai/NLWeb/blob/main/docs/nlweb-rest-api.md for the canonical /ask contract — request params, response shape, and streaming format.
2. Fetch https://github.com/nlweb-ai/NLWeb/blob/main/docs/life-of-a-chat-query.md to trace a request end-to-end.
3. Fetch https://github.com/nlweb-ai/NLWeb/blob/main/docs/nlweb-headers.md for the in-stream "headers" mechanism (license, data-retention, rate-limit messages are NOT HTTP headers).
4. Web-search the latest release notes for the v0.55+ structured POST body shape (query.text, prefer.mode, prefer.streaming, meta.version) — this is newer than the GET-only legacy contract.
5. Check webserver/routes/api.py in the live repo to confirm exact param names.

Conceptual Architecture

Routes

| Route | Method | Purpose | |-------|--------|---------| | /ask | GET, POST | Main NL query | | /who | GET | Site relevance for a query (federated) | | /sites | GET | List configured sites | | /config | GET | Public config (safe subset) |

Request Parameters

Verify exact names against the live routes/api.py. Stable subset:

| Param | Type | Required | Default | Notes | |-------|------|----------|---------|-------| | query | string | yes | — | NL question | | site | string | no | all | Backend partition; in MCP can be array | | prev | string | no | — | Comma-separated previous queries (conversation context) | | decontextualized_query | string | no | — | Pre-resolved query; skips server-side decontextualization | | streaming | bool | no | true | "0" / "false" / "False" disables | | query_id | string | no | auto | Echoed in response | | mode | enum | no | list | list | summarize | generate | | scorer | string | no | default | e.g., nlwebscorer for the neural reranker | | itemType | string | no | — | Schema.org type hint (skip type detection) | | response_format | string | no | — | v0.55 structured-body field |

v0.55 Structured POST Body

The newer body format groups fields:

{
  "query": { "text": "your question" },
  "context": { "prev": ["previous q1", "previous q2"] },
  "prefer": {
    "mode": "list",
    "streaming": true,
    "response_format": "schema"
  },
  "meta": { "version": "0.55" }
}

Verify the exact field names against the live docs before relying on this — fields are still settling.

Streaming Response Format (SSE)

NLWeb uses Server-Sent Events when streaming=true (the default):

Content-Type: text/event-stream
Cache-Control: no-cache
Connection: keep-alive
X-Accel-Buffering: no

Each chunk is:

data: <json>\n\n

The <json> is one of:

• A message object (header-like): {"message_type": "license", "content": {...}}
• A partial result: {"results": [...]} (results may arrive incrementally as FastTrack streams)
• A terminal object: {"query_id": "...", "complete": true} (exact field — verify live)

In-Stream "Headers" (NLWS Mechanism)

NLWeb's "headers" are NOT HTTP response headers — they are JSON objects in the SSE stream with a message_type discriminator. Known types:

| message_type | Purpose | |--------------|---------| | license | Content license terms | | data_retention | How long the agent may cache results | | cache_policy | Caching directives | | ui_component | Optional rendering hint | | usage_terms | Acceptable use | | rate_limits | Calls/sec / day budget | | data_freshness | When the underlying data was last indexed | | api_version | Server's NLWeb version |

Client parsing rule: don't assume results is the first chunk. Buffer message objects until you see the result stream or a terminal marker.

Non-Streaming Response

With streaming=false, the server returns a single application/json body:

{
  "query_id": "abc-123",
  "messages": [{"message_type": "license", "content": {...}}, ...],
  "results": [
    {
      "url": "https://example.com/article/x",
      "name": "Article X",
      "site": "example",
      "score": 0.83,
      "description": "...",
      "schema_object": { "@type": "Article", "@context": "https://schema.org", ... }
    }
  ]
}

The schema_object is the original Schema.org JSON-LD that was indexed — this is what makes NLWeb results agent-actionable, not just text snippets.

Three Modes

| Mode | Behavior | Use case | |------|----------|----------| | list | Return ranked Schema.org results, no LLM synthesis | Agent does its own rendering / re-ranking | | summarize | LLM condenses top results into a short answer + still returns results | Conversational UIs | | generate | Full RAG — LLM synthesizes an answer grounded in results | Q&A endpoints |

Errors

For /ask, errors generally come back as 500 with a JSON envelope. For /mcp, errors use JSON-RPC 2.0:

{
  "jsonrpc": "2.0",
  "id": 1,
  "error": { "code": -32603, "message": "Internal error", "data": {...} }
}

Always check status code before parsing — partial SSE streams can drop with 200 followed by silence.

Implementation Guidance

Server-Side (extending the route)

If you need to extend /ask (e.g., add an auth check or custom param):

1. Locate webserver/routes/api.py
2. Add middleware in webserver/middleware/ rather than modifying the route directly — keeps you upgrade-safe
3. Forward to NLWebHandler (core/baseHandler.py) unchanged so the streaming + ranking pipeline still runs

Client-Side (calling /ask)

# Python streaming client (sketch — verify response shape against the live spec)
import httpx, json

async with httpx.AsyncClient() as client:
    async with client.stream("GET", "http://localhost:8000/ask",
                              params={"query": "best running shoes", "site": "shoes", "mode": "generate"}) as r:
        async for line in r.aiter_lines():
            if not line.startswith("data: "):
                continue
            obj = json.loads(line[6:])
            if "message_type" in obj:
                handle_header(obj)
            elif "results" in obj:
                handle_results(obj["results"])

When to use which mode

• Agent that re-ranks and selects on its own → mode=list
• Quick conversational answer with citations → mode=summarize
• Single synthesized answer (chatbot-style) → mode=generate

Debugging /ask

• Set streaming=false first — easier to inspect a single JSON body.
• Add decontextualized_query to bypass query rewriting and isolate ranking issues.
• Try mode=list to see the raw retrieval — if results are bad here, the problem is ingest/embeddings, not the LLM.
• Pass query_id and grep server logs for it.
• Disable tool_selection_enabled in config_nlweb.yaml to bypass the router and force straight retrieval.

Always verify the exact param names and message_type values against the live spec — they evolve.