neuroskill-com/neuroskill-llm

NeuroSkill LLM — On-Device Inference Server

Control the built-in on-device LLM inference server (OpenAI-compatible, powered by llama.cpp). All subcommands route to the WebSocket/HTTP API.

---

Subcommands

| Subcommand | Description | |---|---| | llm status | Server state (stopped/loading/running), model name, context window. Context size is auto-recommended based on model size and available GPU/RAM (minimum 8192 tokens, configurable in Settings → LLM → Context Size) | | llm start | Load the active model and start the inference server | | llm stop | Stop the server and free GPU/CPU memory | | llm catalog | List all GGUF models with download states and active selections | | llm add <repo> <filename> | Add an external HF model to the catalog and download it | | llm add <hf-url> | Add from a full HuggingFace URL | | llm add ... --mmproj <file> | Also add and download a vision projector from the same repo | | llm select <filename> | Set the active text model | | llm mmproj <filename\|none> | Set the active vision projector (or none to disable) | | llm autoload-mmproj <on\|off> | Toggle auto-loading of vision projector on start | | llm download <filename> | Download a model (fire-and-forget; poll catalog for progress) | | llm pause <filename> | Pause an in-progress download | | llm resume <filename> | Resume a paused download | | llm cancel <filename> | Cancel an in-progress download | | llm delete <filename> | Delete a locally-cached model file | | llm downloads | List all downloads with status and progress | | llm refresh | Re-probe the HF Hub cache for externally downloaded models | | llm fit | Check which models fit in available RAM/VRAM | | llm logs | Print the last 500 LLM server log lines | | llm chat | Interactive multi-turn REPL — type exit to quit (WebSocket only) | | llm chat "message" | Single-shot: send one message, stream the reply, and exit |

---

CLI Examples

# Server lifecycle
npx neuroskill llm status
npx neuroskill llm start
npx neuroskill llm stop

# Model management
npx neuroskill llm catalog
npx neuroskill llm catalog --json | jq '.entries[] | select(.state == "downloaded")'
npx neuroskill llm add bartowski/Phi-4-mini-reasoning-GGUF Phi-4-mini-reasoning-Q4_K_M.gguf
npx neuroskill llm add bartowski/Phi-4-mini-reasoning-GGUF Phi-4-mini-reasoning-Q4_K_M.gguf --mmproj mmproj-Phi-4-mini-reasoning-BF16.gguf
npx neuroskill llm select "Qwen_Qwen3.5-4B-Q4_K_M.gguf"
npx neuroskill llm mmproj "mmproj-Qwen_Qwen3.5-4B-BF16.gguf"
npx neuroskill llm mmproj none
npx neuroskill llm download "Qwen3-1.7B-Q4_K_M.gguf"
npx neuroskill llm delete "Qwen3-1.7B-Q4_K_M.gguf"
npx neuroskill llm fit

# Interactive chat REPL
npx neuroskill llm chat
npx neuroskill llm chat --system "You are a concise EEG assistant."
npx neuroskill llm chat --temperature 0.3 --max-tokens 512

# Single-shot chat
npx neuroskill llm chat "What EEG frequency bands are associated with meditation?"
npx neuroskill llm chat "Explain delta waves" --temperature 0.3 --max-tokens 256

# Image input (requires vision model + mmproj)
npx neuroskill llm chat "What do you see?" --image screenshot.png
npx neuroskill llm chat "Compare these" --image session1.png --image session2.png

Interactive REPL Commands

| Command | Effect | |---|---| | /clear | Clear conversation history and start a new session (system prompt is kept) | | /history | Print all messages in the current conversation | | /image <path> | Stage an image file for the next message (can repeat for multiple images) | | /images | Show count of currently staged images | | /help | Show REPL command help | | exit or quit | End the session |

Image staging example inside the REPL:

You: /image eeg_plot.png
  ✓ image staged: eeg_plot.png (1 pending)
You: What anomalies do you see in this EEG trace?

---

WebSocket Streaming Protocol — llm_chat

llm_chat returns multiple frames per request. Tokens stream as delta frames; generation ends with done or error.

ws.send(JSON.stringify({
  command:  "llm_chat",
  messages: [
    { role: "system", content: "You are a concise EEG assistant." },
    { role: "user",   content: "What does high theta power indicate?" },
  ],
  // Optional: session_id: 42  (continue existing conversation)
}));

// Short-hand for single user message:
ws.send(JSON.stringify({ command: "llm_chat", message: "Hello!" }));

Server Frames

// Session frame (first):
{ "command": "llm_chat", "type": "session", "session_id": 42 }

// Delta frames (one per token batch):
{ "command": "llm_chat", "type": "delta", "text": "High theta" }

// Final done frame:
{
  "command": "llm_chat", "ok": true, "type": "done",
  "finish_reason": "stop", "prompt_tokens": 42,
  "completion_tokens": 87, "n_ctx": 16384, "session_id": 42
}

---

HTTP REST API

# Status / start / stop
curl -s http://127.0.0.1:8375/llm/status | jq '{status, model_name, n_ctx}'
curl -s -X POST http://127.0.0.1:8375/llm/start
curl -s -X POST http://127.0.0.1:8375/llm/stop

# Catalog
curl -s http://127.0.0.1:8375/llm/catalog | jq '.entries[] | select(.state == "downloaded") | .filename'

# Non-streaming chat
curl -s -X POST http://127.0.0.1:8375/llm/chat \
  -H "Content-Type: application/json" \
  -d '{"message":"What is EEG coherence?"}' | jq '{text, finish_reason, session_id}'

# Continue existing session
curl -s -X POST http://127.0.0.1:8375/llm/chat \
  -H "Content-Type: application/json" \
  -d '{"message":"Can you elaborate?","session_id":42}'

# With system prompt and GenParams
curl -s -X POST http://127.0.0.1:8375/llm/chat \
  -H "Content-Type: application/json" \
  -d '{"message":"Summarize in one line.","system":"Be extremely brief.","temperature":0.3}'

# Image input (base64 data-URL)
curl -s -X POST http://127.0.0.1:8375/llm/chat \
  -H "Content-Type: application/json" \
  -d '{"message":"What do you see?","images":["data:image/png;base64,iVBORw0KGgo..."]}'

# Logs
curl -s http://127.0.0.1:8375/llm/logs | jq '.logs[-10:]'

---

WebSocket Commands

| Command | Required | Optional | Description | |---|---|---|---| | llm_status | — | — | Server state + model info | | llm_start | — | — | Start inference server | | llm_stop | — | — | Stop server + free resources | | llm_catalog | — | — | Full model catalog | | llm_add_model | repo, filename | download, mmproj | Add external HF model | | llm_select_model | filename | — | Set active text model | | llm_select_mmproj | filename | — | Set active vision projector | | llm_download | filename | — | Start model download | | llm_pause_download | filename | — | Pause download | | llm_resume_download | filename | — | Resume download | | llm_cancel_download | filename | — | Cancel download | | llm_delete | filename | — | Delete cached model | | llm_downloads | — | — | List downloads with progress | | llm_refresh_catalog | — | — | Re-probe HF cache | | llm_hardware_fit | — | — | Check model fit in RAM/VRAM | | llm_logs | — | — | Last 500 log entries | | llm_chat | messages or message | GenParams | Streaming chat |

GenParams Reference

| Field | Type | Default | Description | |---|---|---|---| | temperature | float | 0.8 | Sampling temperature (0 = deterministic) | | top_k | int | 40 | Top-K sampling | | top_p | float | 0.9 | Nucleus sampling threshold | | repeat_penalty | float | 1.1 | Repetition penalty | | seed | uint | 0xDEADBEEF | RNG seed for reproducible output | | max_tokens | uint | 2048 | Maximum tokens to generate | | thinking_budget | uint/null | 512 | Max tokens in <think> block (0 = skip, null = unlimited) |

---

Built-in Tool Calling

When the LLM server is running, llm_chat supports automatic tool calling — the model can invoke built-in tools and return results within the conversation.

| Tool | Description | |---|---| | date | Get current date/time metadata | | location | Approximate public-IP geolocation | | bash | Execute shell commands | | read_file | Read file contents from disk | | write_file | Write / create files | | edit_file | Surgical find-and-replace edits | | search_output | Navigate large tool outputs (paginated) | | web_search | Search the web | | web_fetch | Fetch a URL and return its content | | skill | Query the NeuroSkill EEG API (status, sessions, labels, search, screenshots, hooks, DND, health, etc.) |

Tool calls are detected in the model's output, executed server-side, and results are fed back automatically. The CLI llm chat handles this transparently.

The skill Tool

The skill tool lets the LLM query and control NeuroSkill™ directly. Commands are sent as { "command": "<ws_command>", "args": { ... } } objects. Supported commands include: status, sessions, session_metrics, search, compare, sleep, sleep_schedule, sleep_schedule_set, label, search_labels, interactive_search, search_screenshots, screenshots_around, screenshots_for_eeg, eeg_for_screenshots, search_screenshots_vision, search_screenshots_by_image_b64, hooks_status, hooks_get, hooks_suggest, hooks_log, dnd, notify, say, health_summary, health_query, health_metric_types, health_sync, and others.

Argument coercion: If the LLM flattens arguments to the top level (e.g. {"command":"search_screenshots","query":"today"} instead of properly nesting in args), the tool system automatically wraps stray properties into the args object before validation.

Blocked commands: For safety, the LLM cannot invoke self-management commands: llm_start, llm_stop, llm_select_model, llm_select_mmproj, llm_add_model, llm_download, llm_cancel_download, llm_pause_download, llm_resume_download, llm_refresh_catalog, llm_logs, llm_delete, run_calibration, timer.

Status formatting: When the LLM calls the status command, the result is automatically formatted from raw JSON into a human-readable text block with clear section headers (Device, Session, EEG Embeddings, Labels, Most Used Apps, Screenshots, Signal Quality, Current Scores, Hooks, Sleep, Recording History) for easier comprehension.

---

Chat Persistence

All conversations are automatically persisted to SQLite (~/.skill/chats/chat_history.sqlite):

• WebSocket / HTTP chats appear in the Chat window sidebar.
• The server returns a session_id for multi-turn persistence across connections.
• New sessions are auto-titled from the first user message.
• Tool calls executed during conversation are also persisted.

---

Server Status Values

| status | Meaning | |---|---| | "stopped" | No model loaded | | "loading" | Model being loaded from disk | | "running" | Model ready; chat and /v1/* endpoints are live |

Download State Values

| state | Meaning | |---|---| | "not_downloaded" | Not present locally | | "downloading" | In progress; check progress (0.0–1.0) | | "downloaded" | Cached locally; ready to use | | "cancelled" | Download was cancelled | | "failed" | Download failed; status_msg has details |

The LLM server also exposes an OpenAI-compatible HTTP API at /v1/chat/completions, /v1/completions, and /v1/embeddings. Use any OpenAI client library by pointing it at http://127.0.0.1:<port>.