aipexstudio/aipex-browser

AIPex Browser Control

AIPex is a Chrome extension that exposes 30+ browser automation tools over the Model Context Protocol (MCP). Once connected, the agent can control any Chrome tab using natural language — clicking, typing, navigating, capturing screenshots, downloading content, and more.

Architecture:

Agent (MCP client) ──stdio──▶ aipex-mcp-bridge ──WebSocket──▶ AIPex Chrome Extension ──▶ Browser APIs

---

When to Use This Skill

Use this skill when the user wants to:

• Navigate to URLs, click links, fill forms, or interact with any web page
• Automate multi-step browser workflows
• Extract or download data from web pages
• Capture screenshots of browser tabs
• Manage multiple tabs across browser windows
• Perform browser-assisted testing (accessibility, UX, regression)

---

Prerequisites

• AIPex Chrome extension installed (available on the Chrome Web Store or via developer build)
• Node.js >= 18 installed on the local machine

The user is assumed to have AIPex installed. The agent only needs to complete the two connection steps below.

---

Step 1: Register the MCP Server

Add the following to the agent's MCP configuration. No manual installation is needed — npx downloads and runs aipex-mcp-bridge automatically.

Cursor (.cursor/mcp.json)

{
  "mcpServers": {
    "aipex-browser": {
      "command": "npx",
      "args": ["-y", "aipex-mcp-bridge"]
    }
  }
}

Claude Desktop (claude_desktop_config.json)

{
  "mcpServers": {
    "aipex-browser": {
      "command": "npx",
      "args": ["-y", "aipex-mcp-bridge"]
    }
  }
}

Claude Code (CLI)

claude mcp add aipex-browser -- npx -y aipex-mcp-bridge

VS Code Copilot (.vscode/mcp.json)

{
  "servers": {
    "aipex-browser": {
      "command": "npx",
      "args": ["-y", "aipex-mcp-bridge"]
    }
  }
}

Windsurf (mcp_config.json)

{
  "mcpServers": {
    "aipex-browser": {
      "command": "npx",
      "args": ["-y", "aipex-mcp-bridge"]
    }
  }
}

Custom port (optional)

The bridge listens on localhost:9223 by default. To use a different port:

{
  "mcpServers": {
    "aipex-browser": {
      "command": "npx",
      "args": ["-y", "aipex-mcp-bridge", "--port", "9224"]
    }
  }
}

Then use ws://localhost:9224 in Step 2.

---

Step 2: Connect the AIPex Extension to the Bridge

After the MCP server is registered and running:

1. Open Chrome and click the AIPex extension icon
2. Go to Options (or right-click the icon → "Extension options")
3. Find the WebSocket Connection section
4. Enter: ws://localhost:9223
5. Click Connect

The bridge and extension will handshake, and all browser tools will become available to the agent.

Verifying the connection: If only a single tool called check_aipex_connection is visible, the extension has not yet connected. Follow Step 2 again, then reload the MCP server in agent settings.

---

Tool Usage Strategy (IMPORTANT)

Always follow this priority order to minimize token cost and latency:

Priority 1 — search_elements (always try first)

Query the page's accessibility tree to find elements and get their UIDs. Fast, cheap, requires no screenshot.

search_elements(tabId, "{button,input,textarea,select,a}*")

Priority 2 — UID-based interaction (preferred)

Use UIDs returned by search_elements to interact directly:

• click(tabId, uid) — click any element
• fill_element_by_uid(tabId, uid, value) — type into inputs
• hover_element_by_uid(tabId, uid) — reveal menus or tooltips

Priority 3 — capture_screenshot + computer (high-cost fallback only)

Use only when search_elements fails after two different query attempts, or when pixel-level interaction is required (canvas, drag-and-drop, sliders).

1. capture_screenshot(sendToLLM=true) — see the page
2. computer(action, coordinate) — click/type at pixel coordinates

Standard Workflow

get_all_tabs()
  → search_elements(tabId, "<pattern>")
  → click(tabId, uid)  OR  fill_element_by_uid(tabId, uid, value)
  → [capture_screenshot(sendToLLM=true) to verify if needed]

---

Available Tool Categories

| Category | Tools | Description | |---|---|---| | Tab Management | 8 tools | Open, close, switch, pin, group tabs | | UI Interaction | 7 tools | Click, fill, hover, keyboard, coordinate-based | | Page Content | 4 tools | Metadata, scroll, highlight elements/text | | Screenshots | 2 tools | Capture visible tab or specific tab | | Downloads | 3 tools | Save text as markdown, download images | | Human Intervention | 4 tools | Request user input mid-automation |

Key tools by category:

| Category | Key Tools | |---|---| | Tab | get_all_tabs, switch_to_tab, create_new_tab, close_tab | | UI | search_elements, click, fill_element_by_uid, computer | | Page | get_page_metadata, scroll_to_element, highlight_element | | Screenshot | capture_screenshot, capture_tab_screenshot | | Download | download_text_as_markdown, download_image | | Intervention | request_intervention, list_interventions |

To load complete parameter schemas and examples for every tool:

read_skill_reference("aipex-browser", "references/tools-reference.md")

---

Common Patterns

Navigate to a URL and click a button

create_new_tab("https://example.com")
→ search_elements(tabId, "*[Ss]ubmit*")
→ click(tabId, uid)

Fill a login form

get_all_tabs()
→ search_elements(tabId, "{input,textbox}*")
→ fill_element_by_uid(tabId, emailUid, "[email protected]")
→ fill_element_by_uid(tabId, passwordUid, "secret")
→ search_elements(tabId, "*[Ll]ogin*")
→ click(tabId, uid)

Extract page content to markdown

get_page_metadata()
→ download_text_as_markdown(content, "page-extract")

Visual verification

capture_screenshot(sendToLLM=true)

---

Troubleshooting

| Symptom | Likely Cause | Fix | |---|---|---| | Only check_aipex_connection visible | Extension not connected to bridge | Open AIPex Options → set WebSocket URL → Connect | | Port 9223 already in use | Port conflict on machine | Use --port 9224 in MCP config and ws://localhost:9224 in extension | | search_elements returns 0 results | Page uses canvas or non-semantic HTML | Fall back to capture_screenshot(sendToLLM=true) + computer tool | | Connection drops frequently | Service worker sleep cycle | AIPex uses keepalive pings; reconnect extension from Options if needed | | Tools appear but calls time out | Bridge not receiving WebSocket messages | Restart bridge: reload MCP server in agent settings |