fruffel/reflex-browser

Reflex Browser Skill

Purpose

Use this skill when the task involves a website or browser: navigating pages, clicking elements, reading content, filling forms, scraping data, or automating a web flow.

This skill covers interactive browser work via reflex browser ... CLI commands. Each command is a single stateless action — run one, inspect the JSON response, then decide the next step.

When the end goal is a reusable Lua or Python browser automation script, still start here first: use reflex browser ... to inspect the site, verify the flow, and stabilize selectors before writing script code. Then switch to the reflex-scripting skill for the script itself.

When To Use This Skill

• User asks to interact with a website (navigate, click, fill, read, scrape)
• User asks to inspect or explore a page
• User asks for browser-based data collection
• User asks whether a page, modal, overlay, or layout looks correct
• User asks for a browser automation script (start here for discovery, then reflex-scripting for the script)

When NOT To Use This Skill

• One-off file manipulation (xlsx, csv, json) without a browser → use reflex-scripting
• REST API calls → use reflex-scripting
• Data transforms or batch processing without a browser → use reflex-scripting
• Writing a script after browser discovery is done → use reflex-scripting

Quick Start

1. Install globally from Gitea npm:
   • npm config set @reflexautomation:registry "https://git.bqa-solutions.nl/api/packages/reflex/npm/" --location=user
   • npm config set "//git.bqa-solutions.nl/api/packages/reflex/npm/:_authToken" "YOUR_DEPLOY_TOKEN" --location=user
   • npm install -g @reflexautomation/reflex-cli
2. Start or reuse a session:
   • reflex browser start
3. Default to auto-session mode for normal agent runs:
   • do not pass --session on follow-up commands
   • example: reflex browser open https://example.com
4. Use explicit --session only for deterministic override:
   • example: reflex browser open https://example.com --session <sessionId>
5. End with reflex browser session-kill in auto-session mode.
   • if you used an explicit session id, use reflex browser session-kill --session <sessionId> (or reflex browser session-kill <sessionId>)

Use the CLI itself as the command/protocol source of truth:

• reflex browser --help
• reflex browser help --json

Agent Defaults

• Do not pass --session by habit after start; normal flows should stay in scoped auto-session mode.
• Start on the target page, not on broad capability audits; do not inventory libraries or call repeated help commands unless a concrete next step needs exact syntax.
• Prefer summary first and fine-tune it (-i -c, then -s, -C, -d) for selector discovery and recovery before trying eval or html; treat html as last-resort evidence, not normal recovery. Start with count 20; increase only when the page has many repeated items.
• Use summary --state --screenshot when selector recovery or page review needs visual evidence; keep screenshots opt-in instead of adding them to every pass.
• Use reflex browser help --json for exact command/response/schema details instead of duplicating those rules here.

Discovery Before Capability Audits (Required)

1. For normal tasks, start with the target page: start -> open -> summary.
2. Do not begin with broad audits such as reflex lua libs, repeated help calls, external doc hunts, or speculative output-format work.
3. Check exact command/library help only when you have a concrete next step that requires it.
4. Once a browser command succeeds, continue the task instead of re-checking availability.

Agent Availability And Recovery (Required)

1. Before assuming browser commands are broken, verify the local agent is available.
2. If a browser command fails with a transport/connect error, use this recovery order:
   • reflex agent status
   • if not running, reflex agent start
   • if the agent jar is missing, reflex agent download
   • if Java is missing or incompatible, reflex agent runtime install
   • retry the original browser command after recovery
3. Prefer the CLI's own guidance text when it suggests reflex agent start or reflex agent download.
4. For local development tasks, the AI agent may run reflex agent ... commands directly instead of only telling the human what to do.
5. If the environment is clearly non-local, remote, or permission-sensitive, explain the exact reflex agent ... command the human should run.
6. Do not start multiple background agents on different ports unless the task explicitly requires it.

Recommended recovery flow:

reflex agent status
reflex agent start
reflex agent download
reflex agent runtime install

Command Lifecycle (Required)

1. Send actions as separate CLI invocations.
2. Do not chain browser commands with &&, long shell pipelines, or opaque wrappers.
3. Keep one logical session context for the task (auto-session by default).
4. Default rule: omit --session on commands unless explicit override is required.
5. Execute sequentially: run one command, inspect response, then run the next.
6. On transport failure, recover the local agent first, then rerun the command as a new invocation.
7. Cleanup with reflex browser session-kill when done unless the user explicitly asks to keep the session open.

Response Handling (Required)

1. Always capture the full JSON response first, then parse fields from that captured payload.
2. Do not pipe command output directly into filtering (jq | head) as the only observable output; this hides critical context.
3. Prefer in-memory capture over temp files; if temp files are needed, use ephemeral files and clean them up.
4. Treat envelope fields (ok, action, session) as source-of-truth; mirrored duplicates may be compacted out of response.
5. Treat response.state and response.artifacts as optional add-ons when explicitly requested; existing summary parsers can ignore them.
6. When screenshots are requested, inspect them as visual evidence alongside response.data.summary.targets[]; do not replace the parser contract with screenshot-only reasoning when actionable selectors are needed.
7. For exact field names such as summary.targets[], use reflex browser help --json as the contract reference.

Parser-First Summary Contract (Required)

1. For selector discovery that feeds agent logic, use summary first.
2. Parse response.data.summary.targets[] as the primary selector feed:
   • selector
   • selectorType
   • confidence
   • reason
   • optional steering:
     • status
     • hint
     • fallback
     • ref — short-lived element ref (@r1, @r2, …); prefer over selector when present
3. Treat summary.version + summary.targets[] as the stable parser contract; avoid parsing deep fields from full summary output by default.
4. Do not parse elements/actions/candidates-style fields from summary output.
5. Keep extraction flow deterministic:
   • summary for candidate selection
   • targeted action (click/text/attribute/etc.)
   • re-run summary only after DOM-changing actions when needed

Summary Count And Flags (Required)

The positional count argument controls how many targets (ranked actionable elements) and snapshot lines are returned. It does not control how much of the DOM is scanned — the agent always scans up to 2000 nodes regardless.

• Default: 20. Hard max: 100.
• Start with summary 20 -i -c. This is enough for most pages.
• If you need more coverage, refine with flags first (-s, -C, -d), not by increasing the count.
• Only increase the count when the page has many similar actionable items (e.g., a product listing with 50+ links) and you need targets for more of them.
• Do not use high counts like summary 80 as a general "show me everything" — it adds low-confidence targets and more snapshot noise without improving discovery quality.

Summary Refinement Ladder (Required)

When the first summary pass is weak, noisy, or incomplete, stay in summary mode and refine it before escalating:

1. Start with summary 20 -i -c for normal interactive discovery.
2. If the results are too broad, add -s "<container>" to scope discovery to the relevant region.
3. If the page relies on non-semantic clickable UI, add -C.
4. If the output is still noisy, tune depth (-d <n>) and scope (-s) instead of increasing count or switching tools.
5. Only increase count beyond 20 when you know the page has many repeated items and you need targets for more of them.
6. Re-run summary after DOM-changing actions and continue from fresh refs/targets.
7. Use html only after at least 2 materially different summary passes still fail to expose enough structure for a targeted click/text/attribute/property call.
8. After consulting html, immediately switch back to targeted commands; do not keep planning from raw HTML dumps.

Visual Evidence And Screenshots (Required)

1. When the user asks for visual judgment such as "does this look good", "is this aligned", "is something covering it", or "what is visible here", capture screenshot evidence before answering.
2. Use reflex browser screenshot for pure visual review when no selector discovery is needed.
3. Use reflex browser summary 20 -i -c --state --screenshot when the task needs both selector recovery and visual confirmation.
4. Prefer screenshot-assisted summary when the page is visually tricky:
   • canvas-like or game-like UIs
   • overlays, sticky headers, consent/chat widgets, or occlusion problems
   • duplicated labels where DOM hints alone are ambiguous
   • non-semantic click targets where visible placement matters
5. Keep screenshots opt-in. If selectors are already clear from normal summary, do not add screenshot payloads on every pass.
6. Use the screenshot to confirm which visible element matches a candidate ref/selector or to explain why something is off-screen or obscured; keep summary.targets[] as the action feed.

Overlay And Noise Recovery (Required)

When summary only surfaces cookie/consent/chat widgets or other overlays:

1. Treat that output as page-state information, not discovery failure.
2. Dismiss or accept the blocking UI with the returned refs/selectors when safe and necessary.
3. Re-run summary immediately after the overlay changes the DOM.
4. Only escalate to html or other last-resort evidence after overlay cleanup plus the normal summary refinement ladder still fails.

Summary Refs (Required)

Each summary call assigns short-lived refs (@r1, @r2, …) to interactive elements:

• Refs appear in targets[].ref and inline in snapshot lines as [ref=@rN].
• Pass a ref directly as the selector to any action: click "@r10", fill "@r5" "text", attribute "@r3" href.
• Prefer refs over CSS/XPath selectors when available — they resolve directly to the live DOM element, bypassing selector fragility entirely.
• Refs are invalidated after any navigation or DOM-mutating action (click, fill, type, enter, tab, open, back, forward, refresh, tab switches). Re-run summary after such actions to obtain fresh refs.
• A stale ref returns errorCode: ELEMENT_STALE with a recoveryHint — always re-run summary on that error.

# summary returns: - link "Fiction" [ref=@r10]
reflex browser click "@r10"      # navigate using ref

# after navigation, refs are invalidated — re-run summary
reflex browser summary 20 -i -c
# - link "Soumission" [ref=@r150]
reflex browser attribute "@r150" title   # read without re-discovering selector
reflex browser attribute "@r150" href

Helper script:

• scripts/capture_json.sh provides rb_capture, rb_jq, and rb_pick_selector for safe full-response handling.
• source via source skills/reflex-browser/scripts/capture_json.sh (from project root).

Lua Trace Generation

The reflex browser lua command generates a Lua trace of the current browser session:

1. Only call reflex browser lua after the browser flow is already stable and validated.
2. Treat the output as a raw trace to refactor, not a finished script.
3. If response.data.generationGuidance is present, apply it when producing the script.
4. Keep discovered selectors as-is when guidance says selectors are fixed.
5. Do not use reflex browser lua to rescue an unstable discovery session; the trace will mirror dead-end navigation.
6. For the actual script authoring, switch to the reflex-scripting skill.

Session Selection Rules

1. Default to scoped auto-session mode.
2. Use start to get-or-create the scoped auto-session.
3. After start, continue commands without --session in normal flows.
4. Use explicit --session <id> only when:
   • the user asks to pin/reuse a specific session id, or
   • the task requires switching between multiple concurrent sessions.
5. Use bare --session (no value) only on start or open when a fresh backend-assigned session id is explicitly required.
6. Do not pass --session by habit in single-flow tasks.
7. Pass --profile only when persistent browser state is intentionally needed.
8. Set --engine selenium|playwright (aliases: sel, play) before bootstrapping when the task specifically needs one engine.

Hard Rules

1. Bridge supports Selenium and Playwright; use options.engine as the canonical engine field.
2. Do not send options.browser.
3. Recompute selectors after DOM changes with summary.
   • Use -i for interactive discovery.
   • Add -C for cursor-interactive components.
   • Add -c to reduce structural noise.
   • Add -s <selector> to scope discovery to a container.
4. Auto-session engine changes are applied by start/open, which recreate the same inferred auto-session id when needed.
5. Stop on first failed command (ok: false) to avoid cascading selector errors.
6. Pass relative links directly to open; CLI resolves them against current session URL.
7. For repeated-item extraction, anchor selectors at the collection parent (for example list/grid item), then index that parent; do not index unrelated descendants.
8. Treat repeated no such element or timeout on the same intent as a selector-state mismatch, not a transient flake.
9. When transport fails, recover via reflex agent status / reflex agent start before switching selector strategy.

Wait Strategy (Required)

1. click, fill, type, and open include waiting behavior; avoid redundant waits.
2. Use explicit wait for real state transitions:
   • after navigation
   • after back / forward / refresh
   • after async UI updates
3. Prefer stable page-level wait targets over fragile positional selectors.
4. If an action fails once, retry once. If it fails again, run summary again with tighter flags and continue with updated selectors.
5. After 2 consecutive failures for the same intent, stop the loop and run recovery; never keep incrementing positional selectors blindly.
6. For wait/visible/enabled/selected, pass per-check timeout as the positional argument (wait "<selector>" 8000); reserve global --cli-timeout for transport/command envelope timeout.

Anti-Patterns (Forbidden)

1. Running commands without checking each JSON response.
2. Using eval as default extraction when text, summary, attribute, or property can answer the task.
3. Running long blind command chains without validating page state.
4. Continuing extraction loops after a failed open.
5. Using positional selectors on the wrong structural level (for example article:nth-of-type(n) when siblings are actually li elements).
6. Repeating the same failing selector pattern across increasing indexes without re-discovery.
7. Jumping to full html dumps after the first weak summary instead of fine-tuning summary scope/flags first.
8. Starting extra sessions during the same task without explicit need and cleanup.
9. Hiding browser flow in long shell scripts/loops instead of observable one-command-at-a-time CLI calls.
10. Passing explicit --session by habit in single-flow tasks that should use default auto-session behavior.
11. Using --help mid-task as a substitute for the documented selector/session workflow.
12. Throwing away a validated browser flow in favor of direct HTTP or side-route fetching just because another layer exists.
13. Starting with broad capability audits (reflex lua libs, repeated help calls, external doc hunts) before trying start -> open -> summary on the target page.
14. Treating cookie/consent/chat-only summary output as permission to jump to eval, html, or external fetching instead of clearing the overlay and re-running summary.

Extending Beyond Browser

When browser commands alone are not enough to complete the task:

1. If the task just needs data export or file output after browser collection, use direct library commands (reflex xlsx ..., reflex csv ...) or reflex lua exec "..." — see the reflex-scripting skill.
2. If the user asks for a reusable script, switch to the reflex-scripting skill after browser discovery is done. Carry forward all discovered selectors, refs, URLs, and page transitions.
3. Do not discard a validated browser flow just because another layer also exists; scripts should continue the workflow, not replace it.

Output Contract

Use reflex browser help --json for the exact machine-readable response schema, including:

• summary target fields such as selector, ref, text, and href
• read-value paths such as response.data.value
• optional summary add-ons such as response.state and response.artifacts.screenshot
• error fields such as response.errorCode and response.recoveryHint
• Lua generation and screenshot payload fields

Repeated Items Pattern (Required)

1. Identify the repeated container first (for example css=ol.row > li).
2. Validate the container count/visibility with wait or visible.
3. Extract child fields within indexed container selectors (for example ... > li:nth-of-type(2) h3 a).
4. Prefer semantic field reads:
   • attribute ... title for titles when present
   • text ... .price_color for visible price text

Bash URL Handling (Optional)

Use this only when writing shell wrappers around CLI JSON output. For normal CLI usage, prefer open <href> directly (relative URLs are auto-resolved).

get_value() { jq -r '.response.data.value // empty'; }
get_url() { reflex url | get_value; }
open_target() {
  local href="$1"
  reflex browser open "$href" >/dev/null
  get_url
}

PowerShell URL Handling (Optional)

Use the same pattern in PowerShell when running on Windows.

function Get-Value {
  param([Parameter(ValueFromPipeline = $true)] $obj)
  process { $obj.response.data.value }
}

function Get-Url {
  reflex url | ConvertFrom-Json | Get-Value
}

function Open-Target {
  param([string]$Href)
  reflex browser open $Href | Out-Null
  Get-Url
}

References

• Command catalog and flags:
  • references/commands.md
• CLI protocol contract:
  • references/protocol.md
• Selector workflow:
  • references/selectors.md
• Session lifecycle and recovery:
  • references/session-management.md
• Worked examples:
  • examples/books-to-scrape-summary.md
  • examples/page-to-export-one-off.md
  • examples/careers-to-xlsx-lua.md