diff options
| author | amankumarsingh77 <amankumarsingh7702@gmal.com> | 2026-05-06 00:06:49 +0530 |
|---|---|---|
| committer | amankumarsingh77 <amankumarsingh7702@gmal.com> | 2026-05-06 00:06:49 +0530 |
| commit | 589cd294931c01b3748779635d17a13dd240af19 (patch) | |
| tree | b635f9fa443cd120de905660fe0d2665184e9a8c /src/domains/click.ts | |
| parent | 20ceaa1c6ade4950ed7d16caa5269ad2f4805db4 (diff) | |
release: v0.4.0 — accessibility snapshot, network capture, snapshot-first prompts
Adds two new tools and pivots the agent's default workflow away from
screenshots toward structured page data.
browser_snapshot — CDP accessibility tree (roles, names, states, hierarchy)
with click coordinates @(x,y) attached to every interactive element via
DOM.getBoxModel (parallel, 1.5s aggregate budget). Pass directly to
browser_click — no screenshot round-trip. Optional includeScreenshot:true
for visual confirmation. format:"json" returns a slim structure with full
box per node.
browser_network_requests — replaces the deprecated browser_get_network_log
placeholder. Filters by urlPattern (substring or /regex/), method, status,
resource type, recency. Optional response-body capture under a 5s aggregate
budget with 50KB per-body cap. Buffer is page-scoped (cleared on tab switch),
500 records/tab, with a bufferOverflowed flag on drain.
CDP plumbing: in-process aggregator in src/cdp/network-buffer.ts wired into
the existing single-consumer event loop. Accessibility domain enabled
alongside Page/DOM/Runtime/Network on every attach.
Ctrl+O expand/collapse on three tools' renderResult — browser_snapshot,
browser_network_requests, browser_execute_js. Compact-by-default views with
keyHint("app.tools.expand") that respects user keybinding remaps.
Tool prompts pivoted snapshot-first: browser_screenshot is documented as
LAST RESORT for visual verification only; browser_snapshot is the default
for understanding pages; browser_execute_js is the default for surgical
reads. browser_click and browser_open_urls guidance follows.
SKILL.md collapsed from ~342 to ~53 lines — keeping only what isn't already
in tool prompts: the decision tree, connection rules, browser_run_script
daemon bindings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Diffstat (limited to 'src/domains/click.ts')
| -rw-r--r-- | src/domains/click.ts | 11 |
1 files changed, 6 insertions, 5 deletions
diff --git a/src/domains/click.ts b/src/domains/click.ts index c269dd8..9946664 100644 --- a/src/domains/click.ts +++ b/src/domains/click.ts @@ -41,14 +41,15 @@ export const clickTool = defineBrowserTool({ name: "browser_click", label: "Browser Click", description: - "Click at viewport CSS-pixel coordinates. Compositor-level click works through iframes, shadow DOM, and cross-origin content. Use browser_screenshot first to find coordinates.", + "Click at viewport CSS-pixel coordinates. Compositor-level click works through iframes, shadow DOM, and cross-origin content. Get coordinates from browser_snapshot's @(x,y) hints — do not screenshot to find click targets.", promptSnippet: "Click at pixel coordinates (x, y) on the page", promptGuidelines: [ - "Use browser_click for all clicks. Coordinates are viewport CSS pixels (not device pixels).", - "Capture a browser_screenshot BEFORE clicking to find the right coordinates.", - "Capture another browser_screenshot AFTER clicking to verify the action worked.", + "Get (x, y) from browser_snapshot — the outline shows '@(x,y)' for every interactive element. NO screenshot needed.", + "If the target isn't an interactive role in the snapshot, fall back to browser_execute_js with `el.getBoundingClientRect()` — still NO screenshot.", + "Coordinates are viewport CSS pixels (not device pixels).", + "After clicking, verify with browser_snapshot (or browser_execute_js for a specific value). Only screenshot if you need to confirm visual rendering.", "Compositor-level clicks pass through iframes, shadow DOM, and cross-origin content — no selector needed.", - "If a click doesn't register, try setting BH_DEBUG_CLICKS=1 to get annotated screenshots showing exact click positions.", + "If a click doesn't register, set BH_DEBUG_CLICKS=1 to get annotated screenshots showing exact click positions (debug only).", "For React/Vue components that don't respond to clicks, try browser_dispatch_key to send DOM-level events.", ], parameters: ClickArgs, |
