| Age | Commit message (Collapse) | Author |
|
Interaction tools could only target elements by raw viewport coordinates
(browser_click) or model-guessed CSS selectors (browser_fill/focus/
select_option/upload). Neither survives a re-render: on React/SPA forms,
every save or edit reflows the page, so captured @(x,y) coordinates go
stale and guessed selectors miss — leaving the agent to fall back to
hand-rolled execute_js DOM walks and unnoticed failed saves.
Add a stable ref system, mirroring Playwright MCP / browser-use:
- snapshot assigns each interactive element a ref (e1, e2, …), shown as
[eN] in the outline, and publishes a per-tab ref → backendNodeId map
- interaction tools accept an optional `ref` that re-resolves the element
at call time (via DOM.resolveNode / getBoxModel), so it stays valid
across re-renders; coordinates/selectors remain as fallbacks
- mutating tools append a compact diff of changed / newly-appeared (*) /
removed interactive elements, so silent failed saves surface immediately
- stale refs return an actionable "re-snapshot" error instead of a miss
Ref map is replaced per snapshot (bounded memory); fill keeps the native
_valueTracker write path so React controlled inputs still update.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
|
prompts
Adds two new tools and pivots the agent's default workflow away from
screenshots toward structured page data.
browser_snapshot — CDP accessibility tree (roles, names, states, hierarchy)
with click coordinates @(x,y) attached to every interactive element via
DOM.getBoxModel (parallel, 1.5s aggregate budget). Pass directly to
browser_click — no screenshot round-trip. Optional includeScreenshot:true
for visual confirmation. format:"json" returns a slim structure with full
box per node.
browser_network_requests — replaces the deprecated browser_get_network_log
placeholder. Filters by urlPattern (substring or /regex/), method, status,
resource type, recency. Optional response-body capture under a 5s aggregate
budget with 50KB per-body cap. Buffer is page-scoped (cleared on tab switch),
500 records/tab, with a bufferOverflowed flag on drain.
CDP plumbing: in-process aggregator in src/cdp/network-buffer.ts wired into
the existing single-consumer event loop. Accessibility domain enabled
alongside Page/DOM/Runtime/Network on every attach.
Ctrl+O expand/collapse on three tools' renderResult — browser_snapshot,
browser_network_requests, browser_execute_js. Compact-by-default views with
keyHint("app.tools.expand") that respects user keybinding remaps.
Tool prompts pivoted snapshot-first: browser_screenshot is documented as
LAST RESORT for visual verification only; browser_snapshot is the default
for understanding pages; browser_execute_js is the default for surgical
reads. browser_click and browser_open_urls guidance follows.
SKILL.md collapsed from ~342 to ~53 lines — keeping only what isn't already
in tool prompts: the decision tree, connection rules, browser_run_script
daemon bindings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|
crosshair overlay
|
|
Implements Task 9: adds src/domains/click.ts with the new architecture,
updates registry.ts to register clickTool, removes the legacy browser_click
block from tools.ts, and wires BrowserClient alongside BrowserDaemon in
index.ts so both systems coexist during migration.
Also fixes AnyBrowserToolDefinition variance: uses never-arg renderCall in
the erased type and a boundary cast in registerBrowserTool so concrete tool
definitions (with specific Static<S> args) are assignable to the registry
array without widening.
|