sz-chrome-remote-cdp.git/src/domains/snapshot.ts, branch master

refactor: remove duplicate browser prompt instructions (#35)

2026-10-11T18:25:38+00:00

Remove browser instruction text repeated across tool descriptions, parameter schemas, tool guidelines and the global browser prompt. Tool names, schemas, handlers and concurrency are unchanged; shared recovery and ownership guidance is now available before and after the browser connects, and the prompt names which tools are serialized versus observations. Co-Authored-By: Claude Fable 5.1

refactor: helpers live in helper modules, not in tool files

2026-08-02T13:42:46+00:00

form.ts exported resolveAndCall and detailsOf to form-batch.ts, snapshot.ts exported buildTree and collectInteractiveTargets to ref-resolve.ts, and screenshot.ts exported captureWithCrosshair to click.ts — so importing one tool pulled in another tool's whole module. The shared halves move out: ax-tree.ts (the AX tree model, its construction and the queries over it), element-call.ts (calling a function on a resolved element, and the PageResult it returns), screenshot-capture.ts (capture, resize, crosshair overlay). Each tool file now holds its tools and its own presentation, nothing else. NavOutcome stops being exported; nothing outside navigate.ts read it. target.ts stays a module rather than folding into click.ts as planned: it is already helper-only and has its own test suite, and moving a tested pure function into a tool file is the thing this commit removes. Having one consumer is not the defect. Co-Authored-By: Claude Opus 5 (1M context)

refactor: one door from a tool to the browser

2026-08-02T13:42:46+00:00

Four routes coexisted: cdpCall, a raw client.session().call, a session held in a local and called directly, and client.evaluateJs — and five files mixed two or three of them. The CdpError-to-ToolErr mapping was hand-copied at nine sites, plus a private copy in isolated-tab.ts. domains/cdp-call.ts now owns all of it: cdpErrToToolErr, cdpCall, cdpCallOnTarget, cdpCallBrowser, evalJs. Every other file under src/domains goes through them, which also removes a pile of `if (!r.success) return err({ kind: "cdp_error", ... })` boilerplate in favour of returning the failed Result as-is. Two scanner rules keep it that way, scoped to src/domains/ so the layer that legitimately owns the session — cdp/ and client.ts — is unaffected. Scoping is new to the rule model; without it the rules fired on the transport itself. One deliberate behaviour change: a CDP timeout now surfaces as kind "timeout" rather than "cdp_error". isolated-tab.ts already did this and the other 13 sites flattened it; the shared mapper uses the honest label rather than preserving the mislabel. Co-Authored-By: Claude Opus 5 (1M context)

refactor(domains): strip comment noise

2026-08-02T13:42:46+00:00

Removes module banners, step markers, and comments restating the code. Keeps one-line notes for safeJs's trust boundary, the select-match null sentinels, the TUI width clamp, the single-window tab rule, the input synthesis quirks (char events, keyCode, mousemove, focus, compositor), the stale AX value, ignored-wrapper recursion, and isolated-tab routing. Co-Authored-By: Claude Opus 5 (1M context)

refactor: close the remaining unguarded type assertions

2026-08-02T13:42:46+00:00

Types the event-queue iterator's TReturn as undefined so the terminal result needs no cast; declares the isError field the pi runtime reads by duck-typing; binds socket to a local const in the daemon transport; replaces the @ts-ignore on the optional sharp import with a non-literal specifier so src/ still typechecks where sharp was not installed. Adds src/util/guards.ts as the single home for narrowing over unknown — isRecord plus scalar/array readers — and converts the two private isObject copies in cdp/types.ts and daemon/protocol.ts along with every inline `as Record` in domains/js.ts and domains/render.ts. Gives Accessibility.getFullAXTree a real result schema, closing the last unvalidated CDP boundary: only nodeId is required, every object carries additionalProperties, and each AX value stays Unknown, so the tree decodes as a whole or fails as a whole rather than being filtered. Guards the five tool-result `details` readers, the daemon attach response, the persisted session state, the browser-namespace flag, and the fs errno narrowing, each feeding an already-present failure path. The daemon-transport socket binding is not a bug fix: send() runs synchronously inside sendWithTimeout's promise executor, so socket cannot be reassigned between the check and the write. It removes the assertion and makes the write immune to a later refactor that adds an await. Two assertions survive by design and are documented in the task report: the never->unknown re-widening in util/tool.ts, which the type-erased tool registry makes unavoidable, and the AsyncFunction constructor lookup in domains/js.ts, which JavaScript exposes no typed binding for. Co-Authored-By: Claude Opus 5 (1M context)

refactor(domains): share target resolution and CDP error mapping

2026-08-02T13:42:46+00:00

The ref-vs-x/y branch turned out to exist in one tool, not four; it is now one helper with tests covering the half-specified-coordinate and stale-ref cases. cdpCall folds the repeated cdp_error mapping at 24 active-session call sites; target-scoped callOnTarget/callBrowser sites are left alone. Co-Authored-By: Claude Opus 5 (1M context)

refactor(tools): declare concurrency in the tool definition

2026-08-02T13:42:46+00:00

serialized: true was applied in registry.ts, away from the tools it describes. Now a required concurrency field on each definition, with a test pinning the serialized set so the classes cannot drift. The lock check defaults to taking the mutex for any class that is not explicitly "parallel", so a definition that bypasses defineBrowserTool at runtime fails safe rather than racing on shared CDP state. The test script gains --conditions=import: pi-coding-agent publishes no "require" condition, so importing any tool definition from a CJS-loaded test could not resolve it. Co-Authored-By: Claude Opus 5 (1M context)

refactor(domains): drop CDP casts, share the box-model unwrap

2026-08-02T13:42:46+00:00

ref-resolve.ts and snapshot.ts had the same box-model unwrap, both using data.model! to work around an untyped response. Extracted to domains/box.ts; the typed schema makes the assertions unnecessary. Both derived the centre identically (quad corner + half the model width/height), so centreOf keeps that derivation; the shared zero-size rejection moved into boxOf, and snapshot keeps its own off-viewport (x/y < 0) filter. evalInIsolatedTab now goes through evaluateJson with the tab's own sessionId and a caller-supplied guard; both callers evaluate expressions ending in JSON.stringify, so they pass isJsonText rather than a check that checks nothing. Runtime.evaluate/callFunctionOn result casts in form.ts, forms.ts and keyboard.ts are replaced by named guards over the in-page result shapes. Runtime.callFunctionOn's params schema gained arguments/returnByValue/ awaitPromise/userGesture: four domain call sites legitimately pass them and typebox's additionalProperties:true does not widen the static type, so the params type was rejecting valid CDP fields. Params schemas are type-only (only results are compiled), so this is a compile-time widening with no runtime effect. tsc is green again for both tsconfigs. Co-Authored-By: Claude Opus 5 (1M context)

feat: address elements by stable refs across re-renders (#8)

2026-06-28T18:32:54+00:00

Interaction tools could only target elements by raw viewport coordinates (browser_click) or model-guessed CSS selectors (browser_fill/focus/ select_option/upload). Neither survives a re-render: on React/SPA forms, every save or edit reflows the page, so captured @(x,y) coordinates go stale and guessed selectors miss — leaving the agent to fall back to hand-rolled execute_js DOM walks and unnoticed failed saves. Add a stable ref system, mirroring Playwright MCP / browser-use: - snapshot assigns each interactive element a ref (e1, e2, …), shown as [eN] in the outline, and publishes a per-tab ref → backendNodeId map - interaction tools accept an optional `ref` that re-resolves the element at call time (via DOM.resolveNode / getBoxModel), so it stays valid across re-renders; coordinates/selectors remain as fallbacks - mutating tools append a compact diff of changed / newly-appeared (*) / removed interactive elements, so silent failed saves surface immediately - stale refs return an actionable "re-snapshot" error instead of a miss Ref map is replaced per snapshot (bounded memory); fill keeps the native _valueTracker write path so React controlled inputs still update. Co-authored-by: Claude Opus 4.8

release: v0.4.0 — accessibility snapshot, network capture, snapshot-first prompts

2026-05-05T18:36:49+00:00

Adds two new tools and pivots the agent's default workflow away from screenshots toward structured page data. browser_snapshot — CDP accessibility tree (roles, names, states, hierarchy) with click coordinates @(x,y) attached to every interactive element via DOM.getBoxModel (parallel, 1.5s aggregate budget). Pass directly to browser_click — no screenshot round-trip. Optional includeScreenshot:true for visual confirmation. format:"json" returns a slim structure with full box per node. browser_network_requests — replaces the deprecated browser_get_network_log placeholder. Filters by urlPattern (substring or /regex/), method, status, resource type, recency. Optional response-body capture under a 5s aggregate budget with 50KB per-body cap. Buffer is page-scoped (cleared on tab switch), 500 records/tab, with a bufferOverflowed flag on drain. CDP plumbing: in-process aggregator in src/cdp/network-buffer.ts wired into the existing single-consumer event loop. Accessibility domain enabled alongside Page/DOM/Runtime/Network on every attach. Ctrl+O expand/collapse on three tools' renderResult — browser_snapshot, browser_network_requests, browser_execute_js. Compact-by-default views with keyHint("app.tools.expand") that respects user keybinding remaps. Tool prompts pivoted snapshot-first: browser_screenshot is documented as LAST RESORT for visual verification only; browser_snapshot is the default for understanding pages; browser_execute_js is the default for surgical reads. browser_click and browser_open_urls guidance follows. SKILL.md collapsed from ~342 to ~53 lines — keeping only what isn't already in tool prompts: the decision tree, connection rules, browser_run_script daemon bindings. Co-Authored-By: Claude Opus 4.7 (1M context)