aboutsummaryrefslogtreecommitdiff
path: root/src/registry.ts
AgeCommit message (Collapse)Author
2026-06-29feat: address elements by stable refs across re-renders (#8)Aman Kumar
Interaction tools could only target elements by raw viewport coordinates (browser_click) or model-guessed CSS selectors (browser_fill/focus/ select_option/upload). Neither survives a re-render: on React/SPA forms, every save or edit reflows the page, so captured @(x,y) coordinates go stale and guessed selectors miss — leaving the agent to fall back to hand-rolled execute_js DOM walks and unnoticed failed saves. Add a stable ref system, mirroring Playwright MCP / browser-use: - snapshot assigns each interactive element a ref (e1, e2, …), shown as [eN] in the outline, and publishes a per-tab ref → backendNodeId map - interaction tools accept an optional `ref` that re-resolves the element at call time (via DOM.resolveNode / getBoxModel), so it stays valid across re-renders; coordinates/selectors remain as fallbacks - mutating tools append a compact diff of changed / newly-appeared (*) / removed interactive elements, so silent failed saves surface immediately - stale refs return an actionable "re-snapshot" error instead of a miss Ref map is replaced per snapshot (bounded memory); fill keeps the native _valueTracker write path so React controlled inputs still update. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21release: v0.6.0 — per-tab isolation, Unix socket daemon, on-demand init (#6)Aman Kumar
* feat: Unix socket daemon with auto-reconnect for persistent Chrome CDP Eliminates the 'Allow Remote Debugging' Chrome prompt that appeared on every pi session by multiplexing pi instances through a single persistent daemon process. Key additions: - Unix socket IPC (net.createServer + readline JSON framing) - CdpBridge with ID multiplexer + event router - DaemonTransport implementing CdpTransport interface - Auto-spawn with ensureDaemon() (lazy, checks socket) Fixes since initial implementation: - Bridge auto-reconnects to Chrome with exponential backoff (1s-60s). Previously the bridge tried once at startup and permanently failed if Chrome wasn't ready — causing 'Chrome not connected' on every browser tool call. - Daemon spawn uses project-local tsx binary instead of global --import flag, fixing silent spawn failure (ERR_MODULE_NOT_FOUND). - Stale ctx crash: persistState in onOwnershipChange wrapped in try/catch to prevent CDP event consumer crashes. 49 test assertions across 4 test files. Zero TypeScript errors. * feat: on-demand browser initialization Browser daemon no longer auto-starts on session_start. Instead: - session_start creates the client lazily but does NOT connect or spawn the daemon. Zero Chrome prompts for non-browser pi sessions. - Browser tool wrapper checks for daemon socket before each call. Socket missing → clear error: 'Run /browser-setup first'. Socket exists → proceeds silently (daemon already running). - /browser-setup is the explicit, one-time init point. It spawns the daemon and connects to Chrome only when the user opts in. - SKILL.md updated with Connection Setup instructions telling agents to direct users to /browser-setup when socket is missing. * fix: checkChromeRunning on macOS — match by substring not exact ps -A -o comm= returns full executable paths on modern macOS (e.g. /Applications/Google Chrome.app/Contents/MacOS/Google Chrome), not the basename. The old exact-match check (browserProcesses.includes(l)) never matched, causing false 'No browser instance running' errors. Changed to substring includes() with helper/renderer/crashpad filtering. * feat: browser_setup as agent-callable tool The agent can now self-recover from 'Browser harness not initialized' errors by calling browser_setup directly — no user mediation needed. - Extracted shared performSetup() from the slash command into setup.ts - Created src/domains/setup.ts with ensureAlive:false to bypass the socket guard (this tool IS the setup, can't require itself) - Registered browser_setup in the tool registry - Updated SKILL.md: agent calls browser_setup directly, not ask user - Removed stale test/manual/spawn-test.ts * release: v0.6.0 — per-tab isolation, Unix socket daemon, on-demand init
2026-05-07release: v0.5.0 — browser_console tool for JS error/console diagnosticsamankumarsingh77
2026-05-06release: v0.4.0 — accessibility snapshot, network capture, snapshot-first ↵amankumarsingh77
prompts Adds two new tools and pivots the agent's default workflow away from screenshots toward structured page data. browser_snapshot — CDP accessibility tree (roles, names, states, hierarchy) with click coordinates @(x,y) attached to every interactive element via DOM.getBoxModel (parallel, 1.5s aggregate budget). Pass directly to browser_click — no screenshot round-trip. Optional includeScreenshot:true for visual confirmation. format:"json" returns a slim structure with full box per node. browser_network_requests — replaces the deprecated browser_get_network_log placeholder. Filters by urlPattern (substring or /regex/), method, status, resource type, recency. Optional response-body capture under a 5s aggregate budget with 50KB per-body cap. Buffer is page-scoped (cleared on tab switch), 500 records/tab, with a bufferOverflowed flag on drain. CDP plumbing: in-process aggregator in src/cdp/network-buffer.ts wired into the existing single-consumer event loop. Accessibility domain enabled alongside Page/DOM/Runtime/Network on every attach. Ctrl+O expand/collapse on three tools' renderResult — browser_snapshot, browser_network_requests, browser_execute_js. Compact-by-default views with keyHint("app.tools.expand") that respects user keybinding remaps. Tool prompts pivoted snapshot-first: browser_screenshot is documented as LAST RESORT for visual verification only; browser_snapshot is the default for understanding pages; browser_execute_js is the default for surgical reads. browser_click and browser_open_urls guidance follows. SKILL.md collapsed from ~342 to ~53 lines — keeping only what isn't already in tool prompts: the decision tree, connection rules, browser_run_script daemon bindings. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05feat: tab ownership — isolate harness tabs from the user's browsingamankumarsingh77
Track which page targets this harness session opened so the agent can never attach to, switch to, or close a tab the user opened themselves. New tabs are spawned in a dedicated Chrome window (newWindow:true on first open, openerId chaining after) so harness activity stays visually separate. - new OwnershipRegistry (src/cdp/ownership.ts) tracks owned targetIds and the harness window seed; mutations notify a single listener that persists to BrowserState (ownedTargetIds, harnessWindowTargetId) - attachFirstPage prefers an already-owned live target; otherwise opens a fresh harness-owned tab in its own window instead of grabbing whatever page the user is on - session subscribes Target.setDiscoverTargets and reaps targetDestroyed events so the registry stays in sync with reality - listTabs returns owned:boolean per tab and reconciles dead ids - browser_list_tabs defaults to scope:"owned"; pass scope:"all" to inspect the user's other tabs (read-only — they cannot be switched to or closed) - browser_switch_tab refuses non-owned tabs, with a clear remediation hint - browser_new_tab opens inside the harness window via openerId, falling back to a fresh window when the seed has been closed - new browser_close_tab tool — owned tabs only, supports unique hex prefix Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04feat: parallel tool execution with automatic mutation serializationamankumarsingh77
- Add lightweight async mutex (src/util/mutex.ts) for FIFO serialization - Add serialized?: boolean to BrowserToolDefinition - Wire mutex acquisition into registerBrowserTool execute wrapper - Expose mutationMutex() on BrowserClient - Mark all mutation tools (click, type, scroll, navigate, switch_tab, etc.) as serialized - Leave observation tools (screenshot, page_info, execute_js, http_get, etc.) unguarded - Update prompt and skill docs to reflect parallel-safe behavior - Fix scroll tool: add Page.bringToFront, explicit button:none/buttons:0/pointerType:mouse - Update .gitignore to ignore scripts/ and data/ folders
2026-05-02feat(domains): migrate execute_js and run_script with path allowlist + ↵amankumarsingh77
timeout + return validation
2026-05-02feat(domains): migrate http_get (timeout covers body) and get_network_log ↵amankumarsingh77
(deprecation note)
2026-05-02feat(domains): migrate drag_and_dropamankumarsingh77
2026-05-02feat(domains): migrate viewport_resizeamankumarsingh77
2026-05-02feat(domains): migrate upload_file (safeJs), download (validated dir), ↵amankumarsingh77
print_to_pdf
2026-05-02feat(domains): migrate tab tools (list, current, switch, new)amankumarsingh77
2026-05-02feat(domains): migrate go_back, go_forward, reloadamankumarsingh77
2026-05-02feat(domains): migrate navigate (NavOutcome) and open_urlsamankumarsingh77
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-02feat(domains): migrate screenshot, add typed sharp shim, restore click ↵amankumarsingh77
crosshair overlay
2026-05-02feat(domains): migrate handle_dialogamankumarsingh77
2026-05-02feat(domains): migrate page_info, wait, wait_for_load with typed timeoutamankumarsingh77
2026-05-02feat(domains): migrate scroll with W3C deltaY semantics and dialog guardamankumarsingh77
2026-05-02feat(domains): migrate type, press_key, dispatch_key with safeJs and ↵amankumarsingh77
matched-count validation
2026-05-02feat(domains): migrate browser_click to defineBrowserTool — vertical sliceamankumarsingh77
Implements Task 9: adds src/domains/click.ts with the new architecture, updates registry.ts to register clickTool, removes the legacy browser_click block from tools.ts, and wires BrowserClient alongside BrowserDaemon in index.ts so both systems coexist during migration. Also fixes AnyBrowserToolDefinition variance: uses never-arg renderCall in the erased type and a boundary cast in registerBrowserTool so concrete tool definitions (with specific Static<S> args) are assignable to the registry array without widening.
2026-05-02feat: add defineBrowserTool helper, truncation, schemas/common, empty registryamankumarsingh77
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>