| Age | Commit message (Collapse) | Author |
|---|
|
Interaction tools could only target elements by raw viewport coordinates
(browser_click) or model-guessed CSS selectors (browser_fill/focus/
select_option/upload). Neither survives a re-render: on React/SPA forms,
every save or edit reflows the page, so captured @(x,y) coordinates go
stale and guessed selectors miss — leaving the agent to fall back to
hand-rolled execute_js DOM walks and unnoticed failed saves.
Add a stable ref system, mirroring Playwright MCP / browser-use:
- snapshot assigns each interactive element a ref (e1, e2, …), shown as
[eN] in the outline, and publishes a per-tab ref → backendNodeId map
- interaction tools accept an optional `ref` that re-resolves the element
at call time (via DOM.resolveNode / getBoxModel), so it stays valid
across re-renders; coordinates/selectors remain as fallbacks
- mutating tools append a compact diff of changed / newly-appeared (*) /
removed interactive elements, so silent failed saves surface immediately
- stale refs return an actionable "re-snapshot" error instead of a miss
Ref map is replaced per snapshot (bounded memory); fill keeps the native
_valueTracker write path so React controlled inputs still update.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
|
* feat: Unix socket daemon with auto-reconnect for persistent Chrome CDP
Eliminates the 'Allow Remote Debugging' Chrome prompt that appeared on
every pi session by multiplexing pi instances through a single persistent
daemon process.
Key additions:
- Unix socket IPC (net.createServer + readline JSON framing)
- CdpBridge with ID multiplexer + event router
- DaemonTransport implementing CdpTransport interface
- Auto-spawn with ensureDaemon() (lazy, checks socket)
Fixes since initial implementation:
- Bridge auto-reconnects to Chrome with exponential backoff (1s-60s).
Previously the bridge tried once at startup and permanently failed
if Chrome wasn't ready — causing 'Chrome not connected' on every
browser tool call.
- Daemon spawn uses project-local tsx binary instead of global --import
flag, fixing silent spawn failure (ERR_MODULE_NOT_FOUND).
- Stale ctx crash: persistState in onOwnershipChange wrapped in
try/catch to prevent CDP event consumer crashes.
49 test assertions across 4 test files. Zero TypeScript errors.
* feat: on-demand browser initialization
Browser daemon no longer auto-starts on session_start. Instead:
- session_start creates the client lazily but does NOT connect or spawn
the daemon. Zero Chrome prompts for non-browser pi sessions.
- Browser tool wrapper checks for daemon socket before each call.
Socket missing → clear error: 'Run /browser-setup first'.
Socket exists → proceeds silently (daemon already running).
- /browser-setup is the explicit, one-time init point. It spawns
the daemon and connects to Chrome only when the user opts in.
- SKILL.md updated with Connection Setup instructions telling agents
to direct users to /browser-setup when socket is missing.
* fix: checkChromeRunning on macOS — match by substring not exact
ps -A -o comm= returns full executable paths on modern macOS
(e.g. /Applications/Google Chrome.app/Contents/MacOS/Google Chrome),
not the basename. The old exact-match check (browserProcesses.includes(l))
never matched, causing false 'No browser instance running' errors.
Changed to substring includes() with helper/renderer/crashpad filtering.
* feat: browser_setup as agent-callable tool
The agent can now self-recover from 'Browser harness not initialized'
errors by calling browser_setup directly — no user mediation needed.
- Extracted shared performSetup() from the slash command into setup.ts
- Created src/domains/setup.ts with ensureAlive:false to bypass the
socket guard (this tool IS the setup, can't require itself)
- Registered browser_setup in the tool registry
- Updated SKILL.md: agent calls browser_setup directly, not ask user
- Removed stale test/manual/spawn-test.ts
* release: v0.6.0 — per-tab isolation, Unix socket daemon, on-demand init
|
|
Replace singleton dialog, pageInfoDirty, networkBuffer, and consoleBuffer
with per-tab TabSession objects stored in a Map<string, TabSession>.
- Add TabSession type (sessionId, targetId, dialog, pageInfoDirty,
networkBuffer, consoleBuffer) and tabs/sessionIdToTargetId Maps
- Add ev.sessionId filter guard in consumeEvents to prevent cross-tab
event contamination
- Rework switchTo to create/reuse TabSession without clearing buffers,
updating sessionId on re-attach
- Route all CDP events (dialog, network, console) through resolveTab
to the correct TabSession
- takeDialog/drainPageInfoInvalidations/drainNetworkBuffer/
drainConsoleBuffer read from current TabSession
- Target.targetDestroyed prunes dead TabSession entries
- attachFirstPage creates initial TabSession and prunes dead entries
- Per-tab pageCaches Map in client.ts replaces pageCache singleton
- listTabs reconciliation prunes dead pageCaches entries
- console-buffer.ts clear() resets nextSeq to 1 (per-tab seq space)
- Multi-agent warning in prompt.ts Parallel Execution section
|
|
|
|
prompts
Adds two new tools and pivots the agent's default workflow away from
screenshots toward structured page data.
browser_snapshot — CDP accessibility tree (roles, names, states, hierarchy)
with click coordinates @(x,y) attached to every interactive element via
DOM.getBoxModel (parallel, 1.5s aggregate budget). Pass directly to
browser_click — no screenshot round-trip. Optional includeScreenshot:true
for visual confirmation. format:"json" returns a slim structure with full
box per node.
browser_network_requests — replaces the deprecated browser_get_network_log
placeholder. Filters by urlPattern (substring or /regex/), method, status,
resource type, recency. Optional response-body capture under a 5s aggregate
budget with 50KB per-body cap. Buffer is page-scoped (cleared on tab switch),
500 records/tab, with a bufferOverflowed flag on drain.
CDP plumbing: in-process aggregator in src/cdp/network-buffer.ts wired into
the existing single-consumer event loop. Accessibility domain enabled
alongside Page/DOM/Runtime/Network on every attach.
Ctrl+O expand/collapse on three tools' renderResult — browser_snapshot,
browser_network_requests, browser_execute_js. Compact-by-default views with
keyHint("app.tools.expand") that respects user keybinding remaps.
Tool prompts pivoted snapshot-first: browser_screenshot is documented as
LAST RESORT for visual verification only; browser_snapshot is the default
for understanding pages; browser_execute_js is the default for surgical
reads. browser_click and browser_open_urls guidance follows.
SKILL.md collapsed from ~342 to ~53 lines — keeping only what isn't already
in tool prompts: the decision tree, connection rules, browser_run_script
daemon bindings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|
Track which page targets this harness session opened so the agent can never
attach to, switch to, or close a tab the user opened themselves. New tabs
are spawned in a dedicated Chrome window (newWindow:true on first open,
openerId chaining after) so harness activity stays visually separate.
- new OwnershipRegistry (src/cdp/ownership.ts) tracks owned targetIds and
the harness window seed; mutations notify a single listener that persists
to BrowserState (ownedTargetIds, harnessWindowTargetId)
- attachFirstPage prefers an already-owned live target; otherwise opens a
fresh harness-owned tab in its own window instead of grabbing whatever
page the user is on
- session subscribes Target.setDiscoverTargets and reaps targetDestroyed
events so the registry stays in sync with reality
- listTabs returns owned:boolean per tab and reconciles dead ids
- browser_list_tabs defaults to scope:"owned"; pass scope:"all" to inspect
the user's other tabs (read-only — they cannot be switched to or closed)
- browser_switch_tab refuses non-owned tabs, with a clear remediation hint
- browser_new_tab opens inside the harness window via openerId, falling
back to a fresh window when the seed has been closed
- new browser_close_tab tool — owned tabs only, supports unique hex prefix
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|
The Image text-fallback render does not respect the rendered width,
so a long file path can blow past the terminal width and crash the
host TUI. Wrap the component to truncate each rendered line with
truncateToWidth so it fits the available width.
Co-authored-by: amankumarsingh77 <amankumarsingh7702@gmal.com>
|
|
- Add lightweight async mutex (src/util/mutex.ts) for FIFO serialization
- Add serialized?: boolean to BrowserToolDefinition
- Wire mutex acquisition into registerBrowserTool execute wrapper
- Expose mutationMutex() on BrowserClient
- Mark all mutation tools (click, type, scroll, navigate, switch_tab, etc.) as serialized
- Leave observation tools (screenshot, page_info, execute_js, http_get, etc.) unguarded
- Update prompt and skill docs to reflect parallel-safe behavior
- Fix scroll tool: add Page.bringToFront, explicit button:none/buttons:0/pointerType:mouse
- Update .gitignore to ignore scripts/ and data/ folders
|
|
setup
|
|
parallelization guidance to SKILL.md
- browser_screenshot now has renderResult that creates an Image component
for inline display in Kitty/iTerm2/Ghostty/WezTerm terminals
- Falls back to dim text placeholder on terminals without image support
- Updated system prompt table description to mention TUI inline rendering
- Added Parallelization section to SKILL.md teaching the agent to batch
independent operations (browser_open_urls, browser_http_get, etc.)
|
|
|
|
matching, download auto-mkdir, session recovery, docs updates
Bug fixes:
- list_tabs: show full 32-char targetIds (was truncated to 8 chars)
- switch_tab: prefix matching fallback for hex prefixes >= 8 chars
- download: auto-create directory with mkdir -p
- ensureAlive: Runtime.evaluate probe to recover from crashed page targets
Docs:
- dispatch_key: clarify DOM-only event behavior
- navigate: warn about Google anti-bot limitation
- network_log: update deprecated promptSnippet
- SKILL.md: document actual daemon API (evaluateJs, pageInfo, session.call)
- SKILL.md: add troubleshooting for sharp/maxDim and Google navigation
- sharp: add to optionalDependencies for screenshot maxDim auto-resize
|
|
document IIFE-wrap heuristic, log shutdown failures, fix CHANGELOG file-size claim
Five tweaks from the whole-implementation review:
- session.ts: log unexpected event-consumer rejections instead of silent swallow
- client.ts: comment the return-substring IIFE-wrap false-positive case
- transport.ts: comment that ws.onclose post-open settle is no-op-by-design
- index.ts: log client.stop() failures during shutdown
- CHANGELOG.md: correct largest-file claim (transport.ts and client.ts are ~220 each, not "client.ts at ~180")
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|
|
|
single tsconfig
|
|
files to strict tsconfig
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
timeout + return validation
|
|
(deprecation note)
|
|
|
|
|
|
print_to_pdf
|
|
|
|
|
|
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
crosshair overlay
|
|
|
|
|
|
|
|
matched-count validation
|
|
Implements Task 9: adds src/domains/click.ts with the new architecture,
updates registry.ts to register clickTool, removes the legacy browser_click
block from tools.ts, and wires BrowserClient alongside BrowserDaemon in
index.ts so both systems coexist during migration.
Also fixes AnyBrowserToolDefinition variance: uses never-arg renderCall in
the erased type and a boundary cast in registerBrowserTool so concrete tool
definitions (with specific Static<S> args) are assignable to the registry
array without widening.
|
|
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
page info shape
|
|
|
|
|
|
|
|
errors
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Adds the missing tools called out in docs/browser-harness-comparison.md:
upload_file, dispatch_key, download (configure dir), viewport_resize,
drag_and_drop, print_to_pdf, get_network_log, plus the iframe-target
helper. Updates promptGuidelines and the system prompt's tools table.
Tracked in todo.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|
screenshots, page info caching
5 performance optimizations (14-36% faster end-to-end, output quality unchanged):
- fast ensureAlive(): WS state check instead of CDP roundtrip (96% faster per-call)
- event-based waitForLoad(): CDP load events instead of 300ms polling (99.5% faster for loaded tabs)
- JPEG screenshot support: 29-49% smaller files, faster CDP transfer
- page info caching: 1s TTL eliminates redundant evaluateJS calls
- parallel domain enables: Promise.all instead of sequential await in switchTab()
Benchmark: 3622ms vs 4243ms baseline on 3-tab navigation + screenshot task.
|
|
Remove register_tool/remove_tool/list_dynamic_tools and the entire
dynamic-tools.ts module. Replace with a single browser_run_script tool
that executes temporary scripts from disk with daemon access.
Scripts are now written to disk before execution, making them auditable
and re-runnable. Same bindings (daemon, require, params, etc.) as before
via the same AsyncFunction mechanism.
|
|
Previously a single consent-pending socket could spawn up to 5 stacked
'Allow remote debugging?' dialogs. Three causes:
- onopen reset _reconnectAttempts, but Chrome accepts the WS upgrade
before the user clicks Allow, so a held socket looked healthy.
- ensureAlive's health probe used the full 15s CDP timeout; while the
user was reading the dialog the probe would time out, trigger a
stop()+start(), and a fresh popup would stack on top.
- onclose auto-reconnected unconditionally even when consent had never
been granted on that connection.
Now a connection is only considered healthy once the first CDP
roundtrip completes (attachFirstPage). ensureAlive does a 2s short
probe before tearing down the socket. onclose bails out of auto-
reconnect when consent was never observed, so a denied dialog
surfaces a clean error instead of looping.
|
|
Drops the Unix socket subprocess in favor of talking CDP directly over
WebSocket from the extension. Removes the protocol's send/health helpers,
simplifies setup to just verifying Chrome's remote debugging port, and
tightens types in dynamic-tools. Bumps Node engine to >=22 and adds the
ws dependency.
|
|
Browser control extension for pi — 20 browser_* tools for navigating,
clicking, typing, screenshotting, and extracting data from real Chrome
via CDP. Includes self-extending harness (register_tool/remove_tool),
guided /browser-setup, session persistence, and custom TUI renderers.
|
