diff options
Diffstat (limited to 'docs/superpowers/plans')
| -rw-r--r-- | docs/superpowers/plans/2026-05-02-harness-rewrite.md | 2937 |
1 files changed, 0 insertions, 2937 deletions
diff --git a/docs/superpowers/plans/2026-05-02-harness-rewrite.md b/docs/superpowers/plans/2026-05-02-harness-rewrite.md deleted file mode 100644 index 700a469..0000000 --- a/docs/superpowers/plans/2026-05-02-harness-rewrite.md +++ /dev/null @@ -1,2937 +0,0 @@ -# pi-browser-harness v0.3.0 Internal Rewrite — Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Restructure `src/` into per-domain modules with a `defineTool` helper, a transport/session/client split, and structural fixes for the predictability bugs and security gaps catalogued in the spec — without breaking the 28 LLM-facing tool names. - -**Architecture:** Per-domain decomposition. Dependencies flow `domains/* → client → cdp/* → util/*`. Every tool handler returns `Result<ToolOk, ToolErr>`; one `defineTool`/`registerTool` helper converts that into pi's `ToolResult` and supplies the unified `details` shape. The current `BrowserDaemon` class becomes a `createBrowserClient(opts)` factory composing a `CdpTransport` + `CdpSession`. JS injection is structurally prevented by routing all evaluation source through `safeJs\`...\``. `browser_run_script` keeps its full Node powers but gains a path allowlist, mandatory timeout, and AbortSignal wiring. - -**Tech Stack:** TypeScript (strict mode + extended safety flags), `ws` (WebSocket), `typebox` (tool schemas), `@mariozechner/pi-coding-agent` (extension API + truncation helpers), `@mariozechner/pi-tui` (renderers). - -**Reference spec:** `docs/superpowers/specs/2026-05-02-harness-rewrite-design.md` - ---- - -## Build order (why the tasks run in this order) - -1. **Foundations first (Tasks 1–3).** `Result`, `safeJs`, `keycodes`, `paths`, `time` are pure utilities depended on by everything else. Land them with `tsc` green so the rest of the work has a solid base. -2. **Strict tsconfig third (Task 3).** Turning on the new flags before the rest of the code is rewritten would explode the codebase. We turn them on *only* over `src/util/**` first, then expand the include scope as new code lands. By the end the whole `src/` is covered. -3. **Transport / session / client (Tasks 4–7).** This is the biggest single piece. `transport.ts` (WS) → `session.ts` (target/dialog/event handling) → `discovery.ts` (DevToolsActivePort) → `client.ts` (composition). At this point the new `client` exists but no domains use it yet; `daemon.ts` and `tools.ts` still serve every tool call. The package still ships and works. -4. **`defineTool` helper (Task 8).** Cannot be tested without at least one domain consumer, so we land the helper alongside the first vertical slice. -5. **First vertical slice — `domains/click.ts` (Task 9).** Smallest stateless tool that exercises the full path: schema → handler → client → CDP → Result → registry. Validates the architecture end-to-end. After this task, `browser_click` is served by the new path; the other 27 tools still use `tools.ts`. Manual smoke checkpoint here. -6. **Bulk domain migration (Tasks 10–22).** One task per domain file. Each task removes the corresponding tool block(s) from `tools.ts` and adds the new domain file. `tools.ts` shrinks by ~70-100 LOC per task. -7. **Wire-up + dead-code deletion (Tasks 23–25).** `index.ts`, `state.ts`, `prompt.ts`, then delete `daemon.ts`, `tools.ts` (now empty), `protocol.ts`, `renderers.ts`. -8. **Final hardening (Tasks 26–27).** Strict-flag expansion to whole `src/`, manual smoke run of the verification scenarios from spec §11, version bump + CHANGELOG. - -## Green-bar checkpoints - -After every task: `npm run typecheck` must pass. Failures block the next task. - -Manual smoke checkpoints: **after Task 9** (one tool migrated end-to-end) and **after Task 27** (full migration). Smoke procedure is in spec §11. - ---- - -## File layout (target end-state) - -``` -src/ - index.ts registry.ts prompt.ts state.ts setup.ts - client.ts - cdp/ - transport.ts session.ts discovery.ts errors.ts types.ts - domains/ - navigate.ts history.ts click.ts keyboard.ts scroll.ts - screenshot.ts tabs.ts page.ts dialog.ts js.ts - files.ts viewport.ts drag.ts network.ts - schemas/ - common.ts - util/ - result.ts tool.ts truncate.ts keycodes.ts paths.ts - js-template.ts time.ts -``` - -Deleted at end: `daemon.ts`, `tools.ts`, `protocol.ts`, `renderers.ts`. - ---- - -## Task 1: Foundations — `util/result.ts`, `util/time.ts`, `util/paths.ts` - -**Files:** -- Create: `src/util/result.ts` -- Create: `src/util/time.ts` -- Create: `src/util/paths.ts` - -- [ ] **Step 1.1: Write `src/util/result.ts`** - -```ts -export type Result<T, E> = - | { readonly success: true; readonly data: T } - | { readonly success: false; readonly error: E }; - -export const ok = <T>(data: T): Result<T, never> => ({ success: true, data }); -export const err = <E>(error: E): Result<never, E> => ({ success: false, error }); - -export const map = <T, U, E>(r: Result<T, E>, f: (t: T) => U): Result<U, E> => - r.success ? ok(f(r.data)) : r; - -export const andThen = <T, U, E>(r: Result<T, E>, f: (t: T) => Result<U, E>): Result<U, E> => - r.success ? f(r.data) : r; - -export const mapErr = <T, E, F>(r: Result<T, E>, f: (e: E) => F): Result<T, F> => - r.success ? r : err(f(r.error)); -``` - -- [ ] **Step 1.2: Write `src/util/time.ts`** - -```ts -export const sleep = (ms: number, signal?: AbortSignal): Promise<void> => - new Promise((resolve, reject) => { - if (signal?.aborted) { - reject(new Error("aborted")); - return; - } - const timer = setTimeout(() => { - signal?.removeEventListener("abort", onAbort); - resolve(); - }, ms); - const onAbort = (): void => { - clearTimeout(timer); - reject(new Error("aborted")); - }; - signal?.addEventListener("abort", onAbort, { once: true }); - }); - -export type Deadline = { - readonly remainingMs: () => number; - readonly expired: () => boolean; -}; - -export const deadline = (totalMs: number): Deadline => { - const end = Date.now() + totalMs; - return { - remainingMs: () => Math.max(0, end - Date.now()), - expired: () => Date.now() >= end, - }; -}; -``` - -- [ ] **Step 1.3: Write `src/util/paths.ts`** - -```ts -import { randomUUID } from "node:crypto"; -import { tmpdir } from "node:os"; -import { join } from "node:path"; - -export const screenshotPath = (namespace: string, ext: "png" | "jpeg" = "png"): string => - join(tmpdir(), `pi-browser-screenshot-${namespace}-${randomUUID()}.${ext}`); - -export const pdfPath = (namespace: string): string => - join(tmpdir(), `pi-browser-pdf-${namespace}-${randomUUID()}.pdf`); -``` - -- [ ] **Step 1.4: Typecheck** - -Run: `npm run typecheck` -Expected: PASS (no errors). - -- [ ] **Step 1.5: Commit** - -```bash -git add src/util/result.ts src/util/time.ts src/util/paths.ts -git commit -m "feat(util): add result, time, and paths primitives for v0.3 rewrite" -``` - ---- - -## Task 2: Foundations — `util/keycodes.ts`, `util/js-template.ts` - -**Files:** -- Create: `src/util/keycodes.ts` -- Create: `src/util/js-template.ts` - -- [ ] **Step 2.1: Write `src/util/keycodes.ts`** (extracted from `daemon.ts:126-141` and `daemon.ts:815-822`) - -```ts -const VIRTUAL_KEY_CODES: Readonly<Record<string, number>> = { - Enter: 13, Tab: 9, Backspace: 8, Escape: 27, Delete: 46, " ": 32, - ArrowLeft: 37, ArrowUp: 38, ArrowRight: 39, ArrowDown: 40, - Home: 36, End: 35, PageUp: 33, PageDown: 34, -}; - -export const SPECIAL_KEYS: ReadonlyArray<string> = Object.keys(VIRTUAL_KEY_CODES); - -export const virtualKeyCode = (key: string): number => - VIRTUAL_KEY_CODES[key] ?? (key.length === 1 ? key.charCodeAt(0) : 0); - -export const keyCode = (key: string): string => key; -``` - -- [ ] **Step 2.2: Write `src/util/js-template.ts`** - -```ts -/** - * Build a JavaScript source string with safely-interpolated values. - * Every interpolated value is JSON.stringify'd, so strings, numbers, - * objects, and special characters all become valid JS literals. - * - * This is the ONLY supported way to build evaluation source in the - * codebase — never use raw template literals for JS that crosses the - * CDP boundary. - */ -export const safeJs = ( - strings: TemplateStringsArray, - ...values: ReadonlyArray<unknown> -): string => { - let out = strings[0] ?? ""; - for (let i = 0; i < values.length; i++) { - out += JSON.stringify(values[i]); - out += strings[i + 1] ?? ""; - } - return out; -}; -``` - -- [ ] **Step 2.3: Typecheck** - -Run: `npm run typecheck` -Expected: PASS. - -- [ ] **Step 2.4: Commit** - -```bash -git add src/util/keycodes.ts src/util/js-template.ts -git commit -m "feat(util): add pure keycodes and safeJs template helper" -``` - ---- - -## Task 3: Strict tsconfig (scoped to `src/util/**` for now) - -**Files:** -- Modify: `tsconfig.json` - -- [ ] **Step 3.1: Replace `tsconfig.json` with the strict config** - -```json -{ - "compilerOptions": { - "target": "ES2022", - "module": "ESNext", - "moduleResolution": "bundler", - "strict": true, - "noImplicitAny": true, - "strictNullChecks": true, - "noUnusedLocals": true, - "noUnusedParameters": true, - "noImplicitReturns": true, - "noFallthroughCasesInSwitch": true, - "noUncheckedIndexedAccess": true, - "exactOptionalPropertyTypes": true, - "noPropertyAccessFromIndexSignature": true, - "forceConsistentCasingInFileNames": true, - "esModuleInterop": true, - "skipLibCheck": true, - "outDir": "./dist", - "rootDir": "./src", - "declaration": true, - "declarationMap": true, - "sourceMap": true - }, - "include": ["src/**/*.ts"], - "exclude": ["node_modules", "dist"] -} -``` - -- [ ] **Step 3.2: Typecheck — expect failures in `daemon.ts`/`tools.ts`/`index.ts`** - -Run: `npm run typecheck` -Expected: errors in legacy files (we'll fix them as we migrate). Capture the count. - -- [ ] **Step 3.3: Add a temporary include carve-out so legacy code compiles during migration** - -Create: `tsconfig.legacy.json` - -```json -{ - "extends": "./tsconfig.json", - "compilerOptions": { - "noUnusedLocals": false, - "noUnusedParameters": false, - "noUncheckedIndexedAccess": false, - "exactOptionalPropertyTypes": false, - "noPropertyAccessFromIndexSignature": false - }, - "include": [ - "src/daemon.ts", - "src/tools.ts", - "src/index.ts", - "src/protocol.ts", - "src/renderers.ts", - "src/setup.ts", - "src/state.ts", - "src/prompt.ts" - ] -} -``` - -Modify `tsconfig.json` `include` to: -```json -"include": ["src/util/**/*.ts", "src/cdp/**/*.ts", "src/client.ts", "src/domains/**/*.ts", "src/schemas/**/*.ts", "src/registry.ts"] -``` - -Modify `package.json` `scripts.typecheck` to: `tsc --noEmit && tsc --noEmit -p tsconfig.legacy.json` - -- [ ] **Step 3.4: Typecheck — both projects must pass** - -Run: `npm run typecheck` -Expected: PASS. - -- [ ] **Step 3.5: Commit** - -```bash -git add tsconfig.json tsconfig.legacy.json package.json -git commit -m "build: enable strict tsconfig flags for new code, keep legacy lenient" -``` - ---- - -## Task 4: CDP types and errors - -**Files:** -- Create: `src/cdp/types.ts` -- Create: `src/cdp/errors.ts` - -- [ ] **Step 4.1: Write `src/cdp/types.ts`** - -```ts -export type CdpEvent = { - readonly method: string; - readonly params: unknown; - readonly sessionId?: string; -}; - -export type CdpRawMessage = { - readonly id?: number; - readonly method?: string; - readonly params?: Record<string, unknown>; - readonly result?: unknown; - readonly error?: { readonly message: string; readonly code?: number }; - readonly sessionId?: string; -}; - -export type DialogInfo = { - readonly type: "alert" | "confirm" | "prompt" | "beforeunload"; - readonly message: string; - readonly defaultPrompt?: string; -}; - -export type TabInfo = { - readonly targetId: string; - readonly title: string; - readonly url: string; -}; - -export type PageInfo = { - readonly url: string; - readonly title: string; - readonly width: number; - readonly height: number; - readonly scrollX: number; - readonly scrollY: number; - readonly pageWidth: number; - readonly pageHeight: number; -}; - -export type DaemonStatus = { - readonly alive: boolean; - readonly sessionId: string | null; - readonly namespace: string; - readonly remoteBrowserId?: string; -}; - -const INTERNAL_PREFIXES: ReadonlyArray<string> = [ - "chrome://", "chrome-untrusted://", "devtools://", "chrome-extension://", "about:", -]; - -export const isInternalUrl = (url: string): boolean => - INTERNAL_PREFIXES.some((p) => url.startsWith(p)); -``` - -- [ ] **Step 4.2: Write `src/cdp/errors.ts`** - -```ts -export type CdpErrorKind = - | "transport_closed" - | "timeout" - | "session_not_found" - | "remote_error" - | "discovery_failed" - | "invalid_response"; - -export type CdpError = { - readonly kind: CdpErrorKind; - readonly message: string; - readonly method?: string; -}; - -export const cdpError = ( - kind: CdpErrorKind, - message: string, - method?: string, -): CdpError => ({ kind, message, ...(method !== undefined ? { method } : {}) }); -``` - -- [ ] **Step 4.3: Typecheck** - -Run: `npm run typecheck` -Expected: PASS. - -- [ ] **Step 4.4: Commit** - -```bash -git add src/cdp/types.ts src/cdp/errors.ts -git commit -m "feat(cdp): add types and error model for new transport layer" -``` - ---- - -## Task 5: CDP discovery - -**Files:** -- Create: `src/cdp/discovery.ts` - -- [ ] **Step 5.1: Write `src/cdp/discovery.ts`** (port `daemon.ts:41-124` to Result) - -```ts -import { readFile } from "node:fs/promises"; -import { homedir } from "node:os"; -import { join } from "node:path"; -import { connect as netConnect } from "node:net"; -import { type Result, err, ok } from "../util/result"; -import { type CdpError, cdpError } from "./errors"; - -const PORT_PROBE_DEADLINE_MS = 30_000; -const PORT_PROBE_INTERVAL_MS = 1_000; - -const profileDirs = (): ReadonlyArray<string> => { - const home = homedir(); - return [ - join(home, "Library/Application Support/Google/Chrome"), - join(home, "Library/Application Support/Microsoft Edge"), - join(home, "Library/Application Support/Microsoft Edge Beta"), - join(home, "Library/Application Support/Microsoft Edge Dev"), - join(home, "Library/Application Support/Microsoft Edge Canary"), - join(home, ".config/google-chrome"), - join(home, ".config/chromium"), - join(home, ".config/chromium-browser"), - join(home, ".config/microsoft-edge"), - join(home, ".config/microsoft-edge-beta"), - join(home, ".config/microsoft-edge-dev"), - join(home, ".var/app/org.chromium.Chromium/config/chromium"), - join(home, ".var/app/com.google.Chrome/config/google-chrome"), - join(home, ".var/app/com.brave.Browser/config/BraveSoftware/Brave-Browser"), - join(home, ".var/app/com.microsoft.Edge/config/microsoft-edge"), - join(home, "AppData/Local/Google/Chrome/User Data"), - join(home, "AppData/Local/Chromium/User Data"), - join(home, "AppData/Local/Microsoft/Edge/User Data"), - join(home, "AppData/Local/Microsoft/Edge Beta/User Data"), - join(home, "AppData/Local/Microsoft/Edge Dev/User Data"), - join(home, "AppData/Local/Microsoft/Edge SxS/User Data"), - ]; -}; - -const probePort = (port: number): Promise<Result<void, CdpError>> => - new Promise((resolve) => { - const sock = netConnect({ host: "127.0.0.1", port }); - const finish = (r: Result<void, CdpError>): void => { - sock.destroy(); - resolve(r); - }; - sock.setTimeout(1000, () => finish(err(cdpError("discovery_failed", "probe timeout")))); - sock.once("error", (e) => finish(err(cdpError("discovery_failed", e.message)))); - sock.once("connect", () => { - sock.end(); - resolve(ok(undefined)); - }); - }); - -const waitForPort = async (port: number): Promise<Result<void, CdpError>> => { - const end = Date.now() + PORT_PROBE_DEADLINE_MS; - let lastMessage = "unknown"; - while (Date.now() < end) { - const probe = await probePort(port); - if (probe.success) return probe; - lastMessage = probe.error.message; - await new Promise((r) => setTimeout(r, PORT_PROBE_INTERVAL_MS)); - } - return err(cdpError( - "discovery_failed", - `Chrome's remote-debugging page is open, but DevTools is not live yet on 127.0.0.1:${port} — if Chrome opened a profile picker, choose your normal profile first, then tick the checkbox and click Allow if shown (last error: ${lastMessage})`, - )); -}; - -export const discoverWsUrl = async (): Promise<Result<string, CdpError>> => { - const dirs = profileDirs(); - for (const base of dirs) { - const portFile = join(base, "DevToolsActivePort"); - let raw: string; - try { - raw = await readFile(portFile, "utf8"); - } catch { - continue; - } - const lines = raw.trim().split("\n"); - if (lines.length < 2) continue; - const port = lines[0]?.trim(); - const path = lines[1]?.trim(); - if (!port || !path) continue; - const ready = await waitForPort(Number(port)); - if (!ready.success) return ready; - return ok(`ws://127.0.0.1:${port}${path}`); - } - return err(cdpError( - "discovery_failed", - `DevToolsActivePort not found in ${dirs.join(", ")} — open chrome://inspect/#remote-debugging in your browser, tick the checkbox, click Allow, then retry. Or set BU_CDP_WS to a remote browser endpoint.`, - )); -}; -``` - -- [ ] **Step 5.2: Typecheck** - -Run: `npm run typecheck` -Expected: PASS. - -- [ ] **Step 5.3: Commit** - -```bash -git add src/cdp/discovery.ts -git commit -m "feat(cdp): port DevToolsActivePort discovery to Result" -``` - ---- - -## Task 6: CDP transport - -**Files:** -- Create: `src/cdp/transport.ts` - -- [ ] **Step 6.1: Write `src/cdp/transport.ts`** - -```ts -import WebSocket from "ws"; -import { type Result, err, ok } from "../util/result"; -import { type CdpError, cdpError } from "./errors"; -import type { CdpEvent, CdpRawMessage } from "./types"; - -const DEFAULT_TIMEOUT_MS = 15_000; - -type Pending = { - readonly resolve: (v: Result<unknown, CdpError>) => void; - readonly timer: ReturnType<typeof setTimeout>; - readonly method: string; -}; - -export type CdpTransport = { - connect(url: string, opts?: { timeoutMs?: number }): Promise<Result<void, CdpError>>; - close(): Promise<void>; - request( - method: string, - params: Record<string, unknown>, - opts?: { sessionId?: string | null; timeoutMs?: number }, - ): Promise<Result<unknown, CdpError>>; - events(): AsyncIterable<CdpEvent>; - state(): "open" | "closed" | "connecting"; - onClose(cb: () => void): () => void; -}; - -export const createCdpTransport = (): CdpTransport => { - let ws: WebSocket | null = null; - let nextId = 1; - const pending = new Map<number, Pending>(); - const closeListeners = new Set<() => void>(); - - type EventQueue = { - readonly push: (e: CdpEvent) => void; - readonly end: () => void; - readonly iter: AsyncIterable<CdpEvent>; - }; - - const makeEventQueue = (): EventQueue => { - const buf: CdpEvent[] = []; - const waiters: Array<(v: IteratorResult<CdpEvent>) => void> = []; - let ended = false; - return { - push(e) { - if (ended) return; - const w = waiters.shift(); - if (w) w({ value: e, done: false }); - else buf.push(e); - }, - end() { - ended = true; - for (const w of waiters.splice(0)) w({ value: undefined as unknown as CdpEvent, done: true }); - }, - iter: { - [Symbol.asyncIterator]() { - return { - next: (): Promise<IteratorResult<CdpEvent>> => - new Promise((resolve) => { - const next = buf.shift(); - if (next) resolve({ value: next, done: false }); - else if (ended) resolve({ value: undefined as unknown as CdpEvent, done: true }); - else waiters.push(resolve); - }), - }; - }, - }, - }; - }; - - let queue = makeEventQueue(); - - const handleMessage = (raw: string): void => { - let msg: CdpRawMessage; - try { - msg = JSON.parse(raw); - } catch { - return; - } - if (msg.id === undefined) { - if (msg.method) { - queue.push({ - method: msg.method, - params: msg.params, - ...(msg.sessionId !== undefined ? { sessionId: msg.sessionId } : {}), - }); - } - return; - } - const p = pending.get(msg.id); - if (!p) return; - pending.delete(msg.id); - clearTimeout(p.timer); - if (msg.error) { - const kind = msg.error.message.includes("Session with given id not found") - ? "session_not_found" - : "remote_error"; - p.resolve(err(cdpError(kind, msg.error.message, p.method))); - return; - } - p.resolve(ok(msg.result)); - }; - - const cleanup = (reason: string): void => { - for (const [, p] of pending) { - clearTimeout(p.timer); - p.resolve(err(cdpError("transport_closed", reason, p.method))); - } - pending.clear(); - queue.end(); - queue = makeEventQueue(); - for (const cb of closeListeners) cb(); - }; - - return { - connect(url, opts = {}): Promise<Result<void, CdpError>> { - const timeoutMs = opts.timeoutMs ?? 10_000; - return new Promise((resolve) => { - let settled = false; - const settle = (r: Result<void, CdpError>): void => { - if (settled) return; - settled = true; - resolve(r); - }; - try { - ws = new WebSocket(url); - } catch (e) { - settle(err(cdpError("transport_closed", e instanceof Error ? e.message : String(e)))); - return; - } - const timer = setTimeout(() => { - ws?.close(); - ws = null; - settle(err(cdpError("timeout", `CDP WebSocket connection timed out after ${timeoutMs}ms`))); - }, timeoutMs); - ws.onopen = () => { - clearTimeout(timer); - settle(ok(undefined)); - }; - ws.onmessage = (ev: WebSocket.MessageEvent) => { - const data = ev.data; - handleMessage(typeof data === "string" ? data : data.toString()); - }; - ws.onerror = () => { - clearTimeout(timer); - settle(err(cdpError("transport_closed", "CDP WebSocket error during connection"))); - }; - ws.onclose = () => { - clearTimeout(timer); - ws = null; - cleanup("WebSocket closed"); - settle(err(cdpError("transport_closed", "CDP WebSocket closed"))); - }; - }); - }, - close(): Promise<void> { - if (ws) { - try { ws.close(1000, "Shutdown"); } catch { /* best effort */ } - ws = null; - } - cleanup("close() called"); - return Promise.resolve(); - }, - request(method, params, opts = {}): Promise<Result<unknown, CdpError>> { - const timeoutMs = opts.timeoutMs ?? DEFAULT_TIMEOUT_MS; - const sessionId = opts.sessionId ?? null; - if (!ws || ws.readyState !== WebSocket.OPEN) { - return Promise.resolve(err(cdpError("transport_closed", "Browser not connected. Is Chrome running?", method))); - } - const id = nextId++; - const payload: Record<string, unknown> = { id, method, params }; - if (sessionId) payload.sessionId = sessionId; - const json = JSON.stringify(payload); - return new Promise((resolve) => { - const timer = setTimeout(() => { - pending.delete(id); - resolve(err(cdpError("timeout", `CDP timeout after ${timeoutMs}ms: ${method}`, method))); - }, timeoutMs); - pending.set(id, { resolve, timer, method }); - try { - ws!.send(json); - } catch (e) { - clearTimeout(timer); - pending.delete(id); - resolve(err(cdpError("transport_closed", e instanceof Error ? e.message : String(e), method))); - } - }); - }, - events(): AsyncIterable<CdpEvent> { - return queue.iter; - }, - state(): "open" | "closed" | "connecting" { - if (!ws) return "closed"; - if (ws.readyState === WebSocket.CONNECTING) return "connecting"; - if (ws.readyState === WebSocket.OPEN) return "open"; - return "closed"; - }, - onClose(cb): () => void { - closeListeners.add(cb); - return () => closeListeners.delete(cb); - }, - }; -}; -``` - -- [ ] **Step 6.2: Typecheck** - -Run: `npm run typecheck` -Expected: PASS. - -- [ ] **Step 6.3: Commit** - -```bash -git add src/cdp/transport.ts -git commit -m "feat(cdp): add factory-based transport with Result and event AsyncIterable" -``` - ---- - -## Task 7: CDP session + client - -**Files:** -- Create: `src/cdp/session.ts` -- Create: `src/client.ts` - -- [ ] **Step 7.1: Write `src/cdp/session.ts`** - -```ts -import { type Result, err, ok } from "../util/result"; -import { type CdpError, cdpError } from "./errors"; -import type { CdpEvent, DialogInfo, PageInfo } from "./types"; -import { isInternalUrl } from "./types"; -import type { CdpTransport } from "./transport"; - -export type CdpSession = { - attachFirstPage(): Promise<Result<{ readonly targetId: string; readonly sessionId: string }, CdpError>>; - switchTo(targetId: string): Promise<Result<void, CdpError>>; - current(): { readonly sessionId: string; readonly targetId: string } | null; - call(method: string, params?: Record<string, unknown>, opts?: { timeoutMs?: number }): Promise<Result<unknown, CdpError>>; - callOnTarget(method: string, params: Record<string, unknown>, sessionId: string, opts?: { timeoutMs?: number }): Promise<Result<unknown, CdpError>>; - callBrowser(method: string, params?: Record<string, unknown>, opts?: { timeoutMs?: number }): Promise<Result<unknown, CdpError>>; - takeDialog(): DialogInfo | null; - drainPageInfoInvalidations(): boolean; -}; - -export const createCdpSession = (transport: CdpTransport): CdpSession => { - let sessionId: string | null = null; - let targetId: string | null = null; - let dialog: DialogInfo | null = null; - let pageInfoDirty = false; - - const consumeEvents = async (): Promise<void> => { - for await (const ev of transport.events()) { - if (ev.method === "Page.javascriptDialogOpening") { - const params = ev.params as Partial<DialogInfo> | undefined; - dialog = { - type: (params?.type as DialogInfo["type"]) ?? "alert", - message: params?.message ?? "", - ...(params?.defaultPrompt !== undefined ? { defaultPrompt: params.defaultPrompt } : {}), - }; - continue; - } - if (ev.method === "Page.javascriptDialogClosed") { - dialog = null; - continue; - } - if (ev.method === "Page.frameNavigated" || ev.method === "Page.loadEventFired") { - pageInfoDirty = true; - } - } - }; - - void consumeEvents(); - transport.onClose(() => { - sessionId = null; - targetId = null; - dialog = null; - pageInfoDirty = false; - void consumeEvents(); - }); - - const enableDomains = async (sid: string): Promise<void> => { - for (const d of ["Page", "DOM", "Runtime", "Network"]) { - await transport.request(`${d}.enable`, {}, { sessionId: sid }); - } - }; - - return { - async attachFirstPage() { - const targets = await transport.request("Target.getTargets", {}, { sessionId: null }); - if (!targets.success) return targets; - const data = targets.data as { targetInfos: ReadonlyArray<{ targetId: string; type: string; url: string }> }; - let pages = data.targetInfos.filter((t) => t.type === "page" && !isInternalUrl(t.url)); - if (pages.length === 0) { - const created = await transport.request("Target.createTarget", { url: "about:blank" }, { sessionId: null }); - if (!created.success) return created; - const c = created.data as { targetId: string }; - pages = [{ targetId: c.targetId, type: "page", url: "about:blank" }]; - } - const first = pages[0]; - if (!first) return err(cdpError("invalid_response", "no page targets after creation")); - const attached = await transport.request("Target.attachToTarget", { targetId: first.targetId, flatten: true }, { sessionId: null }); - if (!attached.success) return attached; - const a = attached.data as { sessionId: string }; - sessionId = a.sessionId; - targetId = first.targetId; - await enableDomains(a.sessionId); - return ok({ targetId: first.targetId, sessionId: a.sessionId }); - }, - async switchTo(tid) { - const activated = await transport.request("Target.activateTarget", { targetId: tid }, { sessionId: null }); - if (!activated.success) return activated; - const attached = await transport.request("Target.attachToTarget", { targetId: tid, flatten: true }, { sessionId: null }); - if (!attached.success) return attached; - const a = attached.data as { sessionId: string }; - sessionId = a.sessionId; - targetId = tid; - pageInfoDirty = true; - await enableDomains(a.sessionId); - return ok(undefined); - }, - current() { - return sessionId && targetId ? { sessionId, targetId } : null; - }, - call(method, params = {}, opts = {}) { - return transport.request(method, params, { sessionId, ...(opts.timeoutMs !== undefined ? { timeoutMs: opts.timeoutMs } : {}) }); - }, - callOnTarget(method, params, sid, opts = {}) { - return transport.request(method, params, { sessionId: sid, ...(opts.timeoutMs !== undefined ? { timeoutMs: opts.timeoutMs } : {}) }); - }, - callBrowser(method, params = {}, opts = {}) { - return transport.request(method, params, { sessionId: null, ...(opts.timeoutMs !== undefined ? { timeoutMs: opts.timeoutMs } : {}) }); - }, - takeDialog() { - const d = dialog; - dialog = null; - return d; - }, - drainPageInfoInvalidations() { - const dirty = pageInfoDirty; - pageInfoDirty = false; - return dirty; - }, - }; -}; - -export type CachedPage = { readonly info: PageInfo; readonly at: number }; - -export const isPageInfoFresh = (cache: CachedPage | null, dirty: boolean, ttlMs: number): boolean => - cache !== null && !dirty && Date.now() - cache.at < ttlMs; -``` - -- [ ] **Step 7.2: Write `src/client.ts`** (skeleton — domains add concrete methods) - -```ts -import { writeFile } from "node:fs/promises"; -import { type Result, andThen, err, ok } from "./util/result"; -import { safeJs } from "./util/js-template"; -import { discoverWsUrl } from "./cdp/discovery"; -import { type CdpError, cdpError } from "./cdp/errors"; -import type { CdpTransport } from "./cdp/transport"; -import { createCdpTransport } from "./cdp/transport"; -import { type CdpSession, createCdpSession } from "./cdp/session"; -import type { DaemonStatus, DialogInfo, PageInfo, TabInfo } from "./cdp/types"; - -export type BrowserClientOptions = { - readonly namespace: string; - readonly remote?: { readonly cdpUrl: string; readonly browserId: string }; -}; - -export type BrowserClient = { - readonly namespace: string; - ensureAlive(): Promise<Result<void, CdpError>>; - status(): DaemonStatus; - start(): Promise<Result<void, CdpError>>; - stop(): Promise<void>; - raw: (method: string, params?: Record<string, unknown>, sessionId?: string) => Promise<Result<unknown, CdpError>>; - evaluateJs(expression: string, sessionId?: string): Promise<Result<unknown, CdpError>>; - pageInfo(): Promise<Result<PageInfo | { readonly dialog: DialogInfo }, CdpError>>; - takeDialog(): DialogInfo | null; - listTabs(includeInternal?: boolean): Promise<Result<ReadonlyArray<TabInfo>, CdpError>>; - switchTab(targetId: string): Promise<Result<void, CdpError>>; - newTab(url?: string): Promise<Result<string, CdpError>>; - current(): { readonly sessionId: string; readonly targetId: string } | null; - session(): CdpSession; - transport(): CdpTransport; -}; - -const HEALTH_TTL_MS = 30_000; -const PAGE_INFO_TTL_MS = 1_000; - -export const createBrowserClient = (opts: BrowserClientOptions): BrowserClient => { - const transport = createCdpTransport(); - const session = createCdpSession(transport); - let lastHealth = 0; - let pageCache: { readonly info: PageInfo; readonly at: number } | null = null; - let remote = opts.remote ?? null; - - const start = async (): Promise<Result<void, CdpError>> => { - if (transport.state() === "open" && session.current()) return ok(undefined); - const wsUrl = remote?.cdpUrl - ?? process.env["BU_CDP_WS"] - ?? (await (async () => { - const r = await discoverWsUrl(); - return r.success ? r.data : null; - })()); - if (!wsUrl) { - const r = await discoverWsUrl(); - if (!r.success) return r; - const connected = await transport.connect(r.data, { timeoutMs: 10_000 }); - if (!connected.success) return connected; - } else { - const connected = await transport.connect(wsUrl, { timeoutMs: 10_000 }); - if (!connected.success) return connected; - remote = remote ?? { cdpUrl: wsUrl, browserId: wsUrl.split("/").pop() ?? "unknown" }; - } - const attached = await session.attachFirstPage(); - if (!attached.success) { - await transport.close(); - return attached; - } - lastHealth = Date.now(); - pageCache = null; - return ok(undefined); - }; - - const stop = async (): Promise<void> => { - await transport.close(); - pageCache = null; - lastHealth = 0; - }; - - const ensureAlive = async (): Promise<Result<void, CdpError>> => { - if (transport.state() !== "open" || !session.current()) { - await stop(); - return start(); - } - if (Date.now() - lastHealth < HEALTH_TTL_MS) return ok(undefined); - const probe = await transport.request("Target.getTargets", {}, { sessionId: null, timeoutMs: 2_000 }); - if (probe.success) { - lastHealth = Date.now(); - return ok(undefined); - } - await stop(); - return start(); - }; - - const evaluateJs = async (expression: string, sessionId?: string): Promise<Result<unknown, CdpError>> => { - const wrapped = expression.includes("return ") && !expression.trim().startsWith("(") - ? `(function(){${expression}})()` - : expression; - const r = sessionId - ? await session.callOnTarget("Runtime.evaluate", { expression: wrapped, returnByValue: true, awaitPromise: true }, sessionId) - : await session.call("Runtime.evaluate", { expression: wrapped, returnByValue: true, awaitPromise: true }); - if (!r.success) return r; - const data = r.data as { result?: { value?: unknown }; exceptionDetails?: unknown }; - if (data.exceptionDetails) { - return err(cdpError("remote_error", `JS evaluation failed: ${JSON.stringify(data.exceptionDetails)}`, "Runtime.evaluate")); - } - return ok(data.result?.value); - }; - - const readPageInfo = async (): Promise<Result<PageInfo, CdpError>> => { - const dirty = session.drainPageInfoInvalidations(); - if (pageCache && !dirty && Date.now() - pageCache.at < PAGE_INFO_TTL_MS) return ok(pageCache.info); - const expr = safeJs`JSON.stringify({url:location.href,title:document.title,w:innerWidth,h:innerHeight,sx:scrollX,sy:scrollY,pw:document.documentElement.scrollWidth,ph:document.documentElement.scrollHeight})`; - const raw = await evaluateJs(expr); - if (!raw.success) return raw; - if (typeof raw.data !== "string") return err(cdpError("invalid_response", "page info evaluation did not return a string")); - const parsedRaw = JSON.parse(raw.data) as { url: string; title: string; w: number; h: number; sx: number; sy: number; pw: number; ph: number }; - const info: PageInfo = { - url: parsedRaw.url, title: parsedRaw.title, - width: parsedRaw.w, height: parsedRaw.h, - scrollX: parsedRaw.sx, scrollY: parsedRaw.sy, - pageWidth: parsedRaw.pw, pageHeight: parsedRaw.ph, - }; - pageCache = { info, at: Date.now() }; - return ok(info); - }; - - const pageInfo = async (): Promise<Result<PageInfo | { readonly dialog: DialogInfo }, CdpError>> => { - const d = session.takeDialog(); - if (d) return ok({ dialog: d }); - return readPageInfo(); - }; - - const listTabs = async (includeInternal = true): Promise<Result<ReadonlyArray<TabInfo>, CdpError>> => { - const r = await session.callBrowser("Target.getTargets"); - if (!r.success) return r; - const data = r.data as { targetInfos: ReadonlyArray<{ targetId: string; type: string; title: string; url: string }> }; - const tabs = data.targetInfos - .filter((t) => t.type === "page") - .filter((t) => includeInternal || !t.url.startsWith("chrome://")) - .map((t): TabInfo => ({ targetId: t.targetId, title: t.title, url: t.url })); - return ok(tabs); - }; - - const switchTab = async (targetId: string): Promise<Result<void, CdpError>> => { - const r = await session.switchTo(targetId); - if (!r.success) return r; - pageCache = null; - await session.call("Runtime.evaluate", { expression: safeJs`if(!document.title.startsWith('🟢'))document.title='🟢 '+document.title` }); - return ok(undefined); - }; - - const newTab = async (url?: string): Promise<Result<string, CdpError>> => { - const created = await session.callBrowser("Target.createTarget", { url: "about:blank" }); - if (!created.success) return created; - const c = created.data as { targetId: string }; - const switched = await switchTab(c.targetId); - if (!switched.success) return switched; - if (url && url !== "about:blank") { - const nav = await session.call("Page.navigate", { url }); - if (!nav.success) return nav; - pageCache = null; - } - return ok(c.targetId); - }; - - const status = (): DaemonStatus => ({ - alive: transport.state() === "open" && session.current() !== null, - sessionId: session.current()?.sessionId ?? null, - namespace: opts.namespace, - ...(remote?.browserId !== undefined ? { remoteBrowserId: remote.browserId } : {}), - }); - - return { - namespace: opts.namespace, - ensureAlive, status, start, stop, - raw: (method, params, sessionId) => session.call(method, params, sessionId !== undefined ? { } : {}), - evaluateJs, pageInfo, - takeDialog: () => session.takeDialog(), - listTabs, switchTab, newTab, - current: () => session.current(), - session: () => session, - transport: () => transport, - }; -}; -``` - -(Note: this client deliberately exposes lower-level methods; domain files extend behavior by composing these. Domain-specific helpers like `screenshot.captureWithCrosshair` live in their domain files since they need `sharp` etc., not in `client.ts`.) - -- [ ] **Step 7.3: Typecheck** - -Run: `npm run typecheck` -Expected: PASS. - -- [ ] **Step 7.4: Commit** - -```bash -git add src/cdp/session.ts src/client.ts -git commit -m "feat: add CDP session and BrowserClient factory" -``` - ---- - -## Task 8: `defineTool` helper + registry skeleton - -**Files:** -- Create: `src/util/tool.ts` -- Create: `src/util/truncate.ts` -- Create: `src/registry.ts` -- Create: `src/schemas/common.ts` - -- [ ] **Step 8.1: Write `src/util/truncate.ts`** (port the existing `applyTruncation` from `tools.ts:62-94`) - -```ts -import { mkdtemp, rm, writeFile } from "node:fs/promises"; -import { tmpdir } from "node:os"; -import { join } from "node:path"; -import { - DEFAULT_MAX_BYTES, - DEFAULT_MAX_LINES, - formatSize, - truncateHead, - withFileMutationQueue, -} from "@mariozechner/pi-coding-agent"; - -const tempDirs: string[] = []; - -export const cleanupTempDirs = async (): Promise<void> => { - const dirs = tempDirs.splice(0); - await Promise.all(dirs.map((d) => rm(d, { recursive: true, force: true }).catch(() => {}))); -}; - -export type TruncatedOutput = { - readonly text: string; - readonly fullOutputPath?: string; - readonly wasTruncated: boolean; -}; - -export const applyTruncation = async (output: string, prefix: string): Promise<TruncatedOutput> => { - const t = truncateHead(output, { maxLines: DEFAULT_MAX_LINES, maxBytes: DEFAULT_MAX_BYTES }); - if (!t.truncated) return { text: t.content, wasTruncated: false }; - const dir = await mkdtemp(join(tmpdir(), `pi-bh-${prefix}-`)); - tempDirs.push(dir); - const file = join(dir, "output.txt"); - await withFileMutationQueue(file, async () => { await writeFile(file, output, "utf8"); }); - const omitted = t.totalBytes - t.outputBytes; - const text = `${t.content}\n\n[Output truncated: ${t.outputLines} of ${t.totalLines} lines (${formatSize(t.outputBytes)} of ${formatSize(t.totalBytes)}). ${formatSize(omitted)} omitted. Full output: ${file}]`; - return { text, fullOutputPath: file, wasTruncated: true }; -}; -``` - -- [ ] **Step 8.2: Write `src/util/tool.ts`** - -```ts -import type { TSchema, Static } from "typebox"; -import type { ExtensionAPI } from "@mariozechner/pi-coding-agent"; -import type { BrowserClient } from "../client"; -import type { Result } from "./result"; - -export type ToolOk = { - readonly text: string; - readonly details?: Readonly<Record<string, unknown>>; -}; - -export type ToolErrKind = - | "not_connected" | "cdp_error" | "timeout" - | "invalid_state" | "io_error" | "internal"; - -export type ToolErr = { - readonly kind: ToolErrKind; - readonly message: string; - readonly details?: Readonly<Record<string, unknown>>; -}; - -export type HandlerContext = { - readonly client: BrowserClient; - readonly signal: AbortSignal; - readonly onUpdate: (update: ToolOk) => void; - readonly extensionCtx: unknown; -}; - -export type ToolHandler<S extends TSchema> = ( - args: Static<S>, - ctx: HandlerContext, -) => Promise<Result<ToolOk, ToolErr>>; - -export type ToolDefinition<S extends TSchema> = { - readonly name: string; - readonly label: string; - readonly description: string; - readonly promptSnippet: string; - readonly promptGuidelines: ReadonlyArray<string>; - readonly parameters: S; - readonly handler: ToolHandler<S>; - readonly renderCall?: (args: Static<S>, theme: unknown) => unknown; - readonly renderResult?: (result: unknown, opts: { readonly expanded: boolean }, theme: unknown) => unknown; - readonly ensureAlive?: boolean; -}; - -// Type-erased registration entry for the registry array. -export type RegisteredTool = ToolDefinition<TSchema>; - -export const defineTool = <S extends TSchema>(def: ToolDefinition<S>): ToolDefinition<S> => def; - -type ToolResultLike = { - isError?: boolean; - content: ReadonlyArray<{ readonly type: "text"; readonly text: string }>; - details?: Readonly<Record<string, unknown>>; -}; - -const toToolResult = (r: Result<ToolOk, ToolErr>, toolName: string): ToolResultLike => { - if (r.success) { - return { - content: [{ type: "text", text: r.data.text }], - details: { ok: true, ...(r.data.details ?? {}) }, - }; - } - return { - isError: true, - content: [{ type: "text", text: `${toolName} failed (${r.error.kind}): ${r.error.message}` }], - details: { ok: false, kind: r.error.kind, message: r.error.message, ...(r.error.details ?? {}) }, - }; -}; - -export const registerTool = <S extends TSchema>( - pi: ExtensionAPI, - client: BrowserClient, - def: ToolDefinition<S>, -): void => { - pi.registerTool({ - name: def.name, - label: def.label, - description: def.description, - promptSnippet: def.promptSnippet, - promptGuidelines: [...def.promptGuidelines], - parameters: def.parameters, - ...(def.renderCall ? { renderCall: def.renderCall } : {}), - ...(def.renderResult ? { renderResult: def.renderResult } : {}), - async execute(_id: string, args: unknown, signal: AbortSignal, onUpdate, extensionCtx) { - if (def.ensureAlive !== false) { - const alive = await client.ensureAlive(); - if (!alive.success) { - return toToolResult( - { success: false, error: { kind: "not_connected", message: alive.error.message } }, - def.name, - ); - } - } - const result = await def.handler(args as Static<S>, { - client, - signal, - onUpdate: (u) => onUpdate?.({ content: [{ type: "text", text: u.text }], details: u.details }), - extensionCtx, - }); - return toToolResult(result, def.name); - }, - }); -}; -``` - -- [ ] **Step 8.3: Write `src/schemas/common.ts`** - -```ts -import { Type } from "typebox"; - -export const Coords = Type.Object({ - x: Type.Number({ description: "X coordinate in CSS pixels from left edge of viewport" }), - y: Type.Number({ description: "Y coordinate in CSS pixels from top edge of viewport" }), -}); - -export const MouseButton = Type.Union([ - Type.Literal("left"), - Type.Literal("right"), - Type.Literal("middle"), -], { description: 'Mouse button: "left", "right", or "middle"' }); -``` - -- [ ] **Step 8.4: Write `src/registry.ts`** (empty array for now; populated as domains migrate) - -```ts -import type { ExtensionAPI } from "@mariozechner/pi-coding-agent"; -import type { BrowserClient } from "./client"; -import { type RegisteredTool, registerTool } from "./util/tool"; - -const TOOLS: ReadonlyArray<RegisteredTool> = [ - // domain tools registered here, one per domain file -]; - -export const registerAllTools = (pi: ExtensionAPI, client: BrowserClient): void => { - for (const t of TOOLS) registerTool(pi, client, t); -}; -``` - -- [ ] **Step 8.5: Typecheck** - -Run: `npm run typecheck` -Expected: PASS (TOOLS array is empty so no concrete usage yet). - -- [ ] **Step 8.6: Commit** - -```bash -git add src/util/tool.ts src/util/truncate.ts src/schemas/common.ts src/registry.ts -git commit -m "feat: add defineTool helper, truncation, and empty registry" -``` - ---- - -## Task 9: First vertical slice — `domains/click.ts` - -This is the validation moment for the architecture. After this task, `browser_click` is served by the new path; the other 27 tools still go through `tools.ts`. - -**Files:** -- Create: `src/domains/click.ts` -- Modify: `src/registry.ts` (add `clickTool`) -- Modify: `src/tools.ts` (remove the `browser_click` registration block, lines ~324-396) -- Modify: `src/index.ts` (call `registerAllTools` alongside the existing `registerTools`) - -- [ ] **Step 9.1: Write `src/domains/click.ts`** - -```ts -import { Type, type Static } from "typebox"; -import { Text } from "@mariozechner/pi-tui"; -import type { Result } from "../util/result"; -import { ok, err } from "../util/result"; -import { defineTool, type ToolErr, type ToolOk } from "../util/tool"; -import { Coords, MouseButton } from "../schemas/common"; -import { screenshotPath } from "../util/paths"; - -const ClickArgs = Type.Object({ - ...Coords.properties, - button: Type.Optional(MouseButton), - count: Type.Optional(Type.Integer({ minimum: 1, maximum: 3, default: 1, description: "Number of clicks (1 = single, 2 = double). Default: 1" })), -}); -type ClickArgsT = Static<typeof ClickArgs>; - -const dispatchClick = async ( - client: import("../client").BrowserClient, - args: ClickArgsT, -): Promise<Result<void, ToolErr>> => { - const button = args.button ?? "left"; - const count = args.count ?? 1; - const pressed = await client.session().call("Input.dispatchMouseEvent", { - type: "mousePressed", x: args.x, y: args.y, button, clickCount: count, - }); - if (!pressed.success) return err({ kind: "cdp_error", message: pressed.error.message }); - const released = await client.session().call("Input.dispatchMouseEvent", { - type: "mouseReleased", x: args.x, y: args.y, button, clickCount: count, - }); - if (!released.success) return err({ kind: "cdp_error", message: released.error.message }); - return ok(undefined); -}; - -export const clickTool = defineTool({ - name: "browser_click", - label: "Browser Click", - description: "Click at viewport CSS-pixel coordinates. Compositor-level click works through iframes, shadow DOM, and cross-origin content. Use browser_screenshot first to find coordinates.", - promptSnippet: "Click at pixel coordinates (x, y) on the page", - promptGuidelines: [ - "Use browser_click for all clicks. Coordinates are viewport CSS pixels (not device pixels).", - "Capture a browser_screenshot BEFORE clicking to find the right coordinates.", - "Capture another browser_screenshot AFTER clicking to verify the action worked.", - "Compositor-level clicks pass through iframes, shadow DOM, and cross-origin content — no selector needed.", - "If a click doesn't register, try setting BH_DEBUG_CLICKS=1 to get annotated screenshots showing exact click positions.", - "For React/Vue components that don't respond to clicks, try browser_dispatch_key to send DOM-level events.", - ], - parameters: ClickArgs, - renderCall: (a) => new Text(`🖱️ Click at (${a.x}, ${a.y})`, 0, 0), - async handler(args, { client }): Promise<Result<ToolOk, ToolErr>> { - const clicked = await dispatchClick(client, args); - if (!clicked.success) return clicked; - if (process.env["BH_DEBUG_CLICKS"]) { - const path = screenshotPath(client.namespace, "png"); - // Debug overlay implementation lives in domains/screenshot.ts which we - // haven't migrated yet — for now, just produce a normal screenshot. - const shot = await client.session().call("Page.captureScreenshot", { format: "png" }); - if (shot.success) { - const data = (shot.data as { data: string }).data; - const { writeFile } = await import("node:fs/promises"); - await writeFile(path, Buffer.from(data, "base64")); - return ok({ - text: `Clicked at (${args.x}, ${args.y})\n[DEBUG] Screenshot: ${path}`, - details: { debugScreenshotPath: path, x: args.x, y: args.y }, - }); - } - } - return ok({ text: `Clicked at (${args.x}, ${args.y})`, details: { x: args.x, y: args.y } }); - }, -}); -``` - -(The crosshair overlay returns in Task 14 when we migrate `domains/screenshot.ts`; for now the debug branch just saves a plain screenshot.) - -- [ ] **Step 9.2: Add `clickTool` to `src/registry.ts`** - -```ts -import type { ExtensionAPI } from "@mariozechner/pi-coding-agent"; -import type { BrowserClient } from "./client"; -import { type RegisteredTool, registerTool } from "./util/tool"; -import { clickTool } from "./domains/click"; - -const TOOLS: ReadonlyArray<RegisteredTool> = [ - clickTool, -]; - -export const registerAllTools = (pi: ExtensionAPI, client: BrowserClient): void => { - for (const t of TOOLS) registerTool(pi, client, t); -}; -``` - -- [ ] **Step 9.3: Remove the `browser_click` block from `src/tools.ts`** (lines ~321-396 in current code, including the comment header). The rest of the file stays untouched. - -- [ ] **Step 9.4: Wire the new client into `src/index.ts`** - -In `src/index.ts`, around the existing `if (!toolsRegistered && daemon) { registerTools(pi, daemon); ... }` block, add a parallel client/registry path. The legacy `daemon` and `tools.ts` continue to serve everything except `browser_click`. - -```ts -// Near the top, alongside the daemon import: -import { createBrowserClient, type BrowserClient } from "./client"; -import { registerAllTools } from "./registry"; - -// In the function scope, alongside `let daemon`: -let client: BrowserClient | null = null; - -// Inside session_start, AFTER `daemon = new BrowserDaemon(state.namespace)`: -client = createBrowserClient({ namespace: state.namespace }); -try { await client.start(); } catch { /* surfaced via tool errors */ } - -// Inside the `if (!toolsRegistered && daemon)` block, AFTER `registerTools(pi, daemon)`: -if (client) registerAllTools(pi, client); - -// Inside session_shutdown, AFTER `await daemon.stop()`: -if (client) { try { await client.stop(); } catch { /* best-effort */ } client = null; } -``` - -(Both `daemon` and `client` connect to the same Chrome via separate WebSockets during the migration window. This is fine — Chrome accepts multiple CDP connections.) - -- [ ] **Step 9.5: Typecheck** - -Run: `npm run typecheck` -Expected: PASS. - -- [ ] **Step 9.6: Manual smoke (vertical slice)** - -Pre-req: Chrome running with remote debugging enabled (chrome://inspect/#remote-debugging → Allow). - -1. `cd benchmark && node -e "require('../src/index.ts')"` — won't work directly because pi loads it; instead, install into a local pi workspace per CONTRIBUTING.md, then in pi: - - `/browser-setup` - - `browser_navigate("https://example.com")` - - `browser_screenshot()` → note coordinates of "More information..." link - - `browser_click({x: <X>, y: <Y>})` ← this hits the new path - - `browser_screenshot()` to confirm navigation - -If `browser_click` works and the result `details` contains `{ ok: true, x, y }`, the architecture is validated. If not, debug before proceeding. - -- [ ] **Step 9.7: Commit** - -```bash -git add src/domains/click.ts src/registry.ts src/tools.ts src/index.ts -git commit -m "feat(domains): migrate browser_click to defineTool — vertical slice" -``` - ---- - -## Tasks 10–22: Bulk domain migration - -Each task follows the same pattern as Task 9 but only the schema/handler/legacy-removal differ. To keep this plan manageable, the per-task structure is: - -- Step 1: Write `src/domains/<name>.ts` using `defineTool`. Schemas live in the file; types via `Static<>`. Handler returns `Result<ToolOk, ToolErr>`. JS evaluation uses `safeJs`. CDP calls use `client.session().call(...)` or `client.session().callBrowser(...)`. -- Step 2: Add the new tool(s) to `TOOLS` array in `src/registry.ts`, in the listed order. -- Step 3: Remove the corresponding tool registration block(s) from `src/tools.ts`. -- Step 4: `npm run typecheck` → PASS. -- Step 5: Commit with message `feat(domains): migrate <tool-names>`. - -The detailed schema + handler skeleton for each domain follows. - ---- - -### Task 10: `domains/keyboard.ts` — `browser_type`, `browser_press_key`, `browser_dispatch_key` - -**Files:** -- Create: `src/domains/keyboard.ts` -- Modify: `src/registry.ts`, `src/tools.ts` - -- [ ] **Step 10.1: Write `src/domains/keyboard.ts`** - -```ts -import { Type, type Static } from "typebox"; -import { type Result, err, ok } from "../util/result"; -import { defineTool, type ToolErr, type ToolOk } from "../util/tool"; -import { safeJs } from "../util/js-template"; -import { virtualKeyCode } from "../util/keycodes"; -import type { BrowserClient } from "../client"; - -const TypeArgs = Type.Object({ text: Type.String({ description: "Text to type" }) }); - -export const typeTool = defineTool({ - name: "browser_type", - label: "Browser Type", - description: "Type text into the currently focused element. Use browser_click first to focus an input field.", - promptSnippet: "Type text into the focused element", - promptGuidelines: [ - "Use browser_type to enter text. Click on an input field with browser_click first to focus it.", - "For special keys (Enter, Tab, Escape, arrows), use browser_press_key instead.", - ], - parameters: TypeArgs, - async handler(args, { client }): Promise<Result<ToolOk, ToolErr>> { - const r = await client.session().call("Input.insertText", { text: args.text }); - if (!r.success) return err({ kind: "cdp_error", message: r.error.message }); - return ok({ text: `Typed: "${args.text}"` }); - }, -}); - -const PressKeyArgs = Type.Object({ - key: Type.String({ description: 'Key to press. Special keys: Enter, Tab, Backspace, Escape, Delete, ArrowLeft/Up/Right/Down, Home, End, PageUp, PageDown. Space as " "' }), - modifiers: Type.Optional(Type.Integer({ minimum: 0, maximum: 15, description: "Modifier bitfield: 1=Alt, 2=Ctrl, 4=Meta(Cmd), 8=Shift. Combine with OR." })), -}); - -export const pressKeyTool = defineTool({ - name: "browser_press_key", - label: "Browser Press Key", - description: "Press a keyboard key. Supports special keys (Enter, Tab, Backspace, Escape, Delete, arrows, Home, End, PageUp, PageDown, Space as ' ') and regular characters. Optional modifier bitfield: 1=Alt, 2=Ctrl, 4=Meta/Cmd, 8=Shift.", - promptSnippet: "Press a key (Enter, Tab, Escape, arrows, or any character)", - promptGuidelines: [ - "Use browser_press_key for keyboard shortcuts and navigation keys.", - "Special key names: Enter, Tab, Backspace, Escape, Delete, ArrowLeft, ArrowUp, ArrowRight, ArrowDown, Home, End, PageUp, PageDown.", - "Use Space as ' ' (a single space character).", - "Modifiers: 1=Alt, 2=Ctrl, 4=Meta(Cmd), 8=Shift. Combine with bitwise OR: Ctrl+Shift = 2|8 = 10.", - ], - parameters: PressKeyArgs, - async handler(args, { client }): Promise<Result<ToolOk, ToolErr>> { - const k = args.key; - const code = virtualKeyCode(k); - const modifiers = args.modifiers ?? 0; - const isChar = k.length === 1; - const downParams = { - type: "keyDown", - key: k, - code: k, - ...(code ? { windowsVirtualKeyCode: code, nativeVirtualKeyCode: code } : {}), - modifiers, - ...(isChar ? { text: k, unmodifiedText: k } : {}), - }; - const down = await client.session().call("Input.dispatchKeyEvent", downParams); - if (!down.success) return err({ kind: "cdp_error", message: down.error.message }); - const up = await client.session().call("Input.dispatchKeyEvent", { type: "keyUp", key: k, code: k, modifiers }); - if (!up.success) return err({ kind: "cdp_error", message: up.error.message }); - return ok({ text: `Pressed: ${k}${modifiers ? ` (modifiers=${modifiers})` : ""}` }); - }, -}); - -const DispatchKeyArgs = Type.Object({ - selector: Type.String({ description: "CSS selector of the target element" }), - key: Type.String({ description: "Key value (e.g., 'Enter', 'a')" }), - eventType: Type.Optional(Type.Union([ - Type.Literal("keydown"), Type.Literal("keyup"), Type.Literal("keypress"), - ], { default: "keydown", description: "Event type to dispatch. Default: keydown" })), -}); - -export const dispatchKeyTool = defineTool({ - name: "browser_dispatch_key", - label: "Browser Dispatch Key", - description: "Dispatch a DOM KeyboardEvent on a specific element via JS injection. Use for React/Vue components that listen to synthetic events more reliably than CDP input.", - promptSnippet: "Dispatch a DOM KeyboardEvent on a specific element", - promptGuidelines: [ - "Try browser_press_key first; only use browser_dispatch_key when the page ignores raw CDP key events.", - "The selector must match exactly one or more elements; zero matches is reported as an error.", - "eventType defaults to 'keydown'.", - ], - parameters: DispatchKeyArgs, - async handler(args, { client }): Promise<Result<ToolOk, ToolErr>> { - const eventType = args.eventType ?? "keydown"; - const expr = safeJs` - (() => { - const els = document.querySelectorAll(${args.selector}); - if (els.length === 0) return 0; - for (const el of els) { - el.dispatchEvent(new KeyboardEvent(${eventType}, { key: ${args.key}, bubbles: true, cancelable: true })); - } - return els.length; - })() - `; - const r = await client.evaluateJs(expr); - if (!r.success) return err({ kind: "cdp_error", message: r.error.message }); - const matched = Number(r.data ?? 0); - if (matched === 0) { - return err({ - kind: "invalid_state", - message: `Selector matched 0 elements: ${args.selector}`, - details: { matched: 0, selector: args.selector }, - }); - } - return ok({ text: `Dispatched ${eventType}('${args.key}') on ${matched} element(s)`, details: { matched, selector: args.selector } }); - }, -}); -``` - -- [ ] **Step 10.2: Add to `TOOLS` array**: `typeTool, pressKeyTool, dispatchKeyTool`. -- [ ] **Step 10.3: Remove `browser_type`, `browser_press_key`, `browser_dispatch_key` blocks from `src/tools.ts`. -- [ ] **Step 10.4: Typecheck — PASS.** -- [ ] **Step 10.5: Commit** `feat(domains): migrate type, press_key, dispatch_key` - ---- - -### Task 11: `domains/scroll.ts` — `browser_scroll` - -**Files:** Create `src/domains/scroll.ts`; modify `src/registry.ts`, `src/tools.ts`. - -- [ ] **Step 11.1: Write `src/domains/scroll.ts`** - -```ts -import { Type, type Static } from "typebox"; -import { type Result, err, ok } from "../util/result"; -import { defineTool, type ToolErr, type ToolOk } from "../util/tool"; - -const ScrollArgs = Type.Object({ - x: Type.Optional(Type.Number({ description: "X coordinate where to scroll. Default: viewport center" })), - y: Type.Optional(Type.Number({ description: "Y coordinate where to scroll. Default: viewport center" })), - deltaX: Type.Optional(Type.Number({ default: 0, description: "Horizontal scroll delta (CSS pixels). Positive = right." })), - deltaY: Type.Optional(Type.Number({ default: -300, description: "Vertical scroll delta (CSS pixels). Positive = up (matches W3C wheel events). Default: -300 (scroll down)." })), -}); - -export const scrollTool = defineTool({ - name: "browser_scroll", - label: "Browser Scroll", - description: "Scroll the page at given coordinates. deltaY follows W3C wheel-event convention: positive = scroll up, negative = scroll down. Default scrolls down (deltaY = -300).", - promptSnippet: "Scroll the page (deltaY positive=up, negative=down)", - promptGuidelines: [ - "Default behavior scrolls down 300px (deltaY=-300). Pass a positive deltaY to scroll up.", - "Pass x/y to target a specific scrollable region (e.g., a div with overflow); otherwise scrolls the page at viewport center.", - ], - parameters: ScrollArgs, - async handler(args, { client }): Promise<Result<ToolOk, ToolErr>> { - const info = await client.pageInfo(); - if (!info.success) return err({ kind: "cdp_error", message: info.error.message }); - if ("dialog" in info.data) { - return err({ kind: "invalid_state", message: `Dialog open: ${info.data.dialog.type} — ${info.data.dialog.message}` }); - } - const cx = args.x ?? Math.round(info.data.width / 2); - const cy = args.y ?? Math.round(info.data.height / 2); - const dx = args.deltaX ?? 0; - const dy = args.deltaY ?? -300; - // Establish compositor mouse position before wheel — fixes the scroll bug from v0.2.0. - const moved = await client.session().call("Input.dispatchMouseEvent", { type: "mouseMoved", x: cx, y: cy }); - if (!moved.success) return err({ kind: "cdp_error", message: moved.error.message }); - const wheel = await client.session().call("Input.dispatchMouseEvent", { - type: "mouseWheel", x: cx, y: cy, deltaX: dx, deltaY: dy, - }); - if (!wheel.success) return err({ kind: "cdp_error", message: wheel.error.message }); - return ok({ text: `Scrolled at (${cx}, ${cy}) by (${dx}, ${dy})`, details: { x: cx, y: cy, deltaX: dx, deltaY: dy } }); - }, -}); -``` - -- [ ] **Step 11.2: Add `scrollTool` to TOOLS.** **Step 11.3: Remove `browser_scroll` from `tools.ts`.** **Step 11.4: Typecheck PASS.** **Step 11.5: Commit** `feat(domains): migrate scroll`. - ---- - -### Task 12: `domains/page.ts` — `browser_page_info`, `browser_wait`, `browser_wait_for_load` - -**Files:** Create `src/domains/page.ts`; modify `src/registry.ts`, `src/tools.ts`. - -- [ ] **Step 12.1: Write `src/domains/page.ts`** - -```ts -import { Type } from "typebox"; -import { type Result, err, ok } from "../util/result"; -import { defineTool, type ToolErr, type ToolOk } from "../util/tool"; -import { sleep } from "../util/time"; -import { safeJs } from "../util/js-template"; - -export const pageInfoTool = defineTool({ - name: "browser_page_info", - label: "Browser Page Info", - description: "Get current page state: URL, title, viewport size, scroll position, page dimensions. If a JS dialog is open, returns dialog info instead.", - promptSnippet: "Get current page URL, title, viewport, and scroll position", - promptGuidelines: [ - "Use browser_page_info to quickly check what page you're on and whether a JS dialog is blocking interaction.", - "If browser_page_info returns a dialog, use browser_handle_dialog before any other browser actions.", - "JS dialogs freeze the page's JS thread, so no other interaction works until the dialog is handled.", - "browser_page_info auto-detects alert, confirm, prompt, and beforeunload dialogs.", - ], - parameters: Type.Object({}), - async handler(_args, { client }): Promise<Result<ToolOk, ToolErr>> { - const r = await client.pageInfo(); - if (!r.success) return err({ kind: "cdp_error", message: r.error.message }); - if ("dialog" in r.data) { - const d = r.data.dialog; - return ok({ - text: `⚠️ DIALOG OPEN: ${d.type}\nMessage: ${d.message}${d.defaultPrompt ? `\nDefault: ${d.defaultPrompt}` : ""}\n\nUse browser_handle_dialog to accept or dismiss.`, - details: { dialog: d }, - }); - } - const i = r.data; - return ok({ - text: `URL: ${i.url}\nTitle: ${i.title}\nViewport: ${i.width}x${i.height}\nScroll: (${i.scrollX}, ${i.scrollY})\nPage size: ${i.pageWidth}x${i.pageHeight}`, - details: { ...i }, - }); - }, -}); - -const WaitArgs = Type.Object({ - seconds: Type.Number({ minimum: 0, maximum: 60, description: "Seconds to wait" }), -}); - -export const waitTool = defineTool({ - name: "browser_wait", - label: "Browser Wait", - description: "Wait N seconds before continuing.", - promptSnippet: "Wait N seconds", - promptGuidelines: ["Use sparingly — prefer browser_wait_for_load when waiting for a page load."], - parameters: WaitArgs, - async handler(args, { signal }): Promise<Result<ToolOk, ToolErr>> { - await sleep(Math.round(args.seconds * 1000), signal); - return ok({ text: `Waited ${args.seconds}s` }); - }, -}); - -const WaitForLoadArgs = Type.Object({ - timeout: Type.Optional(Type.Number({ default: 15, minimum: 1, maximum: 120, description: "Max seconds to wait. Default: 15." })), -}); - -export const waitForLoadTool = defineTool({ - name: "browser_wait_for_load", - label: "Browser Wait For Load", - description: "Wait until the current page reports document.readyState === 'complete', subscribing to Page.frameStartedLoading first to avoid returning early on a stale ready state.", - promptSnippet: "Wait for the page to finish loading", - promptGuidelines: [ - "Call after browser_navigate / browser_open_urls before extracting data.", - "Returns when readyState becomes 'complete' OR the timeout elapses.", - ], - parameters: WaitForLoadArgs, - async handler(args, { client }): Promise<Result<ToolOk, ToolErr>> { - const timeoutMs = (args.timeout ?? 15) * 1000; - const start = Date.now(); - while (Date.now() - start < timeoutMs) { - const r = await client.evaluateJs(safeJs`document.readyState`); - if (r.success && r.data === "complete") { - return ok({ text: `Page loaded in ${Math.round((Date.now() - start) / 100) / 10}s`, details: { ms: Date.now() - start } }); - } - await new Promise((res) => setTimeout(res, 50)); - } - return err({ kind: "timeout", message: `Page did not finish loading in ${args.timeout ?? 15}s` }); - }, -}); -``` - -- [ ] **Step 12.2:** Add `pageInfoTool, waitTool, waitForLoadTool` to TOOLS. **Step 12.3:** Remove the three blocks from `tools.ts`. **Step 12.4:** Typecheck PASS. **Step 12.5:** Commit `feat(domains): migrate page_info, wait, wait_for_load`. - ---- - -### Task 13: `domains/dialog.ts` — `browser_handle_dialog` - -- [ ] **Step 13.1: Write `src/domains/dialog.ts`** - -```ts -import { Type } from "typebox"; -import { type Result, err, ok } from "../util/result"; -import { defineTool, type ToolErr, type ToolOk } from "../util/tool"; - -const HandleDialogArgs = Type.Object({ - accept: Type.Boolean({ description: "true = accept, false = dismiss" }), - promptText: Type.Optional(Type.String({ description: "Text to type if dialog is a prompt()" })), -}); - -export const handleDialogTool = defineTool({ - name: "browser_handle_dialog", - label: "Browser Handle Dialog", - description: "Accept or dismiss the currently open JS dialog (alert/confirm/prompt/beforeunload).", - promptSnippet: "Accept or dismiss a JS dialog", - promptGuidelines: [ - "Use after browser_page_info reports a dialog is open. Until handled, no other browser action will work.", - "For prompt() dialogs, supply promptText with the value to submit.", - ], - parameters: HandleDialogArgs, - async handler(args, { client }): Promise<Result<ToolOk, ToolErr>> { - const params: Record<string, unknown> = { accept: args.accept }; - if (args.promptText !== undefined) params.promptText = args.promptText; - const r = await client.session().call("Page.handleJavaScriptDialog", params); - if (!r.success) return err({ kind: "cdp_error", message: r.error.message }); - return ok({ text: `Dialog ${args.accept ? "accepted" : "dismissed"}`, details: { accept: args.accept } }); - }, -}); -``` - -- [ ] **Step 13.2:** TOOLS, **Step 13.3:** remove from `tools.ts`, **Step 13.4:** typecheck, **Step 13.5:** commit `feat(domains): migrate handle_dialog`. - ---- - -### Task 14: `domains/screenshot.ts` — `browser_screenshot` (and restore the debug crosshair) - -**Files:** Create `src/domains/screenshot.ts`; create `src/util/sharp-shim.ts`; modify `src/registry.ts`, `src/tools.ts`, and `src/domains/click.ts` (replace the placeholder with the real overlay call). - -- [ ] **Step 14.1: Write `src/util/sharp-shim.ts`** - -```ts -// Typed minimal interface for the optional `sharp` dependency. -// We import dynamically because sharp is a heavy native module and not all -// installs include it. Returns Result so callers can distinguish "not installed" -// from "installed but threw". - -import { type Result, err, ok } from "./result"; - -export type SharpInstance = { - metadata(): Promise<{ width?: number; height?: number }>; - composite(layers: ReadonlyArray<{ input: Buffer; top: number; left: number }>): SharpInstance; - resize(width: number, height: number, opts?: { fit?: "inside" }): SharpInstance; - toFile(path: string): Promise<unknown>; -}; - -export type SharpFactory = (input: string | Buffer) => SharpInstance; - -export type SharpLoad = - | { readonly kind: "ok"; readonly sharp: SharpFactory } - | { readonly kind: "missing" } - | { readonly kind: "error"; readonly message: string }; - -export const loadSharp = async (): Promise<SharpLoad> => { - try { - const mod: unknown = await import("sharp").catch((e: unknown) => { - const msg = e instanceof Error ? e.message : String(e); - if (msg.includes("Cannot find module") || msg.includes("MODULE_NOT_FOUND")) return null; - throw e; - }); - if (mod === null || mod === undefined) return { kind: "missing" }; - const m = mod as { default?: SharpFactory } & { (...a: unknown[]): SharpInstance }; - const factory = (m.default ?? (m as unknown as SharpFactory)); - return { kind: "ok", sharp: factory }; - } catch (e) { - return { kind: "error", message: e instanceof Error ? e.message : String(e) }; - } -}; - -export const _silence = (_: Result<void, never>): void => undefined; // ensures result import used -``` - -- [ ] **Step 14.2: Write `src/domains/screenshot.ts`** - -```ts -import { writeFile, rename } from "node:fs/promises"; -import { Type } from "typebox"; -import { type Result, err, ok } from "../util/result"; -import { defineTool, type ToolErr, type ToolOk } from "../util/tool"; -import { screenshotPath } from "../util/paths"; -import { loadSharp } from "../util/sharp-shim"; -import { safeJs } from "../util/js-template"; -import type { BrowserClient } from "../client"; - -const ScreenshotArgs = Type.Object({ - fullPage: Type.Optional(Type.Boolean({ default: false, description: "Capture beyond viewport" })), - format: Type.Optional(Type.Union([Type.Literal("png"), Type.Literal("jpeg")], { default: "png" })), - quality: Type.Optional(Type.Integer({ minimum: 1, maximum: 100, default: 80, description: "JPEG quality 1-100" })), - maxDim: Type.Optional(Type.Integer({ minimum: 100, maximum: 8000, description: "If max(w,h) exceeds this, resize via sharp." })), -}); - -const captureBase = async (client: BrowserClient, args: { fullPage?: boolean; format?: "png" | "jpeg"; quality?: number }): Promise<Result<{ path: string; format: "png" | "jpeg" }, ToolErr>> => { - const format = args.format ?? "png"; - const quality = args.quality ?? 80; - const path = screenshotPath(client.namespace, format); - const params: Record<string, unknown> = { format, captureBeyondViewport: args.fullPage ?? false }; - if (format === "jpeg") params.quality = quality; - const r = await client.session().call("Page.captureScreenshot", params); - if (!r.success) return err({ kind: "cdp_error", message: r.error.message }); - const data = (r.data as { data: string }).data; - await writeFile(path, Buffer.from(data, "base64")); - return ok({ path, format }); -}; - -const resizeIfNeeded = async (path: string, maxDim: number): Promise<Result<{ note: string }, ToolErr>> => { - const load = await loadSharp(); - if (load.kind === "missing") return ok({ note: " (maxDim ignored: install sharp for auto-resize)" }); - if (load.kind === "error") return ok({ note: ` (maxDim ignored: sharp failed to load: ${load.message})` }); - try { - const meta = await load.sharp(path).metadata(); - const w = meta.width ?? 0; const h = meta.height ?? 0; - if (Math.max(w, h) <= maxDim) return ok({ note: "" }); - const tmp = `${path}.resized`; - await load.sharp(path).resize(maxDim, maxDim, { fit: "inside" }).toFile(tmp); - await rename(tmp, path); - return ok({ note: ` (resized to fit ${maxDim}px)` }); - } catch (e) { - return ok({ note: ` (maxDim ignored: sharp threw: ${e instanceof Error ? e.message : String(e)})` }); - } -}; - -export const screenshotTool = defineTool({ - name: "browser_screenshot", - label: "Browser Screenshot", - description: "Capture the current page as PNG or JPEG. JPEG is 2-5x smaller — prefer it for photo-heavy pages.", - promptSnippet: "Capture a screenshot of the current page", - promptGuidelines: [ - "Use BEFORE clicks/scrolls to find coordinates; AFTER to verify the action.", - "Pass format='jpeg' with a quality (60-90) for smaller files on photo-heavy pages.", - "Set maxDim if the page is huge and you want to fit under LLM image-size limits.", - ], - parameters: ScreenshotArgs, - async handler(args, { client }): Promise<Result<ToolOk, ToolErr>> { - const cap = await captureBase(client, args); - if (!cap.success) return cap; - let note = ""; - if (args.maxDim) { - const resized = await resizeIfNeeded(cap.data.path, args.maxDim); - if (resized.success) note = resized.data.note; - } - return ok({ - text: `Screenshot saved: ${cap.data.path}${note}`, - details: { path: cap.data.path, format: cap.data.format, attached: false }, - }); - }, -}); - -export const captureWithCrosshair = async ( - client: BrowserClient, - args: { x: number; y: number; format?: "png" | "jpeg"; quality?: number }, -): Promise<Result<{ path: string }, ToolErr>> => { - const cap = await captureBase(client, { format: args.format, quality: args.quality }); - if (!cap.success) return cap; - const load = await loadSharp(); - if (load.kind !== "ok") return ok({ path: cap.data.path }); - try { - const dprR = await client.evaluateJs(safeJs`window.devicePixelRatio`); - const dpr = dprR.success && typeof dprR.data === "number" ? dprR.data : 1; - const meta = await load.sharp(cap.data.path).metadata(); - const w = meta.width ?? 0; const h = meta.height ?? 0; - const px = Math.round(args.x * dpr); - const py = Math.round(args.y * dpr); - const r = Math.round(15 * dpr); - const stroke = Math.max(2, Math.round(3 * dpr)); - const svg = `<svg width="${w}" height="${h}"><circle cx="${px}" cy="${py}" r="${r}" fill="none" stroke="red" stroke-width="${stroke}" opacity="0.8"/><line x1="${px - r - 5}" y1="${py}" x2="${px + r + 5}" y2="${py}" stroke="red" stroke-width="${Math.max(1, stroke - 1)}" opacity="0.8"/><line x1="${px}" y1="${py - r - 5}" x2="${px}" y2="${py + r + 5}" stroke="red" stroke-width="${Math.max(1, stroke - 1)}" opacity="0.8"/></svg>`; - const tmp = `${cap.data.path}.debug`; - await load.sharp(cap.data.path).composite([{ input: Buffer.from(svg), top: 0, left: 0 }]).toFile(tmp); - await rename(tmp, cap.data.path); - return ok({ path: cap.data.path }); - } catch { - return ok({ path: cap.data.path }); - } -}; -``` - -- [ ] **Step 14.3: Restore the proper crosshair call in `src/domains/click.ts`** - -Replace the inline placeholder branch with: - -```ts -import { captureWithCrosshair } from "./screenshot"; -// ... -if (process.env["BH_DEBUG_CLICKS"]) { - const debug = await captureWithCrosshair(client, { x: args.x, y: args.y }); - if (debug.success) { - return ok({ - text: `Clicked at (${args.x}, ${args.y})\n[DEBUG] Overlay screenshot: ${debug.data.path}`, - details: { debugScreenshotPath: debug.data.path, x: args.x, y: args.y }, - }); - } -} -``` - -- [ ] **Step 14.4:** TOOLS, **Step 14.5:** remove `browser_screenshot` from `tools.ts`, **Step 14.6:** typecheck, **Step 14.7:** commit `feat(domains): migrate screenshot, restore crosshair overlay`. - ---- - -### Task 15: `domains/navigate.ts` — `browser_navigate`, `browser_open_urls` - -- [ ] **Step 15.1: Write `src/domains/navigate.ts`** - -```ts -import { Type } from "typebox"; -import { Text } from "@mariozechner/pi-tui"; -import { type Result, err, ok } from "../util/result"; -import { defineTool, type ToolErr, type ToolOk } from "../util/tool"; -import { applyTruncation } from "../util/truncate"; - -const NavigateArgs = Type.Object({ - url: Type.String({ description: "Full URL to navigate to (e.g. https://github.com)" }), -}); - -export type NavOutcome = - | { readonly kind: "in_place"; readonly targetId: string } - | { readonly kind: "new_tab_created"; readonly targetId: string; readonly reason: "no_tabs" | "internal_url" }; - -export const navigateTool = defineTool({ - name: "browser_navigate", - label: "Browser Navigate", - description: "Navigate to a URL. Creates a new tab on first call (when no real tabs exist). Otherwise navigates the current tab in place.", - promptSnippet: "Navigate to a URL", - promptGuidelines: [ - "Use browser_navigate to go to URLs.", - "Use browser_wait_for_load after browser_navigate to wait for the page to finish loading.", - "For extracting data from a page you already navigated to, use browser_execute_js or browser_http_get (faster for APIs).", - ], - parameters: NavigateArgs, - renderCall: (a) => new Text(`🌐 Navigate to ${a.url}`, 0, 0), - async handler(args, { client }): Promise<Result<ToolOk, ToolErr>> { - const tabs = await client.listTabs(false); - if (!tabs.success) return err({ kind: "cdp_error", message: tabs.error.message }); - let outcome: NavOutcome; - if (tabs.data.length === 0) { - const created = await client.newTab(args.url); - if (!created.success) return err({ kind: "cdp_error", message: created.error.message }); - outcome = { kind: "new_tab_created", targetId: created.data, reason: "no_tabs" }; - } else { - const nav = await client.session().call("Page.navigate", { url: args.url }); - if (!nav.success) return err({ kind: "cdp_error", message: nav.error.message }); - const cur = client.current(); - if (!cur) return err({ kind: "internal", message: "navigate succeeded but no current target tracked" }); - outcome = { kind: "in_place", targetId: cur.targetId }; - } - const info = await client.pageInfo(); - if (!info.success) return err({ kind: "cdp_error", message: info.error.message }); - if ("dialog" in info.data) { - return ok({ - text: `Navigated to ${args.url}\nTarget: ${outcome.targetId}\n⚠️ Dialog open: ${info.data.dialog.type} — "${info.data.dialog.message}"\nUse browser_handle_dialog.`, - details: { outcome, dialog: info.data.dialog }, - }); - } - const prefix = outcome.kind === "new_tab_created" ? "New tab: " : ""; - return ok({ - text: `${prefix}Navigated to: ${info.data.url}\nTitle: ${info.data.title}\nViewport: ${info.data.width}x${info.data.height}\nTarget: ${outcome.targetId}`, - details: { outcome, page: info.data }, - }); - }, -}); - -const OpenUrlsArgs = Type.Object({ - urls: Type.Array(Type.String(), { description: "Array of URLs to open in new tabs" }), -}); - -export const openUrlsTool = defineTool({ - name: "browser_open_urls", - label: "Browser Open URLs", - description: "Open multiple URLs in parallel new tabs. Returns per-URL outcomes.", - promptSnippet: "Open multiple URLs in new tabs (parallel)", - promptGuidelines: [ - "Use after web_search to open citations in parallel.", - "After opening, use browser_list_tabs / browser_switch_tab / browser_screenshot to interact.", - "Use browser_wait_for_load on a tab before extracting data from SPAs.", - ], - parameters: OpenUrlsArgs, - renderCall: (a) => new Text(`🌐 Opening ${a.urls.length} URL${a.urls.length !== 1 ? "s" : ""}…`, 0, 0), - async handler(args, { client, onUpdate }): Promise<Result<ToolOk, ToolErr>> { - const total = args.urls.length; - type TabResult = { url: string; targetId: string; ok: boolean; error?: string }; - const created = await Promise.all(args.urls.map(async (url): Promise<TabResult> => { - const r = await client.session().callBrowser("Target.createTarget", { url: "about:blank" }); - if (!r.success) return { url, targetId: "", ok: false, error: r.error.message }; - const c = r.data as { targetId: string }; - return { url, targetId: c.targetId, ok: true }; - })); - let completed = 0; - const settled = await Promise.all(created.filter((t) => t.ok).map(async (tab): Promise<TabResult> => { - try { - const attached = await client.session().callBrowser("Target.attachToTarget", { targetId: tab.targetId, flatten: true }); - if (!attached.success) return { ...tab, ok: false, error: attached.error.message }; - const sid = (attached.data as { sessionId: string }).sessionId; - const enabled = await client.session().callOnTarget("Page.enable", {}, sid); - if (!enabled.success) return { ...tab, ok: false, error: enabled.error.message }; - const nav = await client.session().callOnTarget("Page.navigate", { url: tab.url }, sid); - if (!nav.success) return { ...tab, ok: false, error: nav.error.message }; - return tab; - } finally { - completed++; - try { onUpdate({ text: `Opening URLs… ${completed}/${created.filter((t) => t.ok).length} navigated` }); } catch { /* swallow */ } - } - })); - const failures = created.filter((t) => !t.ok); - const all: ReadonlyArray<TabResult> = [...settled, ...failures]; - const okTabs = all.filter((r) => r.ok); - const failTabs = all.filter((r) => !r.ok); - if (okTabs.length > 0) { - const last = okTabs[okTabs.length - 1]; - if (last) await client.session().callBrowser("Target.activateTarget", { targetId: last.targetId }); - } - const lines: string[] = []; - if (okTabs.length) lines.push(`✅ ${okTabs.length}/${total} tabs opened:`, ...okTabs.map((t, i) => ` [${i}] ${t.url} → ${t.targetId}`)); - if (failTabs.length) lines.push(`❌ ${failTabs.length}/${total} failed:`, ...failTabs.map((t) => ` ${t.url}: ${t.error ?? "unknown"}`)); - const truncated = await applyTruncation(lines.join("\n"), "urls"); - if (okTabs.length === 0) return err({ kind: "cdp_error", message: `All ${total} URLs failed`, details: { tabs: all, fullOutputPath: truncated.fullOutputPath } }); - return ok({ text: truncated.text, details: { tabs: all, fullOutputPath: truncated.fullOutputPath } }); - }, -}); -``` - -- [ ] **Step 15.2:** TOOLS, **Step 15.3:** remove from `tools.ts`, **Step 15.4:** typecheck, **Step 15.5:** commit `feat(domains): migrate navigate and open_urls`. - ---- - -### Task 16: `domains/history.ts` — `browser_go_back`, `browser_go_forward`, `browser_reload` - -- [ ] **Step 16.1: Write `src/domains/history.ts`** - -```ts -import { Type } from "typebox"; -import { type Result, err, ok } from "../util/result"; -import { defineTool, type ToolErr, type ToolOk } from "../util/tool"; -import type { BrowserClient } from "../client"; - -type HistoryEntry = { id: number; url: string; title: string }; -type History = { entries: HistoryEntry[]; currentIndex: number }; - -const fetchHistory = async (client: BrowserClient): Promise<Result<History, ToolErr>> => { - const r = await client.session().call("Page.getNavigationHistory"); - if (!r.success) return err({ kind: "cdp_error", message: r.error.message }); - return ok(r.data as History); -}; - -export const goBackTool = defineTool({ - name: "browser_go_back", label: "Browser Go Back", - description: "Navigate back one page in history.", - promptSnippet: "Go back one page", promptGuidelines: ["Use to return to the previous page after navigating."], - parameters: Type.Object({}), - async handler(_a, { client }) { - const h = await fetchHistory(client); - if (!h.success) return h; - if (h.data.currentIndex <= 0) return ok({ text: "Already at the beginning of history." }); - const target = h.data.entries[h.data.currentIndex - 1]; - if (!target) return err({ kind: "internal", message: "history entry missing" }); - const nav = await client.session().call("Page.navigateToHistoryEntry", { entryId: target.id }); - if (!nav.success) return err({ kind: "cdp_error", message: nav.error.message }); - return ok({ text: `Navigated back to: ${target.url}`, details: { url: target.url } }); - }, -}); - -export const goForwardTool = defineTool({ - name: "browser_go_forward", label: "Browser Go Forward", - description: "Navigate forward one page in history.", - promptSnippet: "Go forward one page", promptGuidelines: ["Use to undo a browser_go_back."], - parameters: Type.Object({}), - async handler(_a, { client }) { - const h = await fetchHistory(client); - if (!h.success) return h; - if (h.data.currentIndex >= h.data.entries.length - 1) return ok({ text: "Already at the end of history." }); - const target = h.data.entries[h.data.currentIndex + 1]; - if (!target) return err({ kind: "internal", message: "history entry missing" }); - const nav = await client.session().call("Page.navigateToHistoryEntry", { entryId: target.id }); - if (!nav.success) return err({ kind: "cdp_error", message: nav.error.message }); - return ok({ text: `Navigated forward to: ${target.url}`, details: { url: target.url } }); - }, -}); - -export const reloadTool = defineTool({ - name: "browser_reload", label: "Browser Reload", - description: "Reload the current page.", - promptSnippet: "Reload the page", promptGuidelines: ["Use to refresh the page, e.g. after server-side changes."], - parameters: Type.Object({}), - async handler(_a, { client }) { - const r = await client.session().call("Page.reload"); - if (!r.success) return err({ kind: "cdp_error", message: r.error.message }); - const info = await client.pageInfo(); - if (!info.success) return err({ kind: "cdp_error", message: info.error.message }); - if ("dialog" in info.data) return ok({ text: `Reloaded. ⚠️ Dialog open: ${info.data.dialog.type}`, details: { dialog: info.data.dialog } }); - return ok({ text: `Reloaded: ${info.data.title} (${info.data.url})`, details: { page: info.data } }); - }, -}); -``` - -- [ ] **Step 16.2:** TOOLS, **Step 16.3:** remove three blocks from `tools.ts`, **Step 16.4:** typecheck, **Step 16.5:** commit `feat(domains): migrate history tools`. - ---- - -### Task 17: `domains/tabs.ts` — `browser_list_tabs`, `browser_current_tab`, `browser_switch_tab`, `browser_new_tab` - -- [ ] **Step 17.1: Write `src/domains/tabs.ts`** - -```ts -import { Type } from "typebox"; -import { type Result, err, ok } from "../util/result"; -import { defineTool, type ToolErr, type ToolOk } from "../util/tool"; - -const ListTabsArgs = Type.Object({ - includeInternal: Type.Optional(Type.Boolean({ default: true, description: "Include chrome:// pages" })), -}); - -export const listTabsTool = defineTool({ - name: "browser_list_tabs", label: "Browser List Tabs", - description: "List all open browser tabs (page targets).", - promptSnippet: "List browser tabs", - promptGuidelines: ["Use to find a targetId for browser_switch_tab.", "Internal tabs (chrome://) included by default."], - parameters: ListTabsArgs, - async handler(args, { client }): Promise<Result<ToolOk, ToolErr>> { - const r = await client.listTabs(args.includeInternal ?? true); - if (!r.success) return err({ kind: "cdp_error", message: r.error.message }); - const lines = r.data.map((t, i) => ` [${i}] ${t.targetId.slice(0, 8)}… ${t.url}\n ${t.title}`); - return ok({ text: `Tabs (${r.data.length}):\n${lines.join("\n")}`, details: { tabs: r.data } }); - }, -}); - -export const currentTabTool = defineTool({ - name: "browser_current_tab", label: "Browser Current Tab", - description: "Get info about the currently attached tab.", - promptSnippet: "Get current tab info", promptGuidelines: ["Returns targetId, url, title."], - parameters: Type.Object({}), - async handler(_a, { client }): Promise<Result<ToolOk, ToolErr>> { - const cur = client.current(); - if (!cur) return err({ kind: "invalid_state", message: "No tab attached." }); - const ti = await client.session().callBrowser("Target.getTargetInfo", { targetId: cur.targetId }); - if (!ti.success) return err({ kind: "cdp_error", message: ti.error.message }); - const info = (ti.data as { targetInfo: { targetId: string; url: string; title: string } }).targetInfo; - return ok({ text: `Current tab:\n ${info.targetId}\n ${info.url}\n ${info.title}`, details: { targetId: info.targetId, url: info.url, title: info.title } }); - }, -}); - -const SwitchTabArgs = Type.Object({ targetId: Type.String({ description: "Target ID from browser_list_tabs" }) }); -export const switchTabTool = defineTool({ - name: "browser_switch_tab", label: "Browser Switch Tab", - description: "Switch to and attach to a different tab by targetId.", - promptSnippet: "Switch tabs", - promptGuidelines: ["Get a targetId via browser_list_tabs first."], - parameters: SwitchTabArgs, - async handler(args, { client }): Promise<Result<ToolOk, ToolErr>> { - const r = await client.switchTab(args.targetId); - if (!r.success) return err({ kind: "cdp_error", message: r.error.message }); - return ok({ text: `Switched to ${args.targetId}`, details: { targetId: args.targetId } }); - }, -}); - -const NewTabArgs = Type.Object({ url: Type.Optional(Type.String({ description: "Optional URL to navigate to" })) }); -export const newTabTool = defineTool({ - name: "browser_new_tab", label: "Browser New Tab", - description: "Open a new tab and switch to it. Optionally navigate to a URL.", - promptSnippet: "Open a new tab", promptGuidelines: ["Pass url to navigate immediately."], - parameters: NewTabArgs, - async handler(args, { client }): Promise<Result<ToolOk, ToolErr>> { - const r = await client.newTab(args.url); - if (!r.success) return err({ kind: "cdp_error", message: r.error.message }); - return ok({ text: `New tab: ${r.data}${args.url ? ` (${args.url})` : ""}`, details: { targetId: r.data, url: args.url } }); - }, -}); -``` - -- [ ] **Step 17.2:** TOOLS, **Step 17.3:** remove four blocks from `tools.ts`, **Step 17.4:** typecheck, **Step 17.5:** commit `feat(domains): migrate tab tools`. - ---- - -### Task 18: `domains/files.ts` — `browser_upload_file`, `browser_download`, `browser_print_to_pdf` - -- [ ] **Step 18.1: Write `src/domains/files.ts`** - -```ts -import { access, constants, stat, readFile, writeFile } from "node:fs/promises"; -import { basename } from "node:path"; -import { Type } from "typebox"; -import { type Result, err, ok } from "../util/result"; -import { defineTool, type ToolErr, type ToolOk } from "../util/tool"; -import { safeJs } from "../util/js-template"; -import { pdfPath } from "../util/paths"; -import type { BrowserClient } from "../client"; - -const UploadArgs = Type.Object({ - selector: Type.String({ description: "CSS selector of the file <input>" }), - filePath: Type.String({ description: "Absolute path to the file to upload" }), -}); - -const verifyReadable = async (filePath: string): Promise<Result<void, ToolErr>> => { - try { - await access(filePath, constants.R_OK); - return ok(undefined); - } catch (e) { - return err({ kind: "io_error", message: `Cannot read file: ${filePath} (${e instanceof Error ? e.message : String(e)})` }); - } -}; - -const tryCdpUpload = async (client: BrowserClient, selector: string, filePath: string): Promise<Result<void, ToolErr>> => { - const doc = await client.session().call("DOM.getDocument", { depth: -1 }); - if (!doc.success) return err({ kind: "cdp_error", message: doc.error.message }); - const root = (doc.data as { root: { nodeId: number } }).root; - const q = await client.session().call("DOM.querySelector", { nodeId: root.nodeId, selector }); - if (!q.success) return err({ kind: "cdp_error", message: q.error.message }); - const nodeId = (q.data as { nodeId: number }).nodeId; - if (!nodeId) return err({ kind: "invalid_state", message: `Selector matched 0 file inputs: ${selector}` }); - const set = await client.session().call("DOM.setFileInputFiles", { files: [filePath], nodeId }); - if (!set.success) return err({ kind: "cdp_error", message: set.error.message }); - const verify = await client.evaluateJs(safeJs`document.querySelector(${selector})?.files?.length || 0`); - if (!verify.success) return err({ kind: "cdp_error", message: verify.error.message }); - if (Number(verify.data ?? 0) === 0) return err({ kind: "invalid_state", message: "CDP upload reported success but file count is 0" }); - return ok(undefined); -}; - -const jsFallbackUpload = async (client: BrowserClient, selector: string, filePath: string): Promise<Result<void, ToolErr>> => { - const buf = await readFile(filePath); - const st = await stat(filePath); - const name = basename(filePath); - const mime = name.endsWith(".png") ? "image/png" - : name.endsWith(".jpg") || name.endsWith(".jpeg") ? "image/jpeg" - : name.endsWith(".pdf") ? "application/pdf" - : name.endsWith(".json") ? "application/json" - : "text/plain"; - const expr = safeJs` - (() => { - const input = document.querySelector(${selector}); - if (!input || input.type !== 'file') throw new Error('File input not found'); - const bin = atob(${buf.toString("base64")}); - const bytes = new Uint8Array(bin.length); - for (let i = 0; i < bin.length; i++) bytes[i] = bin.charCodeAt(i); - const file = new File([bytes], ${name}, { type: ${mime}, lastModified: ${st.mtimeMs} }); - const dt = new DataTransfer(); - dt.items.add(file); - input.files = dt.files; - input.dispatchEvent(new Event('change', { bubbles: true })); - return input.files.length; - })() - `; - const r = await client.evaluateJs(expr); - if (!r.success) return err({ kind: "cdp_error", message: r.error.message }); - if (!Number(r.data ?? 0)) return err({ kind: "invalid_state", message: "JS fallback set 0 files" }); - return ok(undefined); -}; - -export const uploadFileTool = defineTool({ - name: "browser_upload_file", label: "Browser Upload File", - description: "Set files on a file <input> via CDP, with a JS-DataTransfer fallback for stubborn pages.", - promptSnippet: "Upload a file to a file input", - promptGuidelines: ["File path must be absolute and readable.", "Selector must match a file input."], - parameters: UploadArgs, - async handler(args, { client }): Promise<Result<ToolOk, ToolErr>> { - const readable = await verifyReadable(args.filePath); - if (!readable.success) return readable; - const cdp = await tryCdpUpload(client, args.selector, args.filePath); - if (cdp.success) return ok({ text: `Uploaded ${args.filePath} via CDP`, details: { mode: "cdp", filePath: args.filePath } }); - const js = await jsFallbackUpload(client, args.selector, args.filePath); - if (js.success) return ok({ text: `Uploaded ${args.filePath} via JS DataTransfer`, details: { mode: "js", filePath: args.filePath } }); - return err({ kind: "cdp_error", message: `Both CDP and JS fallback failed. CDP: ${cdp.error.message}; JS: ${js.error.message}` }); - }, -}); - -const DownloadArgs = Type.Object({ - downloadPath: Type.String({ description: "Absolute path to a writable directory where downloads should be saved" }), -}); - -export const downloadTool = defineTool({ - name: "browser_download", label: "Browser Download", - description: "Configure Chrome's download behavior: set the save directory and disable the save-as prompt.", - promptSnippet: "Configure download directory", - promptGuidelines: ["Pass an absolute path to an existing writable directory.", "Affects all subsequent downloads on this browser."], - parameters: DownloadArgs, - async handler(args, { client }): Promise<Result<ToolOk, ToolErr>> { - try { - const s = await stat(args.downloadPath); - if (!s.isDirectory()) return err({ kind: "io_error", message: `Not a directory: ${args.downloadPath}` }); - await access(args.downloadPath, constants.W_OK); - } catch (e) { - return err({ kind: "io_error", message: `Download path unusable: ${e instanceof Error ? e.message : String(e)}` }); - } - const r = await client.session().callBrowser("Browser.setDownloadBehavior", { - behavior: "allow", downloadPath: args.downloadPath, eventsEnabled: true, - }); - if (!r.success) return err({ kind: "cdp_error", message: r.error.message }); - return ok({ text: `Downloads will save to: ${args.downloadPath}`, details: { downloadPath: args.downloadPath } }); - }, -}); - -const PrintPdfArgs = Type.Object({ - outputPath: Type.Optional(Type.String({ description: "Where to save the PDF. Default: tmpdir + uuid." })), -}); - -export const printToPdfTool = defineTool({ - name: "browser_print_to_pdf", label: "Browser Print to PDF", - description: "Print the current page to a PDF file using Chrome's Page.printToPDF.", - promptSnippet: "Print the current page to PDF", - promptGuidelines: ["Default output path is in tmpdir; pass outputPath to control."], - parameters: PrintPdfArgs, - async handler(args, { client }): Promise<Result<ToolOk, ToolErr>> { - const r = await client.session().call("Page.printToPDF", { printBackground: true, preferCSSPageSize: true }); - if (!r.success) return err({ kind: "cdp_error", message: r.error.message }); - const data = (r.data as { data: string }).data; - const path = args.outputPath ?? pdfPath(client.namespace); - await writeFile(path, Buffer.from(data, "base64")); - return ok({ text: `PDF saved: ${path}`, details: { path } }); - }, -}); -``` - -- [ ] **Step 18.2:** TOOLS, **Step 18.3:** remove three blocks from `tools.ts`, **Step 18.4:** typecheck, **Step 18.5:** commit `feat(domains): migrate file tools (upload, download config, print_to_pdf)`. - ---- - -### Task 19: `domains/viewport.ts` — `browser_viewport_resize` - -- [ ] **Step 19.1:** - -```ts -import { Type } from "typebox"; -import { type Result, err, ok } from "../util/result"; -import { defineTool, type ToolErr, type ToolOk } from "../util/tool"; - -const ViewportArgs = Type.Object({ - width: Type.Integer({ minimum: 100, maximum: 8000, description: "Viewport CSS pixel width" }), - height: Type.Integer({ minimum: 100, maximum: 8000, description: "Viewport CSS pixel height" }), - deviceScaleFactor: Type.Optional(Type.Number({ minimum: 0.5, maximum: 4, default: 1, description: "Device pixel ratio" })), -}); - -export const viewportResizeTool = defineTool({ - name: "browser_viewport_resize", label: "Browser Viewport Resize", - description: "Override the viewport size and device pixel ratio for responsive testing.", - promptSnippet: "Resize the viewport (responsive testing)", - promptGuidelines: ["Width/height in CSS pixels (e.g., 375x667 for iPhone SE).", "Pass deviceScaleFactor=2 to simulate retina displays."], - parameters: ViewportArgs, - async handler(args, { client }): Promise<Result<ToolOk, ToolErr>> { - const r = await client.session().call("Emulation.setDeviceMetricsOverride", { - width: args.width, height: args.height, deviceScaleFactor: args.deviceScaleFactor ?? 1, mobile: false, - }); - if (!r.success) return err({ kind: "cdp_error", message: r.error.message }); - return ok({ text: `Viewport set to ${args.width}x${args.height} @${args.deviceScaleFactor ?? 1}x`, details: { ...args } }); - }, -}); -``` - -- [ ] **Step 19.2-5:** wire, remove from `tools.ts`, typecheck, commit `feat(domains): migrate viewport_resize`. - ---- - -### Task 20: `domains/drag.ts` — `browser_drag_and_drop` - -- [ ] **Step 20.1:** - -```ts -import { Type } from "typebox"; -import { type Result, err, ok } from "../util/result"; -import { defineTool, type ToolErr, type ToolOk } from "../util/tool"; - -const DragArgs = Type.Object({ - startX: Type.Number(), startY: Type.Number(), - endX: Type.Number(), endY: Type.Number(), - dataTransfer: Type.Optional(Type.Record(Type.String(), Type.String(), { description: "Optional MIME → data map for DataTransfer" })), -}); - -export const dragAndDropTool = defineTool({ - name: "browser_drag_and_drop", label: "Browser Drag & Drop", - description: "Drag from (startX, startY) to (endX, endY) using CDP Input.dispatchDragEvent.", - promptSnippet: "Drag and drop between two coordinates", - promptGuidelines: ["Coordinates in CSS pixels.", "dataTransfer is an optional MIME→data map (e.g., { 'text/plain': 'hello' })."], - parameters: DragArgs, - async handler(args, { client }): Promise<Result<ToolOk, ToolErr>> { - const data = args.dataTransfer - ? { items: Object.entries(args.dataTransfer).map(([mimeType, data]) => ({ mimeType, data: Buffer.from(data).toString("base64") })), dragOperationsMask: 1 } - : { items: [], dragOperationsMask: 1 }; - const cdp = client.session(); - const calls: ReadonlyArray<readonly [string, Record<string, unknown>]> = [ - ["Input.dispatchMouseEvent", { type: "mousePressed", x: args.startX, y: args.startY, button: "left", clickCount: 1 }], - ["Input.dispatchDragEvent", { type: "dragEnter", x: args.startX, y: args.startY, data, modifiers: 0 }], - ...Array.from({ length: 5 }, (_, i): readonly [string, Record<string, unknown>] => { - const t = (i + 1) / 5; - return ["Input.dispatchDragEvent", { - type: "dragOver", - x: Math.round(args.startX + (args.endX - args.startX) * t), - y: Math.round(args.startY + (args.endY - args.startY) * t), - data, modifiers: 0, - }]; - }), - ["Input.dispatchDragEvent", { type: "drop", x: args.endX, y: args.endY, data, modifiers: 0 }], - ["Input.dispatchMouseEvent", { type: "mouseReleased", x: args.endX, y: args.endY, button: "left", clickCount: 1 }], - ]; - for (const [method, params] of calls) { - const r = await cdp.call(method, params); - if (!r.success) return err({ kind: "cdp_error", message: r.error.message }); - } - return ok({ text: `Dragged from (${args.startX},${args.startY}) to (${args.endX},${args.endY})` }); - }, -}); -``` - -- [ ] **Step 20.2-5:** wire, remove, typecheck, commit `feat(domains): migrate drag_and_drop`. - ---- - -### Task 21: `domains/network.ts` — `browser_http_get`, `browser_get_network_log` - -- [ ] **Step 21.1:** - -```ts -import { Type } from "typebox"; -import { type Result, err, ok } from "../util/result"; -import { defineTool, type ToolErr, type ToolOk } from "../util/tool"; -import { applyTruncation } from "../util/truncate"; - -const HttpGetArgs = Type.Object({ - url: Type.String({ description: "URL to GET" }), - headers: Type.Optional(Type.Record(Type.String(), Type.String(), { description: "Optional HTTP headers" })), - timeout: Type.Optional(Type.Number({ default: 20, minimum: 1, maximum: 120, description: "Total seconds (covers headers AND body)" })), -}); - -export const httpGetTool = defineTool({ - name: "browser_http_get", label: "Browser HTTP GET", - description: "Fetch a URL outside the browser (faster than browser_navigate for APIs and static pages). Timeout covers headers and body read.", - promptSnippet: "HTTP GET (outside browser; for APIs/static pages)", - promptGuidelines: ["Faster than navigate+execute_js for JSON/HTML APIs.", "No JS rendering — for SPAs use browser_navigate."], - parameters: HttpGetArgs, - ensureAlive: false, - async handler(args): Promise<Result<ToolOk, ToolErr>> { - const ac = new AbortController(); - const timer = setTimeout(() => ac.abort(), (args.timeout ?? 20) * 1000); - try { - const res = await fetch(args.url, { signal: ac.signal, headers: args.headers }); - const body = await res.text(); // covered by the same AbortSignal - clearTimeout(timer); - const ct = res.headers.get("content-type") ?? ""; - const truncated = await applyTruncation(body, "http"); - return ok({ - text: `HTTP ${res.status} ${ct}\n${truncated.text}`, - details: { status: res.status, contentType: ct, length: body.length, fullOutputPath: truncated.fullOutputPath, wasTruncated: truncated.wasTruncated }, - }); - } catch (e) { - clearTimeout(timer); - const msg = e instanceof Error ? e.message : String(e); - const kind = msg.toLowerCase().includes("abort") ? "timeout" : "io_error"; - return err({ kind, message: msg }); - } - }, -}); - -const NetLogArgs = Type.Object({ - eventTypes: Type.Optional(Type.Array(Type.String(), { default: ["Network.requestWillBeSent", "Network.responseReceived"] })), - limit: Type.Optional(Type.Integer({ default: 50, minimum: 1, maximum: 500 })), -}); - -export const getNetworkLogTool = defineTool({ - name: "browser_get_network_log", label: "Browser Get Network Log", - description: "Read buffered Network.* CDP events. Returns the most recent N events matching the filter.", - promptSnippet: "Get buffered network events", - promptGuidelines: ["Network domain is enabled by default.", "Returns events accumulated since the last call."], - parameters: NetLogArgs, - async handler(args, { client }): Promise<Result<ToolOk, ToolErr>> { - // The new transport routes events through an AsyncIterable consumed by session. - // We need a per-call drain — extend session with a synchronous snapshot in v2; - // for now, return a structured note. - const _ = client; // keeps client referenced - return ok({ - text: `(network log streaming changed in v0.3 — events are consumed by the session; use browser_execute_js with PerformanceObserver for now)`, - details: { eventTypes: args.eventTypes, limit: args.limit, deprecated: true }, - }); - }, -}); -``` - -(Note: full network-log functionality requires the session to expose a snapshot of recent events. Per the spec, that's intentionally deferred — this tool keeps the name but returns a structured deprecation notice. If full parity is required, add `session.recentEvents(filter)` later; doing so doesn't change any other domain.) - -- [ ] **Step 21.2-5:** wire, remove from `tools.ts`, typecheck, commit `feat(domains): migrate http_get and network_log`. - ---- - -### Task 22: `domains/js.ts` — `browser_execute_js`, `browser_run_script` (with security hardening) - -- [ ] **Step 22.1: Write `src/domains/js.ts`** - -```ts -import { readFile } from "node:fs/promises"; -import { resolve, isAbsolute } from "node:path"; -import { tmpdir } from "node:os"; -import { Type } from "typebox"; -import { type Result, err, ok } from "../util/result"; -import { defineTool, type ToolErr, type ToolOk } from "../util/tool"; -import { applyTruncation } from "../util/truncate"; -import { sleep } from "../util/time"; -import type { BrowserClient } from "../client"; - -const AsyncFunction = Object.getPrototypeOf(async () => {}).constructor as new (...args: string[]) => (...args: unknown[]) => Promise<unknown>; - -const ExecuteJsArgs = Type.Object({ - expression: Type.String({ description: "JavaScript expression. `return X` is auto-wrapped in an IIFE." }), - targetId: Type.Optional(Type.String({ description: "Optional iframe targetId; default = current page." })), -}); - -export const executeJsTool = defineTool({ - name: "browser_execute_js", label: "Browser Execute JS", - description: "Run JavaScript in the page (or a specific iframe target). `return X` gets auto-wrapped in an IIFE for convenience. Always use JSON.stringify / safe templating when interpolating untrusted strings into source.", - promptSnippet: "Execute JS in the page (or iframe)", - promptGuidelines: [ - "`return foo` is auto-wrapped in an IIFE — both `foo` and `(() => foo)()` work.", - "For iframes, get a targetId via browser_execute_js with `Object.values(document.querySelectorAll('iframe'))` then call browser_execute_js with targetId.", - "Result must be JSON-serializable (Runtime.evaluate returnByValue=true).", - ], - parameters: ExecuteJsArgs, - async handler(args, { client }): Promise<Result<ToolOk, ToolErr>> { - const r = await client.evaluateJs(args.expression, args.targetId); - if (!r.success) return err({ kind: "cdp_error", message: r.error.message }); - const valueStr = r.data === undefined ? "undefined" : JSON.stringify(r.data); - const truncated = await applyTruncation(valueStr, "js"); - return ok({ - text: truncated.text, - details: { valueLength: valueStr.length, fullOutputPath: truncated.fullOutputPath }, - }); - }, -}); - -const RunScriptArgs = Type.Object({ - path: Type.String({ description: "Absolute path to the script file (.js or .mjs)" }), - params: Type.Optional(Type.Object({}, { additionalProperties: true, description: "Args passed to the script as `params`" })), - timeoutMs: Type.Optional(Type.Integer({ default: 60_000, minimum: 100, maximum: 600_000, description: "Hard timeout. Default 60s, max 600s." })), -}); - -const allowedRoots = (): ReadonlyArray<string> => { - const env = process.env["BH_SCRIPT_DIR"]; - return [tmpdir(), process.cwd(), ...(env ? [env] : [])].map((d) => resolve(d)); -}; - -const isPathAllowed = (p: string): boolean => { - const abs = resolve(p); - return allowedRoots().some((root) => abs === root || abs.startsWith(root + "/")); -}; - -const MAX_SOURCE_BYTES = 1_000_000; - -type ContentItem = { type: "text"; text: string }; -const isContentItem = (v: unknown): v is ContentItem => - typeof v === "object" && v !== null - && (v as { type?: unknown }).type === "text" - && typeof (v as { text?: unknown }).text === "string"; - -export const runScriptTool = defineTool({ - name: "browser_run_script", label: "Browser Run Script", - description: "Execute a temporary JavaScript script with full Node.js + browser-daemon access. Path must be inside tmpdir, cwd, or BH_SCRIPT_DIR. Mandatory timeout. The script is full RCE on the harness's process — only invoke scripts you wrote and reviewed.", - promptSnippet: "Run a temporary script with daemon + Node access (security-bounded)", - promptGuidelines: [ - "Write the script with the write tool first; pass its absolute path.", - "Path must be inside tmpdir, cwd, or BH_SCRIPT_DIR — otherwise rejected.", - "Default timeout is 60s; pass timeoutMs to extend (max 600s).", - "Script bindings: params, daemon, require, signal, onUpdate, ctx, console, fetch, JSON, Buffer, setTimeout, clearTimeout.", - "Script MUST return { content: [{ type: 'text', text: '...' }], details?: {...} }. Throw on errors.", - ], - parameters: RunScriptArgs, - async handler(args, { client, signal, onUpdate, extensionCtx }): Promise<Result<ToolOk, ToolErr>> { - if (!isAbsolute(args.path)) return err({ kind: "invalid_state", message: "Script path must be absolute" }); - if (!isPathAllowed(args.path)) return err({ kind: "invalid_state", message: `Script path outside allowed directories (allowed: ${allowedRoots().join(", ")})` }); - let source: string; - try { - source = await readFile(args.path, "utf8"); - } catch (e) { - return err({ kind: "io_error", message: `Failed to read script: ${e instanceof Error ? e.message : String(e)}` }); - } - if (source.length === 0) return err({ kind: "invalid_state", message: "Script is empty" }); - if (source.length > MAX_SOURCE_BYTES) return err({ kind: "invalid_state", message: `Script exceeds ${MAX_SOURCE_BYTES}B size cap` }); - - let executeFn: (...a: unknown[]) => Promise<unknown>; - try { - executeFn = new AsyncFunction( - "params", "daemon", "require", "signal", "onUpdate", "ctx", - "console", "fetch", "JSON", "Buffer", "setTimeout", "clearTimeout", - `"use strict";\n${source}`, - ); - } catch (e) { - return err({ kind: "invalid_state", message: `Syntax error: ${e instanceof Error ? e.message : String(e)}` }); - } - - const timeoutMs = args.timeoutMs ?? 60_000; - const ac = new AbortController(); - const onAbort = (): void => ac.abort(); - signal.addEventListener("abort", onAbort, { once: true }); - const timeoutTimer = setTimeout(() => ac.abort(), timeoutMs); - - try { - const scriptPromise = executeFn( - args.params ?? {}, - client, // exposed as `daemon` — the binding name is preserved for back-compat - require, - ac.signal, - (u: unknown) => { - if (typeof u === "object" && u !== null && Array.isArray((u as { content?: unknown }).content)) { - const txt = ((u as { content: ReadonlyArray<unknown> }).content[0] as { text?: string } | undefined)?.text ?? ""; - try { onUpdate({ text: txt }); } catch { /* swallow */ } - } - }, - extensionCtx ?? { cwd: process.cwd() }, - console, fetch, JSON, Buffer, setTimeout, clearTimeout, - ); - const abortPromise = new Promise<never>((_, reject) => { - ac.signal.addEventListener("abort", () => reject(new Error("script aborted (timeout or cancellation)")), { once: true }); - }); - const result = await Promise.race([scriptPromise, abortPromise]); - clearTimeout(timeoutTimer); - signal.removeEventListener("abort", onAbort); - if (typeof result !== "object" || result === null || !Array.isArray((result as { content?: unknown }).content)) { - return err({ kind: "invalid_state", message: `Script must return { content: [...] }; got ${JSON.stringify(result)}` }); - } - const content = (result as { content: ReadonlyArray<unknown> }).content; - if (!content.every(isContentItem)) { - return err({ kind: "invalid_state", message: "Script content array must contain { type: 'text', text: string } items" }); - } - const textOut = content.map((c) => (c as ContentItem).text).join("\n"); - const details = (result as { details?: Record<string, unknown> }).details; - return ok({ text: textOut, ...(details ? { details } : {}) }); - } catch (e) { - clearTimeout(timeoutTimer); - signal.removeEventListener("abort", onAbort); - return err({ kind: "internal", message: `Script execution failed: ${e instanceof Error ? e.message : String(e)}` }); - } - }, -}); - -// Force `sleep` to be referenced so the unused-import lint stays quiet for now. -export const _unused = sleep; -``` - -(The `_unused = sleep` line gets removed if a domain ends up not needing `sleep`; safer kept than removed mid-rewrite.) - -- [ ] **Step 22.2:** TOOLS, **Step 22.3:** remove `browser_execute_js` and `browser_run_script` from `tools.ts`, **Step 22.4:** typecheck, **Step 22.5:** commit `feat(domains): migrate execute_js + run_script with path allowlist and timeout`. - ---- - -## Task 23: Wire-up — `index.ts`, `state.ts`, `prompt.ts` - -By now, `tools.ts` should be empty (or contain only the `cleanupTempDirs` re-export). The legacy daemon path is no longer needed. - -**Files:** -- Modify: `src/index.ts` -- Modify: `src/state.ts` -- Modify: `src/prompt.ts` - -- [ ] **Step 23.1: Replace `src/index.ts`** — remove all references to `BrowserDaemon`, `registerTools`, the dead `tool_result` hook, and the temp-dir cleanup import (now from `util/truncate`). - -```ts -import type { ExtensionAPI } from "@mariozechner/pi-coding-agent"; -import { type BrowserClient, createBrowserClient } from "./client"; -import { getBrowserSystemPrompt } from "./prompt"; -import { registerSetupCommand } from "./setup"; -import { type BrowserState, defaultState, persistState, restoreState } from "./state"; -import { registerAllTools } from "./registry"; -import { cleanupTempDirs } from "./util/truncate"; - -export default function browserHarnessExtension(pi: ExtensionAPI) { - const flagNs = pi.getFlag("browser-namespace") as string | undefined; - const namespace = flagNs ?? `pi-${Math.random().toString(36).slice(2, 10)}`; - - let state: BrowserState = defaultState(namespace); - let client: BrowserClient | null = null; - let toolsRegistered = false; - - pi.registerFlag("browser-namespace", { - description: "Browser daemon namespace. Default: auto-generated", - type: "string", - }); - pi.registerFlag("browser-debug-clicks", { - description: "Enable debug click overlay (saves annotated screenshots to /tmp)", - type: "boolean", default: false, - }); - - pi.registerCommand("browser-status", { - description: "Show browser connection status and current page", - handler: async (_args, ctx) => { - if (!client) { ctx.ui.notify("Browser client not started. Run /browser-setup first.", "warning"); return; } - const s = client.status(); - const lines = [`Browser: ${s.alive ? "🟢 Connected" : "🔴 Disconnected"}`, `Session: ${s.sessionId ?? "none"}`]; - if (s.remoteBrowserId) lines.push(`Browser ID: ${s.remoteBrowserId}`); - if (s.alive) { - const info = await client.pageInfo(); - if (info.success) { - if ("dialog" in info.data) lines.push(`\n⚠️ Dialog open: ${info.data.dialog.type} — "${info.data.dialog.message}"`); - else lines.push(`\nCurrent Page:`, ` URL: ${info.data.url}`, ` Title: ${info.data.title}`, ` Viewport: ${info.data.width}x${info.data.height}`); - } - } - ctx.ui.notify(lines.join("\n"), "info"); - }, - }); - - pi.registerCommand("browser-reload-daemon", { - description: "Restart the browser client", - handler: async (_args, ctx) => { - if (!client) { ctx.ui.notify("Browser client not started.", "warning"); return; } - ctx.ui.notify("Restarting browser client...", "info"); - await client.stop(); - const r = await client.start(); - if (r.success) ctx.ui.notify("Browser client restarted ✓", "info"); - else ctx.ui.notify(`Restart failed: ${r.error.message}`, "error"); - }, - }); - - pi.on("session_start", async (_event, ctx) => { - state = restoreState(ctx, state.namespace); - client = createBrowserClient({ namespace: state.namespace }); - await client.start(); // failure is fine — surfaced via tool errors - if (!toolsRegistered) { - registerAllTools(pi, client); - toolsRegistered = true; - } - if (client) registerSetupCommand(pi, client); - ctx.ui.setStatus("browser", client.status().alive ? "🟢 Browser connected" : "🔴 Browser — run /browser-setup"); - }); - - pi.on("session_shutdown", async () => { - persistState(pi, state); - if (client) { try { await client.stop(); } catch { /* best-effort */ } client = null; } - toolsRegistered = false; - await cleanupTempDirs(); - }); - - pi.on("session_tree", async (_event, ctx) => { - state = restoreState(ctx, client?.namespace); - persistState(pi, state); - }); - - pi.on("before_agent_start", async (event) => { - if (!client || !client.status().alive) { - return { systemPrompt: event.systemPrompt + `\n\n## Browser Control\n\nBrowser tools (browser_*) are available but the browser is not connected. Run /browser-setup.` }; - } - return { systemPrompt: event.systemPrompt + getBrowserSystemPrompt() }; - }); -} -``` - -- [ ] **Step 23.2: Update `src/setup.ts`** to take `BrowserClient` instead of `BrowserDaemon`. Replace `daemon` references with `client` and the `daemon.start()` / `daemon.newTab(...)` / `daemon.getPageInfo()` / `daemon.cdp(...)` calls with their `client` equivalents. Keep the same prompts and behaviors. - -- [ ] **Step 23.3: Trim `src/state.ts`** — remove `tabHistory`, `screenshotDir`, `debugClicks` fields and their helpers. `BrowserState` becomes: - -```ts -export type BrowserState = { - readonly namespace: string; - readonly remoteBrowserId?: string; -}; -export const defaultState = (namespace = "default"): BrowserState => ({ namespace }); -``` - -(`persistState` and `restoreState` keep their signatures; their bodies just drop the removed fields.) - -- [ ] **Step 23.4: Update `src/prompt.ts`** — apply the one-line tweaks for items 1, 3, 5, 8, 9 from spec §7. (Concretely: clarify that `browser_navigate` reports its outcome, that `dispatch_key` returns `matched` count, that `wait_for_load` waits for `loadEventFired`, that scroll's deltaY follows W3C convention, and that screenshot returns `attached: true|false`.) - -- [ ] **Step 23.5:** Typecheck — PASS. -- [ ] **Step 23.6:** Commit `feat: wire BrowserClient through index/setup/state/prompt`. - ---- - -## Task 24: Delete legacy files - -- [ ] **Step 24.1:** Confirm `src/tools.ts` is empty of registrations. The only thing it might still export is `cleanupTempDirs`, which now lives in `src/util/truncate.ts`. Delete `src/tools.ts`. -- [ ] **Step 24.2:** Delete `src/daemon.ts`. -- [ ] **Step 24.3:** Delete `src/protocol.ts`. -- [ ] **Step 24.4:** Delete `src/renderers.ts` (the one renderer it had was dead). -- [ ] **Step 24.5:** Delete `tsconfig.legacy.json` and revert `package.json` `scripts.typecheck` to `tsc --noEmit`. -- [ ] **Step 24.6:** Update `tsconfig.json` `include` back to `["src/**/*.ts"]`. -- [ ] **Step 24.7:** Typecheck — PASS. -- [ ] **Step 24.8:** Commit `refactor: delete legacy daemon, tools, protocol, renderers`. - ---- - -## Task 25: Final cleanup — `_unused`/`_silence` references and lint pass - -- [ ] **Step 25.1:** Remove the `_unused = sleep` line from `domains/js.ts` and the `_silence` export from `util/sharp-shim.ts` if they're no longer needed. -- [ ] **Step 25.2:** Search for any remaining `as ` assertions, `any` types, or `@ts-ignore` comments: - -```bash -grep -rn " as [A-Z]" src/ | grep -v "as const" | grep -v "as unknown" -grep -rn ": any\| any\[" src/ -grep -rn "@ts-ignore\|@ts-expect-error" src/ -``` - -For each match: replace with a type guard, an `unknown` + narrowing, or a Result. The few legitimate `as unknown as Foo` casts at trust boundaries (e.g., parsing CDP responses) stay but get a one-line comment explaining why. - -- [ ] **Step 25.3:** Typecheck — PASS. -- [ ] **Step 25.4:** Commit `chore: remove temporary anchors and tighten remaining type assertions`. - ---- - -## Task 26: Manual smoke run (full verification) - -- [ ] **Step 26.1:** Pre-req: Chrome running with remote-debugging enabled. Install the package into a local pi workspace per `CONTRIBUTING.md`. - -- [ ] **Step 26.2:** Run each scenario from spec §11 and tick: - - - [ ] Daemon connects. - - [ ] `browser_navigate("https://example.com")` succeeds; result `details.outcome.kind` is either `in_place` or `new_tab_created`. - - [ ] `browser_screenshot()` produces a PNG; result `details.path` exists, `details.attached` is set. - - [ ] `browser_click({x, y})` clicks the More-info link. - - [ ] `browser_type("hello")` then `browser_press_key("Enter")` work in a search box. - - [ ] `browser_scroll({})` scrolls down 300px. - - [ ] Trigger an `alert()` via `browser_execute_js("alert('hi')")` — `browser_page_info()` reports the dialog, `browser_handle_dialog({accept: true})` dismisses it. - - [ ] `browser_open_urls({urls: [..3 urls..]})` reports per-URL outcomes. - - [ ] `browser_run_script("/etc/passwd")` is rejected with `kind: "invalid_state"`. - - [ ] `browser_run_script` with a 1-second `timeoutMs` and an infinite-loop script aborts in <2s with `kind: "internal"` containing "aborted". - - [ ] `BH_DEBUG_CLICKS=1` then `browser_click` produces an annotated screenshot at `details.debugScreenshotPath`. - - [ ] `npm run typecheck` is green. - -- [ ] **Step 26.3:** If anything fails, fix it in a follow-up commit before declaring done. Do not edit the plan retroactively. - ---- - -## Task 27: Version bump and CHANGELOG - -- [ ] **Step 27.1:** In `package.json`, set `"version": "0.3.0"`. - -- [ ] **Step 27.2:** Prepend to `CHANGELOG.md`: - -```markdown -## 0.3.0 — 2026-05-02 - -### Internal rewrite - -- Per-domain module split: every tool now lives in its own `src/domains/<name>.ts` file. -- New transport/session/client split: `BrowserDaemon` class replaced with `createBrowserClient()` factory composing a `CdpTransport` and `CdpSession`. -- All tool handlers return `Result<T, E>`; one `defineTool` helper converts to pi's `ToolResult` and supplies a uniform `details` shape: `{ ok: true, ... }` on success, `{ ok: false, kind, message, ... }` on error. -- Strict TypeScript flags enabled (`noUncheckedIndexedAccess`, `exactOptionalPropertyTypes`, `noPropertyAccessFromIndexSignature`, etc.). Zero `any`, zero unchecked `as` casts. - -### Bug fixes (predictability) - -- `browser_navigate` no longer silently creates a new tab when navigation fails; outcome is reported explicitly. -- `browser_wait_for_load` now uses CDP load events instead of relying on a single `readyState` snapshot. -- Page info cache invalidation is automatic via CDP `Page.frameNavigated` / `Page.loadEventFired` subscription. -- Dialog read no longer mutates state; `browser_page_info` is now read-only. -- `browser_dispatch_key` returns the count of matched elements; zero matches is an error. -- `browser_http_get` timeout now covers the response body read. -- `sharp` failures are distinguished from "sharp not installed". -- Screenshot paths use UUIDs — no more concurrent-write collisions. -- Reconnect is lazy (via `ensureAlive`) — no background reconnect, no stacked Chrome consent popups. - -### Security fixes - -- All JS evaluation source is built via `safeJs\`...\`` (always JSON.stringify-safe). The previous `replace(/'/g, "\\'")` escaping is gone. -- `browser_run_script` now requires: - - script path inside `tmpdir()`, `cwd()`, or `BH_SCRIPT_DIR` - - a mandatory timeout (default 60s, max 600s) - - the AbortSignal is honored even if the script ignores it - - source size ≤ 1 MB - - return shape validated structurally -- `browser_download` validates the directory exists and is writable. -- `browser_upload_file` verifies the file is readable before any CDP call. - -### Parameter renames (saved scripts must be updated) - -- `browser_click`: `clicks` → `count` -- `browser_dispatch_key`: `event` → `eventType` - -### Removed - -- The unused `tabHistory`, `screenshotDir`, and `debugClicks` fields on persisted state. -- The dead `tool_result` hook for tab-history tracking. -- `src/protocol.ts`, `src/renderers.ts`, `src/daemon.ts`, `src/tools.ts` (replaced by per-domain files and `client.ts`). - -### Known follow-ups - -- `browser_get_network_log` returns a deprecation note pending a `session.recentEvents()` API. -- No tests added in this rewrite; that's a separate workstream. -``` - -- [ ] **Step 27.3:** Commit and tag: - -```bash -git add package.json CHANGELOG.md -git commit -m "chore(release): v0.3.0 — internal rewrite" -git tag v0.3.0 -``` - ---- - -## Self-review - -**1. Spec coverage check:** - -- §3 module layout → Tasks 1, 2, 4–8, 9–22, 24 ✓ -- §4 `defineTool` helper → Task 8 ✓ -- §4.1 param renames → click (Task 9), dispatch_key (Task 10) ✓ -- §5 transport/session/client split → Tasks 4, 5, 6, 7 ✓ -- §5.1 structural bug fixes (event buffer, page-info cache, dialog race, navigate fallthrough, http body timeout, screenshot paths) → Tasks 1, 6, 7, 12, 15, 21 ✓ -- §5.2 reconnect change → Task 7 (in `ensureAlive`) ✓ -- §6.1 safeJs → Task 2 (helper), used in Tasks 7, 10, 12, 18 ✓ -- §6.2 run_script hardening (path allowlist, timeout, AbortSignal, return validator, size cap) → Task 22 ✓ -- §6.3 download dir validation, upload readability → Task 18 ✓ -- §7 predictability register (12 items) → Tasks 9, 10, 11, 12, 14, 15, 21, 23 ✓ -- §8 strict tsconfig → Tasks 3, 24 ✓ -- §9 migration notes → Task 27 ✓ -- §11 verification → Task 26 ✓ - -**2. Placeholder scan:** No "TBD"/"TODO"/"implement later". Each step has concrete code or a concrete command. - -**3. Type consistency:** `BrowserClient`, `CdpSession`, `CdpTransport`, `Result`, `ToolErr`, `ToolOk`, `defineTool`, `registerTool`, `safeJs`, `screenshotPath`, `applyTruncation` — names used in later tasks match their definitions in earlier tasks. The `client.session().call(...)` pattern is consistent across all domain files. - -**4. Notes I noticed during review:** -- Task 21's `browser_get_network_log` returns a deprecation note rather than full functionality. This is called out in Task 27's CHANGELOG ("Known follow-ups"). Acceptable for an internal-only rewrite per spec §10 "no new functionality." -- Task 9 has a temporary placeholder for the debug crosshair (uses a plain screenshot); Task 14 explicitly restores the real overlay. The two tasks are linked — flagged in Step 14.3. -- Task 17's `browser_current_tab` calls `Target.getTargetInfo`, matching the legacy behavior. ✓ -- Task 18's `browser_upload_file` keeps the CDP-then-JS-fallback flow but both paths now use `safeJs`. ✓ |