diff options
| -rw-r--r-- | CHANGELOG.md | 15 | ||||
| -rw-r--r-- | README.md | 2 | ||||
| -rw-r--r-- | SECURITY.md | 1 | ||||
| -rw-r--r-- | docs/FAQ.md | 4 | ||||
| -rw-r--r-- | extensions/chrome-profile-bridge/browser-extension/manifest.json | 2 | ||||
| -rw-r--r-- | extensions/chrome-profile-bridge/browser-extension/service_worker.js | 420 | ||||
| -rw-r--r-- | extensions/chrome-profile-bridge/index.ts | 195 | ||||
| -rw-r--r-- | package-lock.json | 4 | ||||
| -rw-r--r-- | package.json | 4 | ||||
| -rw-r--r-- | test-suite/unit/background-policy.test.mjs | 4 | ||||
| -rw-r--r-- | test-suite/unit/bridge-resilience.test.mjs | 138 | ||||
| -rw-r--r-- | test-suite/unit/cdp-passthrough.test.mjs | 183 | ||||
| -rw-r--r-- | test-suite/unit/type-evidence.test.mjs | 208 |
13 files changed, 1116 insertions, 64 deletions
diff --git a/CHANGELOG.md b/CHANGELOG.md index 17def6c..c2ff5ea 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,21 @@ All notable user-facing changes to `pi-chrome`. +## 0.15.53 — 2026-09-26 + +Changes adapted from community forks (ardhiqii, kkunkunya, nihar-oracle, steimerbyte). + +- **Connector recovers from stalled connections.** The extension's `/next` long poll now gives up after 45s (the bridge holds it for up to 25s). Before this, a half-open socket (Pi process died, machine slept, network changed) could leave the extension waiting forever until someone reloaded it by hand. Result posts time out, retry on network/5xx failures, and are never posted twice for the same command. +- **Page actions work on a fresh automation tab.** Chrome refuses `chrome.scripting` on a top-level `about:blank`, which is where the dedicated automation tab starts. As a result, `chrome_snapshot`, `chrome_inspect`, console/network listing, uid click targeting, and the `/chrome doctor` page probe failed on that tab with `Cannot access contents of url "about:blank"`. When Chrome refuses scripting for a permission reason, these calls now fall back to CDP `Runtime.evaluate`. Ordinary pages still use `chrome.scripting` first. +- **Chrome tools in subagent sessions.** A second load of the same pi-chrome install in one process (for example, pi-subagents) was skipped as a duplicate, so subagents had no `chrome_*` tools. It now loads as a client of the shared bridge. Subagents share the parent's authorization and get their own automation tab and tab group. Two different install roots are still treated as duplicates. `chrome_launch` in a client session now reports the shared connection state instead of always saying "waiting for extension". +- **New `chrome_cdp` and `chrome_cdp_targets` tools.** `chrome_cdp` runs one raw Chrome DevTools Protocol method on a tab and validates the method/params first. `timeoutMs` can extend the deadline up to 120s, and screenshot/binary or oversized results are summarized instead of returned in full. Background mode blocks `Page.bringToFront` and `Target.activateTarget`. `chrome_cdp_targets` lists CDP targets on a tab, such as password-manager overlays or DevTools, to help diagnose `Detached while handling command`. It never attaches the debugger or creates a tab. +- **`chrome_type` shows what it typed.** Results include the field value before and after, and `insertedAt` (`empty`, `caret-end`, `caret-middle`, `replaced-selection`, `replaced-all`). The text warns when input was spliced into existing content, when Enter may have submitted the spliced value, and when the field did not change at all (keystrokes did not reach it). Password/OTP/card-like fields report only lengths. New `replace: true` selects all with Chrome's platform-neutral `selectAll` editing command and deletes before typing. Tool descriptions now state that `chrome_type` inserts at the caret and `chrome_fill` replaces. +- **`includeSnapshot` waits for navigations.** If a click/type/fill/key action starts a navigation, the included snapshot waits up to 5s for the new page and reports `navigation {from, to, settled, waitedMs}`. Before, it could describe the page being replaced. Actions without navigation do not wait. A page that was already loading is reported but not waited on. +- **CDP timeouts report as timeouts.** A timed-out CDP command now fails with `CDP <method> timed out after Nms`. Before, the cleanup detach surfaced as `Detached while handling command`, and the command was re-sent once, so a slow command could run twice. +- **`chrome_tab new` reports the loaded tab.** It waits up to 5s for the URL to load and returns `loadStatus` instead of Chrome's initial empty `loading` tab. +- **Tool activation.** `chrome_find`, `chrome_inspect`, and the new CDP tools are now activated by `/chrome authorize` and removed from the active tool set by `/chrome revoke` or grant expiry, like the other `chrome_*` tools. +- **Validation.** New unit suites: `bridge-resilience`, `cdp-passthrough`, `type-evidence`. + ## 0.15.51 — 2026-09-10 - **Fewer Chrome commands.** Removed `/chrome status`; use bare `/chrome` for the quick connection, authorization, and background dashboard plus controls. The dashboard remains lightweight and does not run page probes. @@ -16,7 +16,7 @@ Try prompts like these after setup: | **Understand an existing page** | “Find my open dashboard tab and summarize what's on the page. Don't change anything.” | | **Create evidence for a PR** | “On my local app, capture the empty, loading, and populated states of this feature for my PR.” | -Pi gets tools to inspect pages, click, type, fill forms, scroll, upload files, capture screenshots, and inspect captured console logs and `fetch`/`XMLHttpRequest` responses. You describe the task; Pi handles the agent loop. +Pi gets tools to inspect pages, click, type, fill forms, scroll, upload files, capture screenshots, and inspect captured console logs and `fetch`/`XMLHttpRequest` responses. A raw Chrome DevTools Protocol tool (`chrome_cdp`) covers anything else, such as device emulation, cookies, PDFs, or the accessibility tree. You describe the task; Pi handles the agent loop. **Best fit:** interactive workflows in the Chrome profile you already use. For deterministic CI tests, consider a test framework such as Playwright; for fleets of isolated browsers, consider a hosted browser service. See [more workflows](./docs/EXAMPLES.md) and [browser-tool comparisons](./docs/COMPARISON.md). diff --git a/SECURITY.md b/SECURITY.md index 3d07518..0121a1c 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -27,6 +27,7 @@ The Chrome extension under `extensions/chrome-profile-bridge/browser-extension/` - Loopback bridge only. No remote port. No telemetry. - Chrome real input layer for interactive controls. - Chrome control locked by default; `/chrome authorize` unlocks current Pi session after terminal confirmation, `/chrome revoke` locks it again. +- `chrome_cdp` sends raw Chrome DevTools Protocol commands to a tab. It is locked behind `/chrome authorize` like every other tool, but it is not filtered against a safe list. Background mode blocks only its explicit focus methods (`Page.bringToFront`, `Target.activateTarget`). - Hard background mode is on by default: tools cannot override it to explicitly focus windows or activate tabs. `/chrome background off` allows foreground/watch mode. This is not a security sandbox: trusted input, page scripts, native prompts, and Chrome/OS behavior can still affect focus. ## Custom ports diff --git a/docs/FAQ.md b/docs/FAQ.md index a450a93..d717694 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -32,6 +32,8 @@ Chrome control is also locked per Pi session until you run `/chrome authorize`; Yes. The first session opens the local bridge; later sessions detect it and pipe their commands through the same bridge. Each Pi session must be authorized with `/chrome authorize` before its chrome_* tools work. Each session also owns its **own** dedicated automation window (ownership is keyed by session id inside the one extension), so concurrent sessions never navigate into or close each other's tabs. +Subagent sessions that run inside an authorized Pi process (for example with pi-subagents) also get the chrome_* tools. They share the parent's `/chrome authorize` grant and bridge, but each subagent gets its own automation tab and tab group, which are cleaned up when the subagent session ends. + ## Does pi-chrome navigate my current tab? No. The first chrome_* action that has no explicit target opens a **dedicated automation window** that pi-chrome owns (falling back to a dedicated tab only if a separate window can't be created), and reuses it for the rest of the session. Your existing tabs and windows are never reused or overwritten. Pass `targetId`/`urlIncludes`/`titleIncludes` to deliberately act on a tab you already have open. @@ -50,7 +52,7 @@ pi-chrome ships as an unpacked extension so the source and broad browser permiss ## What's the install footprint? -- Pi side: one extension that registers 19 tools and a few slash commands. +- Pi side: one extension that registers 23 tools and a few slash commands. - Chrome side: one unpacked extension, ~2000 LOC of plain JavaScript, no dependencies. ## Can I script it without Pi? diff --git a/extensions/chrome-profile-bridge/browser-extension/manifest.json b/extensions/chrome-profile-bridge/browser-extension/manifest.json index cad26fa..952c9cc 100644 --- a/extensions/chrome-profile-bridge/browser-extension/manifest.json +++ b/extensions/chrome-profile-bridge/browser-extension/manifest.json @@ -1,7 +1,7 @@ { "manifest_version": 3, "name": "Pi Chrome Connector", - "version": "0.15.51", + "version": "0.15.53", "description": "Lets Pi control tabs in Chrome via a local connector at 127.0.0.1.", "permissions": [ "tabs", diff --git a/extensions/chrome-profile-bridge/browser-extension/service_worker.js b/extensions/chrome-profile-bridge/browser-extension/service_worker.js index 8ba0e2d..4bfce34 100644 --- a/extensions/chrome-profile-bridge/browser-extension/service_worker.js +++ b/extensions/chrome-profile-bridge/browser-extension/service_worker.js @@ -1,6 +1,13 @@ const BRIDGE_URL = "http://127.0.0.1:17318"; const CLIENT_NAME = `Pi Chrome Connector ${chrome.runtime.id}`; const POLL_ERROR_BACKOFF_MS = 2000; +// /next is a server-side long poll held for up to 25s (see waitForCommand in index.ts). A fetch that +// outlives this deadline means the socket is half-open (the owning Pi process died, the machine +// slept, the network changed) and will never settle; abort it so pollLoop backs off and reconnects +// instead of parking on a zombie connection until the extension is reloaded by hand. +const POLL_REQUEST_TIMEOUT_MS = 45_000; +const RESULT_REQUEST_TIMEOUT_MS = 15_000; +const RESULT_POST_ATTEMPTS = 3; const DEFAULT_GROUP_COLOR = "blue"; const PI_GROUP_RE = /^Pi(\b|\s*-)/i; const VALID_GROUP_COLORS = new Set(["grey", "blue", "red", "yellow", "green", "pink", "purple", "cyan", "orange"]); @@ -8,6 +15,8 @@ const COMMAND_TIMEOUT_MS = 25_000; const CDP_COMMAND_TIMEOUT_MS = 5_000; const SCRIPTING_TIMEOUT_MS = 8_000; const ATTACH_TIMEOUT_MS = 3_000; +// Raw CDP methods that explicitly focus or activate a tab. Blocked by hard background mode. +const CDP_FOCUS_METHODS = new Set(["Page.bringToFront", "Target.activateTarget"]); let polling = false; // =================== pi-chrome automation target ownership =================== @@ -250,9 +259,12 @@ function withTimeout(promise, ms, label, onTimeout) { return Promise.race([ Promise.resolve(promise).finally(() => clearTimeout(timer)), new Promise((_, reject) => { - timer = setTimeout(async () => { - try { await onTimeout?.(); } catch {} + timer = setTimeout(() => { + // Settle with the timeout first. Cleanup (e.g. debugger detach) makes the pending command + // fail with "Detached while handling command"; if that won the race, cdp() would treat it as + // a stale session and re-send a command that may already have run. reject(new Error(`${label} timed out after ${ms}ms`)); + Promise.resolve().then(() => onTimeout?.()).catch(() => {}); }, ms); }), ]); @@ -435,14 +447,28 @@ setInterval(() => { } }, 5000); -function cdpRaw(tabId, method, params) { +// Deadline policy for cdp.call. Callers may widen the per-command CDP deadline via params.timeoutMs, +// clamped below the MV3 service-worker lifetime so a stuck command cannot pin the debugger forever. +const CDP_CALL_MAX_TIMEOUT_MS = 120_000; +const CDP_CALL_GRACE_MS = 5_000; +function cdpCallTimeoutMs(params) { + const requested = Number(params?.timeoutMs); + if (!Number.isFinite(requested) || requested <= 0) return CDP_COMMAND_TIMEOUT_MS; + return Math.min(Math.floor(requested), CDP_CALL_MAX_TIMEOUT_MS); +} + +// `opts.timeoutMs` overrides the default per-command deadline. On timeout the session is detached +// and forgotten so the next call re-attaches cleanly. +function cdpRaw(tabId, method, params, opts) { const debuggee = attachedTabs.get(tabId)?.debuggee || { tabId }; + const requested = Number(opts?.timeoutMs); + const timeoutMs = Number.isFinite(requested) && requested > 0 ? requested : CDP_COMMAND_TIMEOUT_MS; return withTimeout(new Promise((resolve, reject) => { chrome.debugger.sendCommand(debuggee, method, params || {}, (result) => { if (chrome.runtime.lastError) reject(new Error(`${method}: ${chrome.runtime.lastError.message}`)); else resolve(result); }); - }), CDP_COMMAND_TIMEOUT_MS, `CDP ${method}`, async () => { + }), timeoutMs, `CDP ${method}`, async () => { attachedTabs.delete(tabId); try { await chrome.debugger.detach(debuggee); } catch {} }); @@ -452,6 +478,51 @@ function executeScriptTimed(options, label) { return withTimeout(chrome.scripting.executeScript(options), SCRIPTING_TIMEOUT_MS, label || "chrome.scripting.executeScript"); } +// Chrome refuses chrome.scripting on pages outside the extension's host permissions. Notably a +// top-level about:blank (pi-chrome's fresh automation tab) has an opaque origin that <all_urls> +// does not cover, so snapshot/inspect/probe/console/click-resolution failed there with +// "Cannot access contents of url \"about:blank\"". The debugger API still attaches to about:blank. +function isScriptingAccessDenied(error) { + const message = String(error?.message || error); + return /Cannot access contents of url|Cannot access .+ at origin|Extension manifest must request permission|matchAboutBlank must be true/i.test(message); +} + +// chrome.scripting.executeScript with a CDP Runtime.evaluate fallback when Chrome refuses scripting +// for a permission reason. Scripting stays primary: it is what ordinary pages use and the only path +// that works while DevTools holds the tab's debugger. The fallback runs in the same MAIN world and +// covers both forms: `func` is stringified and invoked with its args; `files` are read from the +// extension package and evaluated. Any other scripting failure surfaces unchanged. +const packagedFileSources = new Map(); +async function executeScriptWithFallback(options, label) { + try { + return await executeScriptTimed(options, label); + } catch (error) { + if (!isScriptingAccessDenied(error) || !chrome.debugger) throw error; + const tabId = options?.target?.tabId; + recordAttachEvent({ kind: "scripting-denied-cdp-fallback", tabId, files: options?.files || null, message: String(error?.message || error).slice(0, 300) }); + if (Array.isArray(options?.files) && options.files.length) { + for (const file of options.files) { + if (!packagedFileSources.has(file)) { + const response = await fetch(chrome.runtime.getURL(file)); + if (!response.ok) throw new Error(`Could not read ${file} for CDP injection (HTTP ${response.status})`); + packagedFileSources.set(file, await response.text()); + } + const injected = await cdpEval(tabId, packagedFileSources.get(file)); + if (injected.exceptionDetails) { + throw new Error(`${label || file}: ${cdpExceptionText(injected.exceptionDetails) || "CDP injection failed"}`); + } + } + return [{ result: undefined }]; + } + const expression = `(${options.func.toString()})(...${JSON.stringify(Array.isArray(options.args) ? options.args : [])})`; + const result = await cdpEval(tabId, expression); + if (result.exceptionDetails) { + throw new Error(`${label || "page script"}: ${cdpExceptionText(result.exceptionDetails) || "evaluation failed"}`); + } + return [{ result: result.result?.value }]; + } +} + // Wraps cdpRaw with one auto-recover on detached/closed sessions: // chrome.debugger.attach can stay cached in attachedTabs even after Chrome killed // the session (tab nav, devtools opened/closed, etc). Recover by detaching the @@ -494,9 +565,9 @@ async function dismissOverlayViaEscape(tabId) { } catch {} } -async function cdp(tabId, method, params) { +async function cdp(tabId, method, params, opts) { try { - return await cdpRaw(tabId, method, params); + return await cdpRaw(tabId, method, params, opts); } catch (error) { const msg = String(error?.message || error); const isStale = /Debugger is not attached|Detached while|Target closed|No tab with id/i.test(msg); @@ -508,7 +579,7 @@ async function cdp(tabId, method, params) { recordAttachEvent({ kind: "foreign-ext-detected", tabId, method, foreignExtId: extractForeignExtId(before), targetCount: before.length }); await dismissOverlayViaEscape(tabId); try { - return await cdpRaw(tabId, method, params); + return await cdpRaw(tabId, method, params, opts); } catch (retryErr) { const retryMsg = String(retryErr?.message || retryErr); if (/Cannot access a chrome-extension:\/\/ URL of different extension/i.test(retryMsg)) { @@ -525,7 +596,7 @@ async function cdp(tabId, method, params) { if (!isStale) throw error; attachedTabs.delete(tabId); await attachDebugger(tabId).catch(() => undefined); - return cdpRaw(tabId, method, params); + return cdpRaw(tabId, method, params, opts); } } @@ -563,7 +634,7 @@ function cdpIsSyntaxError(details) { // Resolve target -> {x, y, rect} in viewport coords by running tiny script in tab. async function resolveTargetInTab(tabId, params) { - const results = await executeScriptTimed({ + const results = await executeScriptWithFallback({ target: { tabId, frameIds: [0] }, world: "MAIN", func: (selector, uid, x, y) => { @@ -721,7 +792,7 @@ async function cdpTypeChar(tabId, ch) { } async function domClickFallback(tabId, params, cause) { - const results = await executeScriptTimed({ + const results = await executeScriptWithFallback({ target: { tabId, frameIds: [0] }, world: "MAIN", func: (selector, uid, x, y) => { @@ -764,7 +835,7 @@ async function chromeInputClick(params) { // focus can leave :focus-visible=true in Chromium, which trips heuristics that expect // Reset focus styling after pointer click when possible. if (params.selector || params.uid) { - await executeScriptTimed({ + await executeScriptWithFallback({ target: { tabId: tab.id, frameIds: [0] }, world: "MAIN", func: (sel, uid) => { @@ -840,7 +911,7 @@ async function chromeInputKey(params) { // the requested editor's entire contents: triple-click only selects a paragraph. // Selection uses the DOM; deletion and insertion still use Chrome's input layer. async function contentEditableInTab(tabId, selectAllParams = null) { - const results = await executeScriptTimed({ + const results = await executeScriptWithFallback({ target: { tabId, frameIds: [0] }, world: "MAIN", func: (selector, uid, selectAll) => { @@ -894,14 +965,143 @@ async function chromeInputType(params) { await sleep(rng(50, 120)); } const text = String(params.text || ""); + // chrome_type inserts at the caret. Read the field before and after so a splice into existing + // text is reported instead of hidden behind a bare character count. The "before" read precedes + // replace's select-all/delete. Reads are best-effort and never block typing. + const before = await readInputStateInTab(tab.id, params).catch(() => null); + if (params.replace === true) await selectAllAndDelete(tab.id); const typing = await typeTextInTab(tab.id, text, params.perCharacter); + const after = await readInputStateInTab(tab.id, params).catch(() => null); if (params.pressEnter) await chromeInputKey({ ...params, targetId: tab.id, key: "Enter" }); - return { input: "chrome", length: text.length, typing }; + return { + input: "chrome", + length: text.length, + typing, + ...inputValueEvidence(before, after, { replaced: params.replace === true }), + }; +} + +// Select the focused field's whole contents with Chrome's editing command, then delete it with real +// key events. The selectAll editing command is platform-neutral (Ctrl+A moves to line start on macOS). +async function selectAllAndDelete(tabId) { + const mac = /Mac/i.test(String(globalThis.navigator?.platform || globalThis.navigator?.userAgent || "")); + const modifiers = mac ? 4 : 2; + const key = mac ? { key: "Meta", code: "MetaLeft", vk: 91 } : { key: "Control", code: "ControlLeft", vk: 17 }; + await cdp(tabId, "Input.dispatchKeyEvent", { type: "rawKeyDown", key: key.key, code: key.code, windowsVirtualKeyCode: key.vk, modifiers }); + await cdp(tabId, "Input.dispatchKeyEvent", { type: "rawKeyDown", key: "a", code: "KeyA", windowsVirtualKeyCode: 65, modifiers, commands: ["selectAll"] }); + await cdp(tabId, "Input.dispatchKeyEvent", { type: "keyUp", key: "a", code: "KeyA", windowsVirtualKeyCode: 65, modifiers }); + await cdp(tabId, "Input.dispatchKeyEvent", { type: "keyUp", key: key.key, code: key.code, windowsVirtualKeyCode: key.vk, modifiers: 0 }); + await cdp(tabId, "Input.dispatchKeyEvent", { type: "keyDown", key: "Delete", code: "Delete", windowsVirtualKeyCode: 46 }); + await cdp(tabId, "Input.dispatchKeyEvent", { type: "keyUp", key: "Delete", code: "Delete", windowsVirtualKeyCode: 46 }); + await sleep(rng(20, 60)); +} + +// Read the typing target's value and caret. Sensitive fields (password/OTP/card/etc., matching the +// snapshot redaction convention) report only valueRedacted + length, never contents. Displayed values +// are truncated to 120 chars like snapshots; lengths and caret offsets are exact. +async function readInputStateInTab(tabId, params) { + const results = await executeScriptWithFallback({ + target: { tabId, frameIds: [0] }, + world: "MAIN", + func: (selector, uid) => { + const state = window.__PI_CHROME_STATE__; + const carrierOf = (node) => { + if (!node) return null; + if ("value" in node && typeof node.value === "string" && /^(INPUT|TEXTAREA)$/.test(node.tagName)) return "value"; + if (node.isContentEditable === true) return "contenteditable"; + return null; + }; + let el = uid ? (state && state.elements ? state.elements[uid] : null) : null; + if (uid && (!el || !el.isConnected)) el = null; + if (!el && selector) el = document.querySelector(selector); + const active = document.activeElement; + // The uid/selector can name a wrapper (role=textbox, label) rather than the element that + // receives text. Prefer the focused carrier in that case so we do not report a false "" -> "". + if (!carrierOf(el) && carrierOf(active)) el = active; + const carrier = carrierOf(el); + if (!carrier) return { found: false }; + const raw = carrier === "value" ? el.value : String(el.textContent || ""); + const attr = (name) => (el.getAttribute ? el.getAttribute(name) : null); + const type = String(el.type || attr("type") || "").toLowerCase(); + const haystack = [type, el.name, attr("name"), el.id, attr("autocomplete"), attr("aria-label"), attr("placeholder"), attr("data-testid")].filter(Boolean).join(" ").toLowerCase(); + const sensitive = type === "password" || /password|passwd|\bpwd\b|secret|token|bearer|api[-_ ]?key|access[-_ ]?key|auth[-_ ]?code|one[-_ ]?time|otp|2fa|mfa|verification[-_ ]?code|recovery[-_ ]?code|credit[-_ ]?card|card[-_ ]?number|cc-number|cc-csc|cvc|cvv|security[-_ ]?code|ssn|social[-_ ]?security/.test(haystack); + let selectionStart = null; + let selectionEnd = null; + try { + if (typeof el.selectionStart === "number") selectionStart = el.selectionStart; + if (typeof el.selectionEnd === "number") selectionEnd = el.selectionEnd; + } catch {} + let caretOffset = null; + if (carrier === "contenteditable") { + try { + const selection = window.getSelection && window.getSelection(); + if (selection && selection.rangeCount > 0 && selection.anchorNode && el.contains(selection.anchorNode)) { + const range = document.createRange(); + range.selectNodeContents(el); + range.setEnd(selection.anchorNode, selection.anchorOffset); + caretOffset = range.toString().length; + if (!selection.isCollapsed) selectionEnd = caretOffset + String(selection.toString()).length; + } + } catch {} + } + return { + found: true, + carrier, + value: sensitive ? undefined : raw.slice(0, 120), + valueLength: raw.length, + valueRedacted: sensitive ? true : undefined, + selectionStart, + selectionEnd, + caretOffset, + }; + }, + args: [params.selector ?? null, params.uid ?? null], + }, `read input value in tab ${tabId}`); + const v = results?.[0]?.result; + return v && typeof v === "object" ? v : { found: false }; +} + +// Where typed text landed relative to the field's existing contents. +function inputInsertPosition(before) { + const length = before.valueLength ?? 0; + if (length === 0) return "empty"; + if (typeof before.selectionStart === "number" && typeof before.selectionEnd === "number") { + if (before.selectionEnd > before.selectionStart) { + return before.selectionStart === 0 && before.selectionEnd >= length ? "replaced-all" : "replaced-selection"; + } + return before.selectionStart < length ? "caret-middle" : "caret-end"; + } + if (typeof before.caretOffset === "number") { + if (typeof before.selectionEnd === "number" && before.selectionEnd > before.caretOffset) { + return before.caretOffset === 0 && before.selectionEnd >= length ? "replaced-all" : "replaced-selection"; + } + return before.caretOffset < length ? "caret-middle" : "caret-end"; + } + return undefined; +} + +function inputValueEvidence(before, after, { replaced = false } = {}) { + const hasBefore = Boolean(before?.found); + const hasAfter = Boolean(after?.found); + if (!hasBefore && !hasAfter) return replaced ? { replaced: true } : {}; + const redacted = Boolean(before?.valueRedacted || after?.valueRedacted); + const evidence = { + existingTextLengthBefore: hasBefore ? before.valueLength : undefined, + valueLengthAfter: hasAfter ? after.valueLength : undefined, + insertedAt: replaced ? "replaced-all" : hasBefore ? inputInsertPosition(before) : undefined, + }; + if (redacted) evidence.valueRedacted = true; + else { + if (hasBefore) evidence.valueBefore = before.value; + if (hasAfter) evidence.valueAfter = after.value; + } + if (replaced) evidence.replaced = true; + return evidence; } async function domFillFallback(tabId, params, cause) { if (!(params.selector || params.uid)) throw cause; - const results = await executeScriptTimed({ + const results = await executeScriptWithFallback({ target: { tabId, frameIds: [0] }, world: "MAIN", func: async (selector, uid, text, submit) => { @@ -1138,9 +1338,9 @@ async function pollLoop() { polling = true; try { while (true) { - const response = await fetch(`${BRIDGE_URL}/next?name=${encodeURIComponent(CLIENT_NAME)}`, { + const response = await fetchWithTimeout(`${BRIDGE_URL}/next?name=${encodeURIComponent(CLIENT_NAME)}`, { cache: "no-store", - }); + }, POLL_REQUEST_TIMEOUT_MS, "bridge /next"); if (!response.ok) throw new Error(`bridge /next HTTP ${response.status}`); const expected = response.headers.get("x-pi-chrome-version"); const ours = chrome.runtime.getManifest().version; @@ -1159,26 +1359,70 @@ async function pollLoop() { } } +// fetch() with an abort deadline that covers connecting and receiving response headers. The timer is +// cleared as soon as headers arrive so a late abort can never fail an already-delivered response. +async function fetchWithTimeout(url, options, timeoutMs, label) { + const controller = new AbortController(); + const timer = setTimeout(() => controller.abort(), timeoutMs); + try { + return await fetch(url, { ...(options || {}), signal: controller.signal }); + } catch (error) { + if (error?.name === "AbortError") throw new Error(`${label || "fetch"} timed out after ${timeoutMs}ms`); + throw error; + } finally { + clearTimeout(timer); + } +} + +function commandTimeoutMs(action, params) { + if (action !== "cdp.call") return COMMAND_TIMEOUT_MS; + // Let the inner CDP deadline fire first so callers see "CDP <method> timed out" instead of the + // generic command wrapper timeout. + return Math.max(COMMAND_TIMEOUT_MS, cdpCallTimeoutMs(params) + CDP_CALL_GRACE_MS); +} + async function handleCommand(command) { + let payload; try { const result = await withTimeout( dispatch(command.action, command.params ?? {}), - COMMAND_TIMEOUT_MS, + commandTimeoutMs(command.action, command.params ?? {}), command.action || "Chrome command", () => detachAll(), ); - await postResult({ id: command.id, ok: true, result }); + payload = { id: command.id, ok: true, result }; } catch (error) { - await postResult({ id: command.id, ok: false, error: error?.message ?? String(error) }); + payload = { id: command.id, ok: false, error: error?.message ?? String(error) }; + } + // Post exactly one result. A failed success-post used to fall into the catch above and post a + // second (error) result for the same command id. + try { + await postResult(payload); + } catch (error) { + console.warn(`[pi-chrome] failed to post result for ${command.action}: ${error?.message ?? String(error)}`); } } async function postResult(result) { - await fetch(`${BRIDGE_URL}/result`, { - method: "POST", - headers: { "content-type": "application/json" }, - body: JSON.stringify(result), - }); + const body = JSON.stringify(result); + let lastError; + for (let attempt = 1; attempt <= RESULT_POST_ATTEMPTS; attempt++) { + try { + const response = await fetchWithTimeout(`${BRIDGE_URL}/result`, { + method: "POST", + headers: { "content-type": "application/json" }, + body, + }, RESULT_REQUEST_TIMEOUT_MS, "bridge /result"); + // 4xx is final: the bridge no longer knows this command (it already timed out) or rejected + // the request. Retrying cannot help. + if (response.ok || (response.status >= 400 && response.status < 500)) return; + lastError = new Error(`bridge /result HTTP ${response.status}`); + } catch (error) { + lastError = error; + } + if (attempt < RESULT_POST_ATTEMPTS) await sleep(250 * attempt); + } + throw lastError; } function isVersionOlder(a, b) { @@ -1285,7 +1529,19 @@ async function dispatch(action, params) { await trackSessionTab(sessionKeyOf(params), tab.id, true); try { await bringToFront(tab, params); - return await groupTab(tab, groupTitle, params.groupColor); + const grouped = await groupTab(tab, groupTitle, params.groupColor); + // chrome.tabs.create returns the t=0 tab ({url:"", status:"loading"}), which reads as "the + // URL never loaded". Wait (bounded, never throwing) and report the settled tab instead. + if (!params.url || params.url === "about:blank") return grouped; + const started = Date.now(); + const settled = await waitForTabSettled(tab.id, SNAPSHOT_NAVIGATION_WAIT_MS); + const latest = await chrome.tabs.get(tab.id).catch(() => null); + return { + ...grouped, + tab: latest ? await formatTab(latest) : grouped.tab, + loadStatus: settled ? "complete" : "timedOut", + waitedMs: Date.now() - started, + }; } catch (error) { if (typeof tab.id === "number") await chrome.tabs.remove(tab.id).catch(() => {}); throw error; @@ -1395,6 +1651,43 @@ async function dispatch(action, params) { } case "page.screenshot": return takeScreenshot(params); + case "cdp.targets": { + // Diagnostics: CDP targets anchored to the resolved tab, including foreign overlay targets + // (password managers, autofill, DevTools front-ends) that explain "Detached while handling + // command" failures. Never creates an automation window. Only the resolved tab's targets are + // echoed; targets on other tabs are counted, not listed. + const hasSelector = params.targetId !== undefined || Boolean(params.urlIncludes) || Boolean(params.titleIncludes); + const tab = await getTabByParams(params, { createOwnedTarget: false }).catch((error) => { + if (hasSelector) throw error; + return null; + }); + const allTargets = await new Promise((resolve) => chrome.debugger.getTargets((t) => resolve(t || []))).catch(() => []); + const targets = tab ? allTargets.filter((t) => t.tabId === tab.id) : []; + return { + tab: tab ? { id: tab.id, windowId: tab.windowId, url: tab.url, title: tab.title } : null, + targets: targets.map((t) => ({ id: t.id, tabId: t.tabId, type: t.type, url: t.url, title: t.title, attached: t.attached, extensionId: t.extensionId })), + otherTabTargetCount: allTargets.filter((t) => typeof t.tabId === "number" && t.tabId !== tab?.id).length, + }; + } + case "cdp.call": { + // Validate before touching the debugger: a clear error beats Chrome's opaque + // "Invalid parameters" for a missing method or non-object params. + if (typeof params.method !== "string" || !params.method.trim()) { + throw new Error('cdp.call requires a non-empty string "method" (for example "Runtime.evaluate" or "DOM.getDocument").'); + } + if (params.params !== undefined && params.params !== null && (typeof params.params !== "object" || Array.isArray(params.params))) { + throw new Error('cdp.call "params" must be a plain object of CDP parameters when provided.'); + } + const method = params.method.trim(); + // Background mode is a hard policy. Raw CDP must not become a way around it. + if (!foregroundRequested(params) && CDP_FOCUS_METHODS.has(method)) { + throw new Error(`${method} is blocked by background mode because it focuses or activates a tab. Ask the user to run /chrome background off to allow foreground work.`); + } + const tab = await getTabByParams(params); + await bringToFront(tab, params); + await attachDebugger(tab.id); + return await cdp(tab.id, method, params.params ?? {}, { timeoutMs: cdpCallTimeoutMs(params) }); + } case "automation.status": { // Report this session's owned automation target (ids only). Used for diagnostics/tests. await hydrateAutomationTargets(); @@ -1557,7 +1850,7 @@ async function executeInTab(params, func, args) { // Phase 2: run the action via chrome.scripting.executeScript. The `func:` form is // injected by Chrome itself (not `new Function`), so it is CSP-safe, and it lets Chrome // serialize the invocation args. The wrapper references window.__piAction defined above. - const results = await executeScriptTimed({ + const results = await executeScriptWithFallback({ target: { tabId: tab.id }, world: "MAIN", func: async (invocationArgs) => { @@ -1639,13 +1932,45 @@ async function evaluateInTab(params) { return v; } +// Run an input action, then optionally snapshot. An action that starts a navigation (Enter submits a +// form, a click follows a link) used to be snapshotted immediately, so the snapshot described the +// OUTGOING document and read as "the action did nothing". Anchor the tab's url/status first; when +// the action plausibly started a load, wait (bounded) for it before snapshotting and report +// navigation {from, to, settled, waitedMs}. A timeout never fails the action. A tab that was already +// loading with the same URL is reported but not waited on, so a hanging subresource cannot add a wait +// to every action. +const SNAPSHOT_NAVIGATION_WAIT_MS = 5_000; async function withOptionalSnapshot(params, actionFn) { - const result = await actionFn(params); - if (params.includeSnapshot) { - const snapshot = await snapshotInTab({ ...params, foreground: false }); - return { result, snapshot }; + if (!params.includeSnapshot) return actionFn(params); + let anchor = null; + try { + const tab = await getTabByParams(params); + if (tab && typeof tab.id === "number") { + anchor = { tabId: tab.id, params: { ...params, targetId: String(tab.id) }, url: String(tab.url ?? ""), status: String(tab.status ?? "") }; + } + } catch { + anchor = null; // Let the action surface the targeting error itself. + } + const result = await actionFn(anchor ? anchor.params : params); + let navigation; + if (anchor) { + const after = await chrome.tabs.get(anchor.tabId).catch(() => null); + const afterUrl = String(after?.url ?? ""); + const urlChanged = Boolean(afterUrl && afterUrl !== anchor.url); + const loading = after?.status === "loading"; + if (urlChanged || loading) { + const started = Date.now(); + const startedLoading = loading && (urlChanged || anchor.status === "complete"); + let settled = !loading; + if (startedLoading) { + settled = await waitForTabSettled(anchor.tabId, SNAPSHOT_NAVIGATION_WAIT_MS); + } + const finalTab = await chrome.tabs.get(anchor.tabId).catch(() => after); + navigation = { from: anchor.url, to: String(finalTab?.url ?? afterUrl), settled, waitedMs: Date.now() - started }; + } } - return result; + const snapshot = await snapshotInTab({ ...(anchor ? anchor.params : params), foreground: false }); + return navigation ? { result, snapshot, navigation } : { result, snapshot }; } // Snapshot/inspect run from a packaged MAIN-world script (snapshot_injected.js) injected via @@ -1664,12 +1989,12 @@ async function snapshotInTab(params) { params.query ?? null, params.maxTextChars ?? null, ]; - await executeScriptTimed({ + await executeScriptWithFallback({ target: { tabId: tab.id, frameIds: [0] }, world: "MAIN", files: ["snapshot_injected.js"], }, `inject snapshot script in tab ${tab.id}`); - const results = await executeScriptTimed({ + const results = await executeScriptWithFallback({ target: { tabId: tab.id, frameIds: [0] }, world: "MAIN", func: async (invocationArgs) => { @@ -1700,12 +2025,12 @@ async function inspectInTab(params) { const tab = await getTabByParams(params); await bringToFront(tab, params); const args = [params.uid ?? null, params.selector ?? null, params.scrollIntoView === true]; - await executeScriptTimed({ + await executeScriptWithFallback({ target: { tabId: tab.id, frameIds: [0] }, world: "MAIN", files: ["snapshot_injected.js"], }, `inject inspect script in tab ${tab.id}`); - const results = await executeScriptTimed({ + const results = await executeScriptWithFallback({ target: { tabId: tab.id, frameIds: [0] }, world: "MAIN", func: async (invocationArgs) => { @@ -1796,6 +2121,31 @@ function waitForTabComplete(tabId, timeoutMs) { }); } +// Like waitForTabComplete, but for a load that may already be underway (or already finished): +// re-checks the tab after subscribing so a completion between the caller's read and the listener +// registration is not missed. Resolves true when complete, false on timeout; never throws. +function waitForTabSettled(tabId, timeoutMs) { + return new Promise((resolve) => { + let done = false; + const finish = (value) => { + if (done) return; + done = true; + clearTimeout(timer); + chrome.tabs.onUpdated.removeListener(listener); + resolve(value); + }; + const timer = setTimeout(() => finish(false), timeoutMs); + const listener = (updatedTabId, changeInfo) => { + if (updatedTabId === tabId && changeInfo.status === "complete") finish(true); + }; + chrome.tabs.onUpdated.addListener(listener); + chrome.tabs.get(tabId).then((tab) => { + if (!tab) finish(false); + else if (tab.status === "complete") finish(true); + }, () => finish(false)); + }); +} + async function captureTabScreenshot(tabId, params) { const format = params.format || "png"; try { diff --git a/extensions/chrome-profile-bridge/index.ts b/extensions/chrome-profile-bridge/index.ts index 1917956..46be3d9 100644 --- a/extensions/chrome-profile-bridge/index.ts +++ b/extensions/chrome-profile-bridge/index.ts @@ -173,8 +173,67 @@ function formatChromeSnapshot(snapshot: any): string { } function formatIncludedSnapshotText(raw: unknown, text: string): string { - const snapshot = raw && typeof raw === "object" ? (raw as { snapshot?: unknown }).snapshot : undefined; - return snapshot ? `${text}\n\n${formatChromeSnapshot(snapshot)}` : text; + const payload = raw && typeof raw === "object" + ? (raw as { snapshot?: unknown; navigation?: { settled?: boolean; from?: string; to?: string; waitedMs?: number } }) + : undefined; + const snapshot = payload?.snapshot; + const navigation = payload?.navigation; + let body = text; + if (navigation) { + body += navigation.settled === false + ? `\n⚠ The action started a navigation that had not finished after ${navigation.waitedMs ?? "?"}ms (${navigation.from || "?"} → ${navigation.to || "?"}). The snapshot below may describe the page being replaced; re-check with chrome_snapshot.` + : `\nNavigated ${navigation.from || "?"} → ${navigation.to || "?"} (waited ${navigation.waitedMs ?? "?"}ms for load).`; + } + return snapshot ? `${body}\n\n${formatChromeSnapshot(snapshot)}` : body; +} + +// Pi-side text for chrome_type's before/after evidence. The worker returns valueBefore/valueAfter +// (or valueRedacted), existingTextLengthBefore, and insertedAt. +function describeTypeEvidence(result: unknown, typedLength: number, pressEnter: boolean): string[] { + if (!result || typeof result !== "object") return []; + const r = result as Record<string, unknown>; + const lines: string[] = []; + if (r.valueRedacted === true) { + lines.push(`Field value [redacted] (${r.existingTextLengthBefore ?? "?"} → ${r.valueLengthAfter ?? "?"} chars; insertedAt=${r.insertedAt ?? "unknown"}).`); + } else if (typeof r.valueBefore === "string" || typeof r.valueAfter === "string") { + const before = typeof r.valueBefore === "string" ? JSON.stringify(r.valueBefore) : "?"; + const after = typeof r.valueAfter === "string" ? JSON.stringify(r.valueAfter) : "?"; + lines.push(`Field went from ${before} to ${after}${r.insertedAt ? ` (insertedAt=${r.insertedAt})` : ""}.`); + } + const unchanged = typedLength > 0 && r.replaced !== true + && typeof r.existingTextLengthBefore === "number" && r.existingTextLengthBefore === r.valueLengthAfter + && (r.valueRedacted === true || r.valueBefore === r.valueAfter); + if (unchanged) { + lines.push("\u26a0 The field value did not change: the keystrokes did not reach this field. It may not have keyboard focus (background/hidden tabs often cannot take focus); focus it with a uid/selector click, or ask the user to run /chrome background off, then verify."); + } else if (r.insertedAt === "caret-middle" && typedLength > 0) { + lines.push("⚠ Text was spliced into the middle of existing content; chrome_type does NOT replace. Use chrome_fill (or chrome_type replace=true) to replace a field's value."); + if (pressEnter) lines.push("⚠ If Enter submitted the form, it submitted the spliced value above, not just your text."); + } else if (r.insertedAt === "caret-end" && typedLength > 0) { + lines.push("Note: text was appended to existing content (chrome_type does not replace)."); + } + return lines; +} + +// Keep raw CDP payloads (screenshots, huge DOM dumps) out of the model context and transcript. +const CDP_OVERSIZE_JSON_CHARS = 262_144; +function formatCdpResult(method: string, value: unknown): ToolTextResult { + const data = value && typeof value === "object" ? (value as { data?: unknown }).data : undefined; + if (typeof data === "string" && (/captureScreenshot|printToPDF|screencast/i.test(method) || data.length >= CDP_OVERSIZE_JSON_CHARS)) { + const padding = data.endsWith("==") ? 2 : data.endsWith("=") ? 1 : 0; + const bytes = Math.floor((data.length / 4) * 3) - padding; + const fields = Object.keys(value as object).filter((key) => key !== "data"); + const text = `CDP ${method} returned ~${bytes} bytes in its "data" field; the payload was omitted to protect the context window. Use chrome_screenshot to save images to disk.${fields.length ? ` Other fields: ${fields.join(", ")}.` : ""}`; + return { content: [{ type: "text", text }], details: { value: { omitted: "data-field", bytes, fields } } }; + } + const text = value === undefined ? "undefined" : typeof value === "string" ? value : (safeJson(value) ?? "undefined"); + if (text.length > CDP_OVERSIZE_JSON_CHARS) { + const fields = value && typeof value === "object" ? Object.keys(value as object) : []; + return { + content: [{ type: "text", text: `${truncateText(text)}\n\n[details omitted: ${text.length} chars of JSON]` }], + details: { value: { omitted: "oversized-result", chars: text.length, fields } }, + }; + } + return { content: [{ type: "text", text: truncateText(text) }], details: { value } }; } function formatChromeInspect(inspect: any): string { @@ -339,6 +398,31 @@ class ChromeProfileBridge { await this.bindServerOrClient(); } + // A client-mode session (another Pi session or a subagent owns the port) never sees extension + // polls itself, so its local status always reads "not connected". Ask the owner instead. + async connectionStatus(): Promise<Record<string, unknown>> { + const local = this.status(); + if (this.mode !== "client") return local; + const controller = new AbortController(); + const timer = setTimeout(() => controller.abort(), 1_000); + try { + const response = await fetch(`${this.url}/status`, { signal: controller.signal }); + if (!response.ok) return local; + const owner = (await response.json()) as Record<string, unknown>; + return { + ...local, + connected: owner.connected === true, + lastSeenAt: typeof owner.lastSeenAt === "number" ? owner.lastSeenAt : local.lastSeenAt, + clientName: typeof owner.clientName === "string" ? owner.clientName : local.clientName, + ownerMode: owner.mode, + }; + } catch { + return local; + } finally { + clearTimeout(timer); + } + } + // Try to own the bridge port. On success we are the server; on EADDRINUSE another Pi // session owns it and we run as a client that forwards commands to that owner. private async bindServerOrClient(): Promise<void> { @@ -643,6 +727,8 @@ const CHROME_TOOL_NAMES = [ "chrome_launch", "chrome_tab", "chrome_snapshot", + "chrome_find", + "chrome_inspect", "chrome_navigate", "chrome_evaluate", "chrome_click", @@ -659,6 +745,8 @@ const CHROME_TOOL_NAMES = [ "chrome_tap", "chrome_scroll", "chrome_upload_file", + "chrome_cdp", + "chrome_cdp_targets", ] as const; const CHROME_TOOL_NAME_SET = new Set<string>(CHROME_TOOL_NAMES); @@ -674,16 +762,19 @@ export default function (pi: ExtensionAPI): void { [PI_CHROME_AUTH_KEY]?: { until: number | "indefinite" }; }; const alreadyLoaded = globalState[PI_CHROME_GLOBAL_KEY]; - if (alreadyLoaded?.token || (alreadyLoaded && alreadyLoaded.root !== currentRoot)) { + // Only a *different* install root (two copies of pi-chrome) is a duplicate. A same-root re-entry is + // legitimate: subagent sessions (e.g. pi-subagents) load extensions into their own runner in this + // process, so this factory runs once per session. Skipping it left subagents with no chrome_* tools. + // Each instance gets its own bridge; the second binds as a client of the port owner (EADDRINUSE), + // so all sessions share the one Chrome connector. Stale flags from older releases (<=0.15.19) that + // point at this same root are harmless for the same reason. + if (alreadyLoaded && alreadyLoaded.root !== currentRoot) { console.warn( `pi-chrome already loaded from ${alreadyLoaded.root} (v${alreadyLoaded.version}); skipping duplicate from ${currentRoot}.`, ); return; } - // pi-chrome <=0.15.19 set the singleton flag but did not clear it on reload. - // If the stale flag points at this same extension root, replace it instead of - // skipping the freshly reloaded extension. - globalState[PI_CHROME_GLOBAL_KEY] = { version: PI_CHROME_VERSION, root: currentRoot, token: instanceToken }; + if (!alreadyLoaded) globalState[PI_CHROME_GLOBAL_KEY] = { version: PI_CHROME_VERSION, root: currentRoot, token: instanceToken }; const bridge = new ChromeProfileBridge(DEFAULT_HOST, DEFAULT_PORT); let backgroundEnabled = true; @@ -904,11 +995,11 @@ export default function (pi: ExtensionAPI): void { if ((action === "tab.new" || action === "tab.group") && sessionTitle !== undefined) { wireParams = { ...wireParams, groupTitle: sessionTitle }; } - // Any tab Pi *uses* (page.* interactions) should join this session's group, mirroring the + // Any tab Pi *uses* (page.* interactions and raw cdp.call) should join this session's group, mirroring the // auto-grouping that tab.new already does. Tagging the wire params lets getTabByParams pull - // the resolved tab into the session group on the service-worker side. We skip tab.* actions: + // the resolved tab into the session group on the service-worker side. We skip tab.* and cdp.targets actions: // tab.new/group are forced above, and activate/close/ungroup/list must not group tabs. - const shouldJoinGroup = action.startsWith("page.") && sessionTitle !== undefined && params.sessionGroupTitle === undefined; + const shouldJoinGroup = (action.startsWith("page.") || action === "cdp.call") && sessionTitle !== undefined && params.sessionGroupTitle === undefined; if (shouldJoinGroup) { wireParams = { ...wireParams, sessionGroupTitle: sessionTitle, joinSessionGroup: true }; } @@ -974,7 +1065,9 @@ Capability model (important): - Interactive controls (click/type/fill/key/hover/drag/scroll/tap) use Chrome's real input layer via chrome.debugger / CDP. Events satisfy normal user-activation gates. - Input bypasses page CSP because it is injected at browser input layer, not page JavaScript. Chrome may show the “Pi Chrome Connector started debugging this browser” banner while attached. - \`chrome_evaluate\` and \`chrome_snapshot\` run in MAIN world via **CDP \`Runtime.evaluate\`**, which is not subject to the page's Content-Security-Policy. They work even on strict-CSP pages (e.g. github.com, many bank/SaaS apps) that block \`'unsafe-eval'\`. \`chrome_navigate initScript\` likewise injects at document_start via CDP and bypasses CSP. \`chrome_screenshot\`, \`chrome_tab\`, and Chrome input also work under any CSP. -- Input tools return structured details and support \`includeSnapshot=true\` on click/type/fill/key. Use the fresh snapshot to verify state instead of repeating blindly. +- Input tools return structured details and support \`includeSnapshot=true\` on click/type/fill/key. Use the fresh snapshot to verify state instead of repeating blindly. If the action started a navigation, the snapshot waits (up to 5s) for the new page. +- \`chrome_type\` inserts at the caret and never replaces; check its before/after report. Use \`chrome_fill\` to replace a field. +- \`chrome_cdp\` runs any raw CDP method on a tab (emulation, cookies, PDF, accessibility tree, etc.) when no dedicated chrome_* tool fits; \`chrome_cdp_targets\` diagnoses debugger/overlay conflicts. Usage rules: 1. If a chrome_* tool says Chrome control is locked, ask the user to run \`/chrome authorize\` before retrying. @@ -1311,9 +1404,10 @@ Usage rules: headless: Type.Optional(Type.Boolean({ description: "Ignored." })), }), async execute(_id, params, signal, _onUpdate, ctx): Promise<ToolTextResult> { - if (params.url && bridge.connected) { + const status = await bridge.connectionStatus(); + if (params.url && status.connected === true) { const result = await authorizedBridgeSend("tab.new", { url: params.url }, DEFAULT_TIMEOUT_MS, signal); - return { content: [{ type: "text", text: `Chrome bridge connected; opened ${params.url}` }], details: { status: bridge.status(), result } }; + return { content: [{ type: "text", text: `Chrome bridge connected; opened ${params.url}` }], details: { status, result } }; } return { content: [ @@ -1326,10 +1420,10 @@ Usage rules: `2. Enable Developer mode.\n` + `3. Click “Load unpacked”.\n` + `4. Select: ${browserExtensionPath()}\n\n` + - `Status: ${bridge.connected ? "connected" : "waiting for extension"}.`, + `Status: ${status.connected === true ? "connected" : "waiting for extension"}.`, }, ], - details: { status: bridge.status(), extensionPath: browserExtensionPath() }, + details: { status, extensionPath: browserExtensionPath() }, }; }, }); @@ -1560,12 +1654,13 @@ Usage rules: name: "chrome_type", label: "Chrome Type", description: - "Focus an optional snapshot uid or CSS selector, then type using Chrome's real input. Contenteditables use one native text insertion; other fields use key events. Set perCharacter=true for editors needing individual keydown events. Pass includeSnapshot=true to verify after typing.", - promptSnippet: "Type text into Chrome, optionally focusing a snapshot uid or selector first.", + "Focus an optional snapshot uid or CSS selector, then type at the caret using Chrome's real input. It does NOT replace existing text: in a non-empty field the text is inserted wherever the caret lands after the focus click. Use chrome_fill (or replace=true) to replace a field's value. Contenteditables use one native text insertion; other fields use key events. Set perCharacter=true for editors needing individual keydown events. The result reports the field value before/after and insertedAt (empty|caret-end|caret-middle|replaced-selection|replaced-all), with a warning when text was spliced into existing content. Pass includeSnapshot=true to verify after typing; if the action starts a navigation, the snapshot waits (up to 5s) for it to load.", + promptSnippet: "Type text at the caret in Chrome (does not replace existing text; use chrome_fill or replace=true).", parameters: Type.Object({ text: Type.String(), uid: Type.Optional(Type.String({ description: "Stable element uid from chrome_snapshot." })), selector: Type.Optional(Type.String({ description: "CSS selector to focus before typing." })), + replace: Type.Optional(Type.Boolean({ default: false, description: "If true, select all of the focused field's contents and delete them before typing (real key events). Reports replaced:true." })), perCharacter: Type.Optional(Type.Boolean({ default: false, description: "Send individual key events even in contenteditables. Default: one native text insertion for contenteditables; key events for other fields." })), includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, include a fresh chrome_snapshot result after typing." })), maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })), @@ -1582,9 +1677,9 @@ Usage rules: const result = (params.includeSnapshot ? (raw as { result: unknown }).result : raw) as Json; const summary = summarizeActionResult(result); const into = params.uid || params.selector ? ` into ${params.uid ?? params.selector}` : ""; - const base = `Typed ${params.text.length} character(s)${into}.`; - const text = summary ? `${base} (${summary})` : base; - return { content: [{ type: "text", text: formatIncludedSnapshotText(raw, text) }], details: { result: raw as Json } }; + const base = `Typed ${params.text.length} character(s)${into}${params.replace ? " (replacing existing contents)" : ""}.`; + const lines = [summary ? `${base} (${summary})` : base, ...describeTypeEvidence(result, params.text.length, params.pressEnter === true)]; + return { content: [{ type: "text", text: formatIncludedSnapshotText(raw, lines.join("\n")) }], details: { result: raw as Json } }; }, }); @@ -1592,7 +1687,7 @@ Usage rules: name: "chrome_fill", label: "Chrome Fill", description: - "Set the full value of a text input, textarea, or contenteditable using Chrome click/select/delete/type input. Contenteditables use one native text insertion; perCharacter=true retains individual keydown events. Accepts a snapshot uid or CSS selector. Pass includeSnapshot=true to verify after filling.", + "Replace the whole value of a text input, textarea, or contenteditable using Chrome click/select/delete/type input. Unlike chrome_type, existing contents are cleared first. Contenteditables use one native text insertion; perCharacter=true retains individual keydown events. Accepts a snapshot uid or CSS selector. Pass includeSnapshot=true to verify after filling.", promptSnippet: "Fill a Chrome form field by snapshot uid or selector, optionally returning a fresh snapshot.", parameters: Type.Object({ text: Type.String(), @@ -1910,6 +2005,64 @@ Usage rules: return { content: [{ type: "text", text: `Uploaded ${paths.length} file(s) to ${params.uid ?? params.selector}` }], details: { result: result as Json } }; }, }); + + pi.registerTool({ + name: "chrome_cdp", + label: "Chrome CDP Call", + description: + "Low-level escape hatch: run one Chrome DevTools Protocol (CDP) method against a tab (no target = this session's automation tab), e.g. Emulation.setDeviceMetricsOverride, Network.getCookies, DOM.getDocument, Page.printToPDF, Accessibility.getFullAXTree. Put CDP fields inside params. Nothing is filtered against a safe list: destructive methods run as given. Prefer dedicated chrome_* tools when they cover the task. Background mode blocks Page.bringToFront and Target.activateTarget, but other methods can still change what the user sees. Screenshot/binary payloads and results over 256 KiB are summarised instead of returned in full. Domain events are not streamed back; only the method's direct result is returned.", + promptSnippet: "Run a raw Chrome DevTools Protocol method against a Chrome tab (low-level escape hatch).", + parameters: Type.Object({ + method: Type.String({ description: "CDP method name, e.g. \"Runtime.evaluate\" or \"Emulation.setDeviceMetricsOverride\"." }), + params: Type.Optional(Type.Object({}, { additionalProperties: true, description: "CDP parameter object for the method." })), + timeoutMs: Type.Optional(Type.Number({ description: "Deadline for the CDP command in milliseconds. Default 5000, max 120000. On timeout the debugger session is detached and the next call re-attaches." })), + targetId: Type.Optional(Type.String()), + urlIncludes: Type.Optional(Type.String()), + titleIncludes: Type.Optional(Type.String()), + background: Type.Optional(Type.Boolean({ description: BACKGROUND_PARAM_DESCRIPTION })), + }), + async execute(_id, params, signal): Promise<ToolTextResult> { + // Raw CDP fields passed at the top level would reach Chrome as params:{} and fail with an + // opaque "Invalid parameters". Point the caller at params instead. + const knownKeys = new Set(["method", "params", "timeoutMs", "targetId", "urlIncludes", "titleIncludes", "background", "foreground", "host", "port"]); + const unknownKeys = Object.keys(params).filter((key) => !knownKeys.has(key)); + if (unknownKeys.length > 0) { + throw new Error(`chrome_cdp received unknown top-level parameter(s): ${unknownKeys.join(", ")}. Put CDP fields inside "params", e.g. { method: "Runtime.evaluate", params: { expression: "1+1" } }.`); + } + // Keep the bridge deadline above the worker's (CDP deadline + 5s grace) so the precise + // "CDP <method> timed out" error wins. + const requested = Number(params.timeoutMs); + const bridgeTimeoutMs = Number.isFinite(requested) && requested > 0 + ? Math.max(DEFAULT_TIMEOUT_MS, Math.min(Math.floor(requested), 120_000) + 10_000) + : DEFAULT_TIMEOUT_MS; + const value = await authorizedBridgeSend("cdp.call", params, bridgeTimeoutMs, signal); + return formatCdpResult(params.method, value); + }, + }); + + pi.registerTool({ + name: "chrome_cdp_targets", + label: "Chrome CDP Targets", + description: + "List Chrome DevTools Protocol targets attached to a tab (type, url, attached, extensionId). Use it to diagnose chrome_* / chrome_cdp failures such as \"Detached while handling command\": password-manager/autofill overlays and DevTools front-ends show up here. Targets on other tabs are only counted. Does not attach the debugger or create an automation tab; with no target it reports this session's automation tab if one exists.", + promptSnippet: "List CDP targets (including foreign extension overlays) attached to a Chrome tab.", + parameters: Type.Object({ + targetId: Type.Optional(Type.String()), + urlIncludes: Type.Optional(Type.String()), + titleIncludes: Type.Optional(Type.String()), + }), + async execute(_id, params, signal): Promise<ToolTextResult> { + const value = await authorizedBridgeSend("cdp.targets", params, DEFAULT_TIMEOUT_MS, signal); + const result = value as { tab?: { id?: number; title?: string; url?: string } | null; targets?: Array<{ type?: string; attached?: boolean; url?: string; extensionId?: string }>; otherTabTargetCount?: number } | undefined; + const targets = result?.targets ?? []; + const text = [ + result?.tab ? `Tab ${result.tab.id}: ${result.tab.title || "(untitled)"} \u2014 ${result.tab.url ?? ""}` : "No resolved tab (pass targetId/urlIncludes/titleIncludes, or run chrome_navigate first).", + `${targets.length} CDP target(s) on this tab${result?.otherTabTargetCount ? ` (${result.otherTabTargetCount} on other tabs, not listed)` : ""}:`, + ...targets.map((t) => `- ${t.type ?? "?"}\t${t.attached ? "attached" : "detached"}\t${t.extensionId ? `ext=${t.extensionId}\t` : ""}${t.url ?? ""}`), + ].join("\n"); + return { content: [{ type: "text", text: truncateText(text) }], details: { value: (value ?? null) as Json } }; + }, + }); } } diff --git a/package-lock.json b/package-lock.json index 930e3b1..a850062 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "pi-chrome", - "version": "0.15.51", + "version": "0.15.53", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "pi-chrome", - "version": "0.15.51", + "version": "0.15.53", "license": "MIT", "peerDependencies": { "@earendil-works/pi-coding-agent": "*", diff --git a/package.json b/package.json index 9f0af8c..3ae9aa3 100644 --- a/package.json +++ b/package.json @@ -1,8 +1,8 @@ { "name": "pi-chrome", - "version": "0.15.51", + "version": "0.15.53", "scripts": { - "test": "node test-suite/unit/csp-eval.test.mjs && node test-suite/unit/automation-target.test.mjs && node test-suite/unit/session-cleanup.test.mjs && node test-suite/unit/background-policy.test.mjs && node test-suite/unit/input-reliability.test.mjs && node test-suite/unit/chrome-command.test.mjs", + "test": "node test-suite/unit/csp-eval.test.mjs && node test-suite/unit/automation-target.test.mjs && node test-suite/unit/session-cleanup.test.mjs && node test-suite/unit/background-policy.test.mjs && node test-suite/unit/input-reliability.test.mjs && node test-suite/unit/chrome-command.test.mjs && node test-suite/unit/bridge-resilience.test.mjs && node test-suite/unit/cdp-passthrough.test.mjs && node test-suite/unit/type-evidence.test.mjs", "version": "node scripts/sync-manifest-version.js", "prepublishOnly": "node scripts/sync-manifest-version.js" }, diff --git a/test-suite/unit/background-policy.test.mjs b/test-suite/unit/background-policy.test.mjs index e78fa49..fe71634 100644 --- a/test-suite/unit/background-policy.test.mjs +++ b/test-suite/unit/background-policy.test.mjs @@ -24,7 +24,7 @@ function piHarness({ session = "alpha", send } = {}) { let authorized = true; const ctx = { key: `session:${session}`, title: `Pi Session: ${session}`, cwd: "/fixture", ui: { notify: (...args) => notices.push(args) } }; const bridge = { - connected: true, status: () => ({}), + connected: true, status: () => ({}), connectionStatus: async () => ({ connected: true }), async send(action, params, timeout, signal) { calls.push({ action, params: clone(params), timeout, signal }); if (signal?.aborted) throw new Error("Chrome command aborted"); @@ -48,6 +48,7 @@ function piHarness({ session = "alpha", send } = {}) { tabActionValues: [], snapshotModeValues: [], waitForValues: [], imageFormatValues: [], safeJson: JSON.stringify, truncateText: (s) => s, formatChromeSnapshot: JSON.stringify, formatChromeInspect: JSON.stringify, summarizeActionResult: () => "", formatIncludedSnapshotText: (_r, text) => text, + describeTypeEvidence: () => [], formatCdpResult: (_method, value) => ({ content: [{ type: "text", text: JSON.stringify(value) }], details: { value } }), workspaceCwd: () => ctx.cwd, ...path, mkdir: async () => {}, writeFile: async (...args) => writes.push(args), }; @@ -107,6 +108,7 @@ test("every registered page tool, tab.new, and chrome_launch(url) use the centra ["chrome_get_network_request", { requestId: "1" }, "page.network.get"], ["chrome_screenshot", {}, "page.screenshot"], ["chrome_hover", {}, "page.hover"], ["chrome_drag", {}, "page.drag"], ["chrome_tap", {}, "page.tap"], ["chrome_scroll", {}, "page.scroll"], ["chrome_upload_file", { paths: ["fixture.txt"] }, "page.upload"], + ["chrome_cdp", { method: "Runtime.evaluate", params: { expression: "1" } }, "cdp.call"], ]; for (const [name, params, action] of cases) { await h.tool(name, { ...params, background: false, foreground: true }); diff --git a/test-suite/unit/bridge-resilience.test.mjs b/test-suite/unit/bridge-resilience.test.mjs new file mode 100644 index 0000000..47495fc --- /dev/null +++ b/test-suite/unit/bridge-resilience.test.mjs @@ -0,0 +1,138 @@ +// Bridge transport resilience in the shipped worker, plus same-process (subagent) loading in index.ts. +// Worker timers are scaled 1000x down so the real 45s /next deadline elapses in ~45ms. +import assert from "node:assert/strict"; +import fs from "node:fs"; +import vm from "node:vm"; +import { stripTypeScriptTypes } from "node:module"; +import { test } from "node:test"; + +const workerSource = fs.readFileSync(new URL("../../extensions/chrome-profile-bridge/browser-extension/service_worker.js", import.meta.url), "utf8"); +const indexSource = fs.readFileSync(new URL("../../extensions/chrome-profile-bridge/index.ts", import.meta.url), "utf8"); + +function loadWorker(fetchImpl) { + const listener = { addListener() {}, removeListener() {} }; + const chrome = { + runtime: { id: "test", getManifest: () => ({ version: "0.0.1" }), getURL: (f) => `chrome-extension://test/${f}`, onInstalled: listener, onStartup: listener, reload() {} }, + alarms: { create() {}, onAlarm: listener }, action: { onClicked: listener }, webNavigation: { onCommitted: listener }, + debugger: { onDetach: listener }, scripting: {}, tabs: { onUpdated: listener }, + }; + const sandbox = { + chrome, console, AbortController, URL, + navigator: { userAgent: "unit-test" }, + setTimeout: (fn, ms, ...args) => setTimeout(fn, Math.ceil((ms || 0) / 1000), ...args), + clearTimeout, setInterval: () => 0, clearInterval() {}, + fetch: fetchImpl, + }; + sandbox.self = sandbox; + vm.runInNewContext(workerSource, sandbox); + return sandbox; +} + +function response(status, body, headers = {}) { + return { ok: status >= 200 && status < 300, status, headers: { get: (name) => headers[name.toLowerCase()] ?? null }, json: async () => body }; +} + +test("a half-open /next long poll is aborted at its deadline and polling resumes", async () => { + const requests = []; + let aborted = 0; + const worker = loadWorker((url, options = {}) => { + requests.push({ url, options }); + if (requests.length === 1) { + // Zombie socket: never settles unless aborted. + return new Promise((_resolve, reject) => { + options.signal.addEventListener("abort", () => { + aborted++; + const error = new Error("The operation was aborted"); + error.name = "AbortError"; + reject(error); + }); + }); + } + return Promise.reject(new Error("bridge down")); + }); + const first = worker.pollLoop(); + assert.ok(requests[0]?.options.signal, "the /next fetch carries an abort signal"); + // Resolves only after the scaled 45s deadline fires and the 2s backoff elapses. + await first; + assert.equal(aborted, 1, "the stalled /next fetch was aborted"); + await worker.pollLoop(); + assert.equal(requests.length, 2, "a later poll opens a fresh /next request instead of staying parked"); +}); + +test("a command result is posted exactly once, retried on transport failure, and not retried on 4xx", async () => { + const posts = []; + let postStatus = [500, 200]; + const worker = loadWorker(async (url, options = {}) => { + if (url.includes("/result")) { + posts.push(JSON.parse(options.body)); + const status = postStatus.shift() ?? 200; + if (status === 0) throw new Error("fetch failed"); + return response(status, {}); + } + throw new Error("not polling in this test"); + }); + await worker.handleCommand({ id: "cmd-1", action: "tab.version", params: {} }); + assert.equal(posts.length, 2, "one retry after HTTP 500"); + assert.ok(posts.every((p) => p.id === "cmd-1" && p.ok === true), "the retry re-sends the success payload, not an error payload"); + + posts.length = 0; + postStatus = [0, 0, 0]; + await worker.handleCommand({ id: "cmd-2", action: "tab.version", params: {} }); + assert.equal(posts.length, 3, "network failures retry up to the attempt limit"); + assert.ok(posts.every((p) => p.ok === true), "a failed success-post never turns into a second, error result"); + + posts.length = 0; + postStatus = [404]; + await worker.handleCommand({ id: "cmd-3", action: "no.such.action", params: {} }); + assert.equal(posts.length, 1, "4xx (unknown command id) is final"); + assert.equal(posts[0].ok, false); + assert.match(posts[0].error, /Unknown action/); +}); + +test("cdp.call widens only its own command deadline", () => { + const worker = loadWorker(async () => { throw new Error("offline"); }); + assert.equal(worker.commandTimeoutMs("page.click", { timeoutMs: 90_000 }), 25_000); + assert.equal(worker.commandTimeoutMs("cdp.call", {}), 25_000); + assert.equal(worker.commandTimeoutMs("cdp.call", { timeoutMs: 60_000 }), 65_000); + assert.equal(worker.commandTimeoutMs("cdp.call", { timeoutMs: 10_000_000 }), 125_000, "capped below the MV3 worker lifetime"); +}); + +// ---- index.ts: a subagent session loads pi-chrome again in the same process. ---- +function loadFactoryPrelude(globalState, root) { + const from = indexSource.indexOf("\tconst alreadyLoaded = globalState[PI_CHROME_GLOBAL_KEY];"); + const to = indexSource.indexOf("\tconst bridge = new ChromeProfileBridge(", from); + assert.ok(from > 0 && to > from, "factory prelude section"); + const body = stripTypeScriptTypes(`(() => {\n${indexSource.slice(from, to)}\nreturn true;\n})()`); + const warnings = []; + const token = Symbol(root); + const sandbox = { + globalState, currentRoot: root, instanceToken: token, PI_CHROME_GLOBAL_KEY: "loaded", PI_CHROME_VERSION: "9.9.9", + console: { warn: (msg) => warnings.push(msg) }, + }; + const loaded = vm.runInNewContext(body, sandbox); + return { loaded: loaded === true, token, warnings }; +} + +test("a same-root second load (subagent session) is not skipped, and does not steal the singleton flag", () => { + const globalState = {}; + const parent = loadFactoryPrelude(globalState, "/pkg/pi-chrome"); + assert.equal(parent.loaded, true); + assert.equal(globalState.loaded.token, parent.token); + const subagent = loadFactoryPrelude(globalState, "/pkg/pi-chrome"); + assert.equal(subagent.loaded, true, "subagent gets its own pi-chrome instance and chrome_* tools"); + assert.equal(subagent.warnings.length, 0); + assert.equal(globalState.loaded.token, parent.token, "only the first instance owns (and later clears) the flag"); +}); + +test("a second install root is still treated as a duplicate", () => { + const globalState = {}; + loadFactoryPrelude(globalState, "/npm/pi-chrome"); + const other = loadFactoryPrelude(globalState, "/checkout/pi-chrome"); + assert.equal(other.loaded, false); + assert.match(other.warnings[0], /already loaded from \/npm\/pi-chrome/); +}); + +test("a stale same-root flag from an older release does not block loading", () => { + const globalState = { loaded: { version: "0.15.19", root: "/pkg/pi-chrome" } }; + assert.equal(loadFactoryPrelude(globalState, "/pkg/pi-chrome").loaded, true); +}); diff --git a/test-suite/unit/cdp-passthrough.test.mjs b/test-suite/unit/cdp-passthrough.test.mjs new file mode 100644 index 0000000..c14c4d7 --- /dev/null +++ b/test-suite/unit/cdp-passthrough.test.mjs @@ -0,0 +1,183 @@ +// Raw CDP passthrough (cdp.call / cdp.targets / chrome_cdp formatting) and the scripting→CDP fallback +// that keeps page actions working on about:blank automation tabs. +import assert from "node:assert/strict"; +import fs from "node:fs"; +import vm from "node:vm"; +import { stripTypeScriptTypes } from "node:module"; +import { test } from "node:test"; + +const workerSource = fs.readFileSync(new URL("../../extensions/chrome-profile-bridge/browser-extension/service_worker.js", import.meta.url), "utf8"); +const indexSource = fs.readFileSync(new URL("../../extensions/chrome-profile-bridge/index.ts", import.meta.url), "utf8"); +const ABOUT_BLANK_DENIED = 'Cannot access contents of url "about:blank". Extension manifest must request permission to access this host.'; + +function harness({ targets = [], respond } = {}) { + const calls = [], attached = [], hung = []; + const listener = { addListener() {}, removeListener() {} }; + const page = vm.createContext({ location: { href: "about:blank" } }); + page.window = page; + page.globalThis = page; + const chrome = { + runtime: { id: "test", getManifest: () => ({ version: "0.0.1" }), getURL: (f) => `chrome-extension://test/${f}`, onInstalled: listener, onStartup: listener, lastError: null }, + alarms: { create() {}, onAlarm: listener }, action: { onClicked: listener }, webNavigation: { onCommitted: listener }, + debugger: { + onDetach: listener, + getTargets: (cb) => cb(targets), + attach: async () => {}, + // Real Chrome fails in-flight commands with "Detached while handling command" on detach. + detach: async () => { + for (const cb of hung.splice(0)) { + chrome.runtime.lastError = { message: "Detached while handling command." }; + cb(undefined); + chrome.runtime.lastError = null; + } + }, + sendCommand: (debuggee, method, params, cb) => { + calls.push({ method, params: JSON.parse(JSON.stringify(params)) }); + const reply = respond ? respond(method, params) : undefined; + if (reply === "never") { hung.push(cb); return; } // simulate a hung CDP command + Promise.resolve(reply).then((value) => { + if (method === "Runtime.evaluate" && value === undefined) { + Promise.resolve().then(() => vm.runInContext(params.expression, page)).then( + (v) => { const json = JSON.stringify(v); cb({ result: json === undefined ? { type: typeof v } : { value: JSON.parse(json) } }); }, + (error) => cb({ exceptionDetails: { text: error.message, exception: { description: error.message } } }), + ); + return; + } + cb(value); + }); + }, + }, + scripting: { executeScript: async () => { throw new Error(ABOUT_BLANK_DENIED); } }, + tabs: { onUpdated: listener, get: async (id) => ({ id, windowId: 1, url: "about:blank", status: "complete" }) }, + }; + const sources = new Map(); + const worker = { + chrome, console, AbortController, URL, + navigator: { userAgent: "unit-test" }, + setTimeout, clearTimeout, setInterval: () => 0, clearInterval() {}, + fetch: async (url) => { + const file = String(url).replace("chrome-extension://test/", ""); + if (!sources.has(file)) return { ok: false, status: 404, text: async () => "" }; + return { ok: true, status: 200, text: async () => sources.get(file) }; + }, + }; + worker.self = worker; + vm.runInNewContext(workerSource, worker); + worker.getTabByParams = async (params, opts = {}) => { + if (params.targetId === "missing") throw new Error("No Chrome tab with id 999"); + if (opts.createOwnedTarget === false && params.targetId === undefined) return null; + return { id: Number(params.targetId ?? 5), windowId: 1, url: "https://fixture.test/", title: "Fixture" }; + }; + worker.bringToFront = async (tab) => tab; + const realAttach = worker.attachDebugger; + worker.attachDebugger = async (tabId) => { attached.push(tabId); return realAttach(tabId); }; + return { worker, chrome, calls, attached, page, sources, cdpMethods: () => calls.map((c) => c.method) }; +} + +test("cdp.call validates method/params before touching the debugger", async () => { + const h = harness(); + await assert.rejects(h.worker.dispatch("cdp.call", { method: " " }), /non-empty string "method"/); + await assert.rejects(h.worker.dispatch("cdp.call", { method: "Runtime.evaluate", params: [1] }), /must be a plain object/); + await assert.rejects(h.worker.dispatch("cdp.call", { method: "Runtime.evaluate", params: "x" }), /must be a plain object/); + assert.equal(h.attached.length, 0); + assert.equal(h.calls.length, 0); +}); + +test("cdp.call sends the exact method and params to the resolved tab and returns the raw result", async () => { + const h = harness({ respond: (method) => method === "Network.getCookies" ? { cookies: [{ name: "sid" }] } : {} }); + const result = await h.worker.dispatch("cdp.call", { targetId: "7", method: " Network.getCookies ", params: { urls: ["https://fixture.test"] } }); + assert.deepEqual(JSON.parse(JSON.stringify(result)), { cookies: [{ name: "sid" }] }); + assert.deepEqual(h.attached, [7]); + assert.deepEqual(h.calls.at(-1), { method: "Network.getCookies", params: { urls: ["https://fixture.test"] } }); +}); + +test("background mode blocks focus-stealing raw CDP methods; foreground allows them", async () => { + const h = harness(); + for (const method of ["Page.bringToFront", "Target.activateTarget"]) { + await assert.rejects(h.worker.dispatch("cdp.call", { method, background: true, foreground: false }), /blocked by background mode/); + } + assert.equal(h.calls.length, 0); + await h.worker.dispatch("cdp.call", { method: "Page.bringToFront", background: false, foreground: true }); + assert.equal(h.cdpMethods().at(-1), "Page.bringToFront"); +}); + +test("cdp.call honours timeoutMs and detaches a hung session", async () => { + const h = harness({ respond: (method) => method === "Debugger.pause" ? "never" : undefined }); + const started = Date.now(); + await assert.rejects(h.worker.dispatch("cdp.call", { method: "Debugger.pause", timeoutMs: 60 }), /CDP Debugger\.pause timed out after 60ms/); + assert.ok(Date.now() - started < 2_000, "custom deadline used instead of a longer default"); + assert.equal(h.cdpMethods().filter((m) => m === "Debugger.pause").length, 1, "a timed-out command is not re-sent after the cleanup detach"); + assert.equal(h.worker.cdpCallTimeoutMs({}), 5_000); + assert.equal(h.worker.cdpCallTimeoutMs({ timeoutMs: 999_999 }), 120_000); +}); + +test("cdp.targets lists only the resolved tab's targets, never creates a target, and surfaces explicit misses", async () => { + const h = harness({ targets: [ + { id: "a", tabId: 5, type: "page", url: "https://fixture.test/", attached: true }, + { id: "b", tabId: 5, type: "other", url: "chrome-extension://pw/overlay.html", attached: false, extensionId: "pw" }, + { id: "c", tabId: 9, type: "page", url: "https://private.example/secret", attached: false }, + ] }); + const listed = await h.worker.dispatch("cdp.targets", { targetId: "5" }); + assert.deepEqual(listed.targets.map((t) => t.id), ["a", "b"]); + assert.equal(listed.otherTabTargetCount, 1); + assert.ok(!JSON.stringify(listed).includes("private.example"), "other tabs' URLs are not echoed"); + const none = await h.worker.dispatch("cdp.targets", {}); + assert.equal(none.tab, null); + assert.equal(none.targets.length, 0); + await assert.rejects(h.worker.dispatch("cdp.targets", { targetId: "missing" }), /No Chrome tab with id/); + assert.equal(h.attached.length, 0, "diagnostics never attach the debugger"); +}); + +test("scripting denied on about:blank falls back to CDP for func and packaged-file injection", async () => { + const h = harness(); + h.sources.set("snapshot_injected.js", "globalThis.__piChromeSnapshotPage = async (max, _c, _r, _n, mode) => ({ url: location.href, max, mode });"); + const snapshot = await h.worker.dispatch("page.snapshot", { targetId: "5", maxElements: 12, mode: "forms" }); + assert.deepEqual(JSON.parse(JSON.stringify(snapshot)), { url: "about:blank", max: 12, mode: "forms" }); + assert.ok(h.cdpMethods().filter((m) => m === "Runtime.evaluate").length >= 2, "file and func both ran through Runtime.evaluate"); + + const clicked = await h.worker.resolveTargetInTab(5, { x: 10, y: 20 }); + assert.equal(clicked.found, true, "input target resolution also survives denied scripting"); +}); + +test("non-permission scripting failures are not masked by the CDP fallback", async () => { + const h = harness(); + h.chrome.scripting.executeScript = async () => { throw new Error("Frame with ID 0 was removed."); }; + await assert.rejects(h.worker.executeScriptWithFallback({ target: { tabId: 5 }, func: () => 1 }, "x"), /Frame with ID 0 was removed/); + assert.equal(h.calls.length, 0); + assert.equal(h.worker.isScriptingAccessDenied(new Error(ABOUT_BLANK_DENIED)), true); +}); + +test("CDP fallback reports page exceptions with the call-site label", async () => { + const h = harness(); + await assert.rejects( + h.worker.executeScriptWithFallback({ target: { tabId: 5 }, func: () => { throw new Error("boom in page"); }, args: [] }, "probe page"), + /probe page: .*boom in page/, + ); +}); + +// ---- Pi-side formatting for chrome_cdp results ---- +function section(start, end) { + const from = indexSource.indexOf(start); + const to = indexSource.indexOf(end, from); + assert.ok(from >= 0 && to > from, `Missing source section: ${start}`); + return indexSource.slice(from, to); +} +const formatCdpResult = vm.runInNewContext(`${stripTypeScriptTypes(section("const CDP_OVERSIZE_JSON_CHARS", "\nfunction extensionRoot"))}; formatCdpResult`, { + safeJson: (v) => JSON.stringify(v, null, 2), + truncateText: (s) => (s.length > 30_000 ? `${s.slice(0, 30_000)}[truncated]` : s), +}); + +test("chrome_cdp never returns screenshot/binary payloads or oversized details verbatim", () => { + const data = "A".repeat(4000) + "=="; + const shot = formatCdpResult("Page.captureScreenshot", { data }); + assert.ok(!shot.content[0].text.includes("AAAA")); + assert.match(shot.content[0].text, /~2999 bytes/); + assert.equal(shot.details.value.omitted, "data-field"); + + const huge = formatCdpResult("DOM.getOuterHTML", { outerHTML: "x".repeat(300_000) }); + assert.equal(huge.details.value.omitted, "oversized-result"); + assert.ok(huge.content[0].text.length < 31_000); + + const small = formatCdpResult("Runtime.evaluate", { result: { type: "number", value: 2 } }); + assert.deepEqual(small.details.value, { result: { type: "number", value: 2 } }); +}); diff --git a/test-suite/unit/type-evidence.test.mjs b/test-suite/unit/type-evidence.test.mjs new file mode 100644 index 0000000..f0a51c1 --- /dev/null +++ b/test-suite/unit/type-evidence.test.mjs @@ -0,0 +1,208 @@ +// chrome_type before/after evidence, replace=true, navigation-aware includeSnapshot, and tab.new load +// settling. Chrome and the page are mocked; live caret behavior is covered by challenge 44. +import assert from "node:assert/strict"; +import fs from "node:fs"; +import vm from "node:vm"; +import { stripTypeScriptTypes } from "node:module"; +import { test } from "node:test"; + +const workerSource = fs.readFileSync(new URL("../../extensions/chrome-profile-bridge/browser-extension/service_worker.js", import.meta.url), "utf8"); +const indexSource = fs.readFileSync(new URL("../../extensions/chrome-profile-bridge/index.ts", import.meta.url), "utf8"); +const clone = (value) => JSON.parse(JSON.stringify(value)); + +function makeInput({ value = "", caret = value.length, type = "text", name = "q" } = {}) { + return { + tagName: "INPUT", type, name, id: name, isConnected: true, isContentEditable: false, + value, selectionStart: caret, selectionEnd: caret, + getAttribute(attr) { return attr === "type" ? this.type : attr === "name" ? this.name : null; }, + scrollIntoView() {}, getBoundingClientRect: () => ({ left: 10, top: 10, width: 300, height: 30 }), + focus() {}, + }; +} + +function harness(element, { platform = "MacIntel", tabs = {} } = {}) { + const calls = []; + const listeners = new Set(); + const page = vm.createContext({ + document: { activeElement: element, querySelector: (s) => (s === "#q" ? element : null), createRange: () => ({}) }, + getSelection: () => null, + location: { href: "https://fixture.test/" }, + __PI_CHROME_STATE__: { elements: { "el-1": element } }, + }); + page.window = page; + const listener = { addListener() {}, removeListener() {} }; + const chrome = { + runtime: { id: "test", getManifest: () => ({ version: "0.0.1" }), onInstalled: listener, onStartup: listener }, + alarms: { create() {}, onAlarm: listener }, action: { onClicked: listener }, webNavigation: { onCommitted: listener }, + debugger: { onDetach: listener }, + scripting: { executeScript: async ({ func, args = [] }) => [{ result: await vm.runInContext(`(${func.toString()})`, page)(...args) }] }, + tabs: { + onUpdated: { addListener: (fn) => listeners.add(fn), removeListener: (fn) => listeners.delete(fn) }, + get: async (id) => clone(tabs[id] ?? { id, windowId: 1, url: "https://fixture.test/", status: "complete" }), + }, + }; + const worker = { chrome, console, AbortController, setTimeout, clearTimeout, setInterval: () => 0, navigator: { userAgent: "unit-test", platform }, fetch: async () => { throw new Error("offline"); } }; + worker.self = worker; + vm.runInNewContext(workerSource, worker); + worker.sleep = async () => {}; + worker.getTabByParams = async (params) => clone(tabs[Number(params.targetId ?? 2)] ?? { id: Number(params.targetId ?? 2), windowId: 1, url: "https://fixture.test/", status: "complete" }); + worker.bringToFront = async (tab) => tab; + worker.attachDebugger = async () => {}; + worker.cdpMoveTo = async () => {}; + const splice = (text) => { + if (typeof element.value !== "string") return; + const { value, selectionStart: s, selectionEnd: e } = element; + element.value = value.slice(0, s) + text + value.slice(e); + element.selectionStart = element.selectionEnd = s + text.length; + }; + worker.cdp = async (_tabId, method, params = {}) => { + calls.push({ method, params: clone(params) }); + if (method === "Input.dispatchKeyEvent") { + if ((params.commands || []).includes("selectAll")) { element.selectionStart = 0; element.selectionEnd = element.value.length; } + else if (params.type === "keyDown" && params.key === "Delete") { + if (element.selectionEnd > element.selectionStart) splice(""); + } else if (params.type === "keyDown" && params.text && params.key !== "Enter") splice(params.text); + } + return {}; + }; + return { worker, calls, element, listeners, fire: (tabId, info) => { for (const fn of [...listeners]) fn(tabId, info); } }; +} + +test("typing into the middle of a prefilled field reports the splice instead of a bare count", async () => { + const h = harness(makeInput({ value: "why do flamingos stand", caret: 7 })); + const result = await h.worker.dispatch("page.type", { targetId: "2", selector: "#q", text: "cats " }); + assert.equal(h.element.value, "why do cats flamingos stand"); + assert.equal(result.valueBefore, "why do flamingos stand"); + assert.equal(result.valueAfter, "why do cats flamingos stand"); + assert.equal(result.existingTextLengthBefore, 22); + assert.equal(result.insertedAt, "caret-middle"); + assert.equal(result.replaced, undefined); +}); + +test("empty and caret-end targets are classified without warnings", async () => { + const empty = harness(makeInput()); + assert.equal((await empty.worker.dispatch("page.type", { targetId: "2", uid: "el-1", text: "hi" })).insertedAt, "empty"); + const end = harness(makeInput({ value: "hello" })); + const result = await end.worker.dispatch("page.type", { targetId: "2", uid: "el-1", text: "!" }); + assert.equal(result.insertedAt, "caret-end"); + assert.equal(result.valueAfter, "hello!"); +}); + +for (const [platform, modifiers] of [["MacIntel", 4], ["Win32", 2]]) { + test(`replace=true selects all with the platform shortcut + selectAll command, then deletes (${platform})`, async () => { + const h = harness(makeInput({ value: "old query text", caret: 3 }), { platform }); + const result = await h.worker.dispatch("page.type", { targetId: "2", selector: "#q", text: "new", replace: true }); + assert.equal(h.element.value, "new"); + assert.equal(result.replaced, true); + assert.equal(result.insertedAt, "replaced-all"); + assert.equal(result.valueBefore, "old query text", "before is read prior to select-all/delete"); + const selectAll = h.calls.find((c) => (c.params.commands || []).includes("selectAll")); + assert.equal(selectAll.params.modifiers, modifiers); + }); +} + +test("sensitive fields never expose contents in type evidence", async () => { + const h = harness(makeInput({ value: "hunter2", type: "password", name: "password" })); + const result = await h.worker.dispatch("page.type", { targetId: "2", selector: "#q", text: "x" }); + assert.equal(result.valueRedacted, true); + assert.equal(result.valueBefore, undefined); + assert.equal(result.valueAfter, undefined); + assert.ok(!JSON.stringify(result).includes("hunter2")); + assert.equal(result.existingTextLengthBefore, 7); +}); + +test("a non-text focus target reports no value evidence rather than a false empty field", async () => { + const h = harness({ tagName: "DIV", isConnected: true, isContentEditable: false, scrollIntoView() {}, getBoundingClientRect: () => ({ left: 0, top: 0, width: 10, height: 10 }) }); + const result = await h.worker.dispatch("page.type", { targetId: "2", selector: "#q", text: "x" }); + assert.equal(result.valueBefore, undefined); + assert.equal(result.insertedAt, undefined); +}); + +test("includeSnapshot waits for a navigation the action started and snapshots the new page", async () => { + const tabs = { 2: { id: 2, windowId: 1, url: "https://fixture.test/", status: "complete" } }; + const h = harness(makeInput(), { tabs }); + h.worker.chromeInputKey = async () => { + tabs[2] = { ...tabs[2], url: "https://fixture.test/search?q=x", status: "loading" }; + setTimeout(() => { tabs[2].status = "complete"; h.fire(2, { status: "complete" }); }, 20); + return { input: "chrome", key: "Enter" }; + }; + h.worker.snapshotInTab = async (params) => ({ url: tabs[Number(params.targetId)].url, status: tabs[Number(params.targetId)].status }); + const out = await h.worker.dispatch("page.key", { targetId: "2", key: "Enter", includeSnapshot: true }); + assert.equal(out.navigation.from, "https://fixture.test/"); + assert.equal(out.navigation.to, "https://fixture.test/search?q=x"); + assert.equal(out.navigation.settled, true); + assert.deepEqual(clone(out.snapshot), { url: "https://fixture.test/search?q=x", status: "complete" }); + assert.equal(h.listeners.size, 0, "listener removed"); +}); + +test("a non-navigating action snapshots immediately; an already-loading page is not waited on", async () => { + const tabs = { 2: { id: 2, windowId: 1, url: "https://fixture.test/", status: "complete" } }; + const h = harness(makeInput(), { tabs }); + h.worker.chromeInputClick = async () => ({ input: "chrome" }); + h.worker.snapshotInTab = async () => ({ ok: true }); + const plain = await h.worker.dispatch("page.click", { targetId: "2", selector: "#q", includeSnapshot: true }); + assert.equal(plain.navigation, undefined); + + tabs[2].status = "loading"; // e.g. a hanging subresource, same URL + const started = Date.now(); + const busy = await h.worker.dispatch("page.click", { targetId: "2", selector: "#q", includeSnapshot: true }); + assert.ok(Date.now() - started < 1_000, "no bounded wait for a load the action did not start"); + assert.equal(busy.navigation.settled, false); + assert.equal(busy.navigation.waitedMs < 1_000, true); +}); + +test("tab.new reports the settled tab and loadStatus instead of the t=0 loading stub", async () => { + const tabs = {}; + const h = harness(makeInput(), { tabs }); + h.worker.findGroupRecordByTitle = async () => null; + h.worker.trackSessionTab = async () => {}; + h.worker.groupTab = async (tab) => ({ tab: { id: tab.id, url: tab.url, status: tab.status }, group: { title: "Pi Session: t" } }); + h.worker.formatTab = async (tab) => ({ id: tab.id, url: tab.url, status: tab.status, title: tab.title }); + h.worker.chrome.tabs.create = async ({ url }) => { + tabs[9] = { id: 9, windowId: 1, url: "", title: "", status: "loading" }; + setTimeout(() => { tabs[9] = { ...tabs[9], url, title: "Loaded", status: "complete" }; h.fire(9, { status: "complete" }); }, 20); + return clone(tabs[9]); + }; + const out = await h.worker.dispatch("tab.new", { url: "https://fixture.test/page", background: true }); + assert.equal(out.loadStatus, "complete"); + assert.deepEqual(clone(out.tab), { id: 9, url: "https://fixture.test/page", status: "complete", title: "Loaded" }); + assert.equal(out.group.title, "Pi Session: t"); +}); + +// ---- Pi-side text ---- +function section(start, end) { + const from = indexSource.indexOf(start); + const to = indexSource.indexOf(end, from); + assert.ok(from >= 0 && to > from, `Missing source section: ${start}`); + return indexSource.slice(from, to); +} +const pi = vm.runInNewContext(`${stripTypeScriptTypes(section("function formatIncludedSnapshotText(", "\n// Keep raw CDP"))}; ({ formatIncludedSnapshotText, describeTypeEvidence })`, { + formatChromeSnapshot: (s) => `SNAPSHOT ${s.url}`, +}); + +test("chrome_type text warns on splices, mentions spliced submits, and redacts secrets", () => { + const spliced = pi.describeTypeEvidence({ valueBefore: "abc", valueAfter: "aXbc", insertedAt: "caret-middle" }, 1, true).join("\n"); + assert.match(spliced, /Field went from "abc" to "aXbc"/); + assert.match(spliced, /spliced into the middle/); + assert.match(spliced, /submitted the spliced value/); + const appended = pi.describeTypeEvidence({ valueBefore: "abc", valueAfter: "abcX", insertedAt: "caret-end" }, 1, false).join("\n"); + assert.match(appended, /appended to existing content/); + assert.doesNotMatch(appended, /⚠/); + const secret = pi.describeTypeEvidence({ valueRedacted: true, existingTextLengthBefore: 7, valueLengthAfter: 8, insertedAt: "caret-end" }, 1, false).join("\n"); + assert.match(secret, /\[redacted\] \(7 → 8 chars/); + assert.equal(pi.describeTypeEvidence({ input: "chrome" }, 1, false).length, 0); + const lost = pi.describeTypeEvidence({ valueBefore: "abc", valueAfter: "abc", existingTextLengthBefore: 3, valueLengthAfter: 3, insertedAt: "caret-middle" }, 5, true).join("\n"); + assert.match(lost, /did not change/); + assert.doesNotMatch(lost, /spliced/, "an unchanged field is not reported as a splice"); + const lostSecret = pi.describeTypeEvidence({ valueRedacted: true, existingTextLengthBefore: 3, valueLengthAfter: 3, insertedAt: "caret-end" }, 2, false).join("\n"); + assert.match(lostSecret, /did not change/); +}); + +test("included snapshots state whether a navigation settled", () => { + const settled = pi.formatIncludedSnapshotText({ snapshot: { url: "https://b.test/" }, navigation: { from: "https://a.test/", to: "https://b.test/", settled: true, waitedMs: 120 } }, "Pressed Enter."); + assert.match(settled, /Navigated https:\/\/a\.test\/ → https:\/\/b\.test\/ \(waited 120ms/); + assert.match(settled, /SNAPSHOT https:\/\/b\.test\//); + const stale = pi.formatIncludedSnapshotText({ snapshot: { url: "https://a.test/" }, navigation: { from: "https://a.test/", to: "https://b.test/", settled: false, waitedMs: 5000 } }, "Pressed Enter."); + assert.match(stale, /⚠ The action started a navigation that had not finished after 5000ms/); + assert.equal(pi.formatIncludedSnapshotText({ result: {} }, "Clicked."), "Clicked."); +}); |