aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorTianren Dong <tianrendong.dtr@gmail.com>2026-05-14 22:46:31 -0400
committerTianren Dong <tianrendong.dtr@gmail.com>2026-05-14 22:46:31 -0400
commit58e86b70a0c7f6bf8e44f8b658e8194444ade943 (patch)
treeb56f638493c4967544b39c465596dddac1d1b4a9
parent1ec189f8f7f137555e8ae90d6753ee226eefa457 (diff)
Audit pi-chrome docs
-rw-r--r--CHANGELOG.md5
-rw-r--r--CONTRIBUTING.md6
-rw-r--r--README.md24
-rw-r--r--docs/COMPARISON.md6
-rw-r--r--docs/EXAMPLES.md8
-rw-r--r--docs/FAQ.md21
-rw-r--r--extensions/chrome-profile-bridge/browser-extension/manifest.json2
-rw-r--r--extensions/chrome-profile-bridge/browser-extension/service_worker.js12
-rw-r--r--extensions/chrome-profile-bridge/index.ts11
-rw-r--r--package.json4
10 files changed, 45 insertions, 54 deletions
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 719b171..4c60542 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -2,6 +2,11 @@
All notable user-facing changes to `pi-chrome`.
+## 0.15.17 — 2026-05-14
+
+- **Docs accuracy pass.** Updated README, FAQ, comparison, contributing notes, and package metadata for the current real-input-only, terminal-authorized tool surface.
+- **Input verification fix.** `includeSnapshot=true` now works for `chrome_click`, `chrome_type`, `chrome_fill`, and `chrome_key`, returning the Chrome-input result plus a fresh snapshot.
+
## 0.15.16 — 2026-05-14
- **Visible `/chrome` loading state.** Bare `/chrome` and `/chrome status` now immediately say “Checking Chrome connection…” before probing the companion extension, so a slow Chrome bridge no longer looks like the command did nothing.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index c16b5fe..3b8ff0f 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -5,8 +5,8 @@ Thanks for considering a contribution. pi-chrome aims to be the **de-facto brows
## Non-negotiables
1. **No re-login.** Every change must keep working against the user's already-signed-in Chrome profile. Anything that requires a fresh profile or extra auth steps is out of scope.
-2. **Honest result envelopes.** Every action tool returns `pageMutated`, `defaultPrevented`, `elementVisible`, `occludedBy` (when relevant), `valueMatches` (for input). Agents need to know **why** something didn't take effect.
-3. **Quiet by default, trusted by opt-in.** Synthetic DOM events first. CDP/`chrome.debugger` only when explicitly requested (`trusted: true`) or when the smart-auto heuristic detects an obvious user-activation gate.
+2. **Verifiable action results.** Input tools must return structured details and support `includeSnapshot` where verification matters. Agents need enough evidence to avoid blind retries.
+3. **Chrome real input.** Interactive controls use Chrome's input layer through `chrome.debugger`; do not re-expose synthetic/untrusted input as public UX.
4. **Benchmarks gate features.** Add a page in `test-suite/` that fails before your change and passes after. We accept PRs faster when there's a green/red verdict to point at.
## Local dev
@@ -25,7 +25,7 @@ python3 -m http.server 8765
1. Register in `extensions/chrome-profile-bridge/index.ts` (the `register*Tool` calls near line 840+).
2. Implement the handler in `extensions/chrome-profile-bridge/browser-extension/service_worker.js`.
-3. Return a `pageMutated` + relevant fields.
+3. Return structured details and support `includeSnapshot` for user-visible state changes when relevant.
4. Add a benchmark page under `test-suite/challenges/` and a manifest entry.
5. Update `README.md` "What an agent gets" table.
6. Add a `CHANGELOG.md` entry.
diff --git a/README.md b/README.md
index 948b77e..1210b71 100644
--- a/README.md
+++ b/README.md
@@ -1,7 +1,7 @@
# pi-chrome
> **The fastest way to give a [Pi](https://pi.dev) agent your real Chrome.**
-> No CDP. No throwaway profile. No re-login. Watch it work — or run silent.
+> No remote-debug port. No throwaway profile. No re-login. Watch it work — or run silent.
**MIT · 0 runtime deps · loopback-only bridge (`127.0.0.1:17318`) · inspect [`extensions/chrome-profile-bridge/browser-extension/`](./extensions/chrome-profile-bridge/browser-extension) before loading.** Verify connectivity in one command: `/chrome doctor`.
@@ -12,7 +12,7 @@ Agent: chrome_tab(list) → chrome_snapshot(uid:…) → chrome_screenshot(...)
You: [keeps coding — agent never asked you to log in]
```
-`pi-chrome` ships **20+ browser tools** for Pi agents, backed by a small MIT-licensed Chrome extension that runs inside the Chrome profile **you already use** — including every site you're already signed into.
+`pi-chrome` ships **19 browser tools** for Pi agents, backed by a small MIT-licensed Chrome extension that runs inside the Chrome profile **you already use** — including every site you're already signed into.
---
@@ -30,7 +30,7 @@ Then in Pi:
On macOS this opens `chrome://extensions`, reveals the bundled `browser-extension/` folder in Finder, and copies its path to your clipboard. In Chrome: **Developer mode** → **Load unpacked** → paste the path. Done.
-Verify, then authorize current Pi session in Pi:
+Verify, then authorize current Pi session from the terminal:
```text
/chrome doctor
@@ -120,27 +120,23 @@ You: [files the ticket with the folder attached]
---
-## Honest results
+## Verifiable actions
-Most browser-automation libraries return `void` or a generic ack. `pi-chrome` returns a structured envelope on every interaction:
+Input tools return structured details such as the coordinates used, target tag, uploaded paths, key pressed, or scroll distance. For click/type/fill/key calls, pass `includeSnapshot: true` to get a fresh page snapshot in the same result:
```text
-chrome_click(occluded-button) →
- "Clicked el-3 — pageMutated=false; occluded by <div#overlay>"
+chrome_click(uid:"el-3", includeSnapshot:true) →
+ result: { input:"chrome", x:412, y:238, tag:"BUTTON" }
+ snapshot: { title, url, text, elements:[...] }
```
-```text
-chrome_type(react-input, "hello") →
- "Typed into el-7 — valueMatches=true; pageMutated=true"
-```
-
-This is why agents using pi-chrome don't get stuck in retry loops on broken sites. They get the **reason** the action didn't land and can fix course in one turn.
+Agents can verify page state immediately instead of blindly retrying.
---
## What an agent gets
-**20 tools**, grouped by job. Every one runs against your already-open tabs.
+**19 tools**, grouped by job. Every one runs against your already-open tabs.
| Category | Tools |
| --------------- | ---------------------------------------------------------------------------------------------- |
diff --git a/docs/COMPARISON.md b/docs/COMPARISON.md
index d30cb6a..7dfe5ed 100644
--- a/docs/COMPARISON.md
+++ b/docs/COMPARISON.md
@@ -52,7 +52,7 @@ We benchmark in public — see [`../test-suite/`](../test-suite). Where exact sc
1. **Profile attach, not driver launch.** Every other driver fights cookie persistence, login walls, MFA, and extension state. pi-chrome inherits all of it because it *is* your Chrome.
2. **Chrome input against your real profile.** Interactive tools use CDP input for reliability while still controlling the Chrome profile you already use.
3. **Extension bridge transport.** No `--remote-debugging-port`, no throwaway Chromium. Survives Chrome auto-updates. Works alongside your normal Chrome usage.
-4. **Honest result envelopes.** Every action returns `pageMutated`, `defaultPrevented`, `elementVisible`, `occludedBy`, `valueMatches`. Competitors return `void` or generic acks; agents loop blindly on broken clicks.
+4. **Structured action results.** Input tools return target coordinates/tags and can include a fresh snapshot (`includeSnapshot`) so agents can verify state instead of blindly retrying.
5. **Multi-session shared bridge.** Planner + worker + audit Pi sessions all drive the same Chrome concurrently.
6. **Stable element uids.** `chrome_snapshot` returns deterministic uids you can pass to subsequent actions — similar to BrowserGym's `bid`, but built into the snapshot tool itself.
@@ -101,9 +101,9 @@ These wrap a driver with an LLM loop. They are **higher-level than pi-chrome** a
`pi-chrome` exposes tools that any Pi agent can call. If you want to use it from outside Pi:
-1. The local bridge speaks HTTP JSON-RPC at `127.0.0.1:17318` (default). The API is internal but stable across patch versions.
+1. The local bridge speaks HTTP JSON over `127.0.0.1:17318` (default). The API is internal; use the Pi tool surface unless you are building an adapter.
2. Tool surface mirrors Playwright closely (click/type/navigate/snapshot/screenshot/evaluate/wait_for) so adapter code is short.
-3. Honest envelopes (`pageMutated`, `valueMatches`, `occludedBy`) let agent harnesses skip retry/heal logic.
+3. `includeSnapshot` on input tools lets agent harnesses verify state after actions.
If you want a first-class pi-chrome adapter for Browser Use / Stagehand / LangGraph, file an issue with your use case.
diff --git a/docs/EXAMPLES.md b/docs/EXAMPLES.md
index a5cccf4..b3c6286 100644
--- a/docs/EXAMPLES.md
+++ b/docs/EXAMPLES.md
@@ -119,10 +119,8 @@ On my staging app:
### React controlled inputs
```text
-chrome_fill (not chrome_type) for React inputs — it uses the
-framework-aware native value setter so the form's state actually updates.
-After each fill, the result envelope's valueMatches=true confirms the
-component re-rendered with the new value.
+Use `chrome_fill` for React inputs when you want to replace the full value.
+Pass `includeSnapshot=true` to verify the component re-rendered with the new value.
```
### File upload without the native picker
@@ -160,7 +158,7 @@ Interactive tools use Chrome's real input layer by default: clicks, typing, fill
- sign-in flows
- guarded buttons
- audio/video controls
-- fullscreen / permission prompts
+- fullscreen and other user-activation checks
- pages with strict CSP or user-activation checks
Chrome may show its debugger banner while pi-chrome is attached.
diff --git a/docs/FAQ.md b/docs/FAQ.md
index 33902d3..f7f871f 100644
--- a/docs/FAQ.md
+++ b/docs/FAQ.md
@@ -32,9 +32,9 @@ Chrome control is also locked per Pi session until you run `/chrome authorize`;
Yes. The first session opens the local bridge; later sessions detect it and pipe their commands through the same bridge. Each Pi session must be authorized with `/chrome authorize` before its chrome_* tools work.
-## Why can't this be on the Chrome Web Store?
+## Why ship as an unpacked extension?
-Web Store extensions cannot communicate with a local process bridge controlled by another tool — Google's policy. pi-chrome must ship as an unpacked extension you load yourself. The upside: you can read the source. The downside: each Chrome update may prompt you to re-confirm.
+pi-chrome ships as an unpacked extension so the source and broad browser permissions are easy to inspect and update with the npm package. The downside: you load it manually from `chrome://extensions` and reload it after package updates.
## What happens when I update pi-chrome?
@@ -42,8 +42,8 @@ Web Store extensions cannot communicate with a local process bridge controlled b
## What's the install footprint?
-- Pi side: one extension that registers ~20 tools and a few slash commands.
-- Chrome side: one unpacked extension, ~5000 LOC of plain JavaScript, no dependencies.
+- Pi side: one extension that registers 19 tools and a few slash commands.
+- Chrome side: one unpacked extension, ~2000 LOC of plain JavaScript, no dependencies.
## Can I script it without Pi?
@@ -53,18 +53,11 @@ The Pi-facing tools are thin wrappers around an HTTP bridge at `127.0.0.1:17318`
Not always. `chrome_evaluate` and `chrome_snapshot` run in the page's MAIN world through the Function constructor, so pages whose CSP blocks `'unsafe-eval'` can reject them. `chrome_screenshot`, `chrome_navigate`, tab tools, and real Chrome input still work because they use extension/browser APIs rather than page JavaScript.
-## Why does my click return `pageMutated=false`?
+## How do I tell whether a click or type worked?
-Either:
-- The element was occluded (look for `occludedBy: <selector>` in the envelope).
-- The click handler called `event.preventDefault()` and the page intentionally ignored it.
-- The target changed after your snapshot; take a fresh snapshot or screenshot.
+Use `includeSnapshot=true` on `chrome_click`, `chrome_type`, `chrome_fill`, or `chrome_key`. The tool returns the Chrome-input result plus a fresh snapshot, so the agent can verify text, URL, visible elements, or form values before continuing.
-The result envelope tells you which one. **Don't blind-retry.**
-
-## Why does `chrome_type` return `valueMatches=false`?
-
-The field rejected or transformed the typed value. Common culprits: contenteditable rich-text editors, native date pickers, masked-input libraries, or masks. Try `chrome_fill`, then verify with `includeSnapshot=true`.
+If the page did not change, take a fresh snapshot or screenshot and check for overlays, disabled controls, stale element uids, or app-side validation.
## How do I attach a file to a React file input?
diff --git a/extensions/chrome-profile-bridge/browser-extension/manifest.json b/extensions/chrome-profile-bridge/browser-extension/manifest.json
index 66f396e..76d4305 100644
--- a/extensions/chrome-profile-bridge/browser-extension/manifest.json
+++ b/extensions/chrome-profile-bridge/browser-extension/manifest.json
@@ -1,7 +1,7 @@
{
"manifest_version": 3,
"name": "Pi Chrome Connector",
- "version": "0.15.16",
+ "version": "0.15.17",
"description": "Lets Pi control tabs in Chrome via a local connector at 127.0.0.1.",
"permissions": [
"tabs",
diff --git a/extensions/chrome-profile-bridge/browser-extension/service_worker.js b/extensions/chrome-profile-bridge/browser-extension/service_worker.js
index 08d712b..88391a7 100644
--- a/extensions/chrome-profile-bridge/browser-extension/service_worker.js
+++ b/extensions/chrome-profile-bridge/browser-extension/service_worker.js
@@ -712,7 +712,7 @@ async function dispatch(action, params) {
case "page.evaluate":
return evaluateInTab(params);
case "page.click":
- return chromeInputClick(params);
+ return withOptionalSnapshot(params, chromeInputClick);
case "page.hover":
return chromeInputHover(params);
case "page.drag":
@@ -720,11 +720,11 @@ async function dispatch(action, params) {
case "page.upload":
return chromeInputUpload(params);
case "page.type":
- return chromeInputType(params);
+ return withOptionalSnapshot(params, chromeInputType);
case "page.fill":
- return chromeInputFill(params);
+ return withOptionalSnapshot(params, chromeInputFill);
case "page.key":
- return chromeInputKey(params);
+ return withOptionalSnapshot(params, chromeInputKey);
case "page.scroll":
return chromeInputScroll(params);
case "page.tap":
@@ -932,8 +932,8 @@ async function evaluateInTab(params) {
return v;
}
-async function executeActionInTab(params, func, args) {
- const result = await executeInTab(params, func, args);
+async function withOptionalSnapshot(params, actionFn) {
+ const result = await actionFn(params);
if (params.includeSnapshot) {
const snapshot = await executeInTab({ ...params, foreground: false }, snapshotPage, [params.maxElements || 80, null, null, null]);
return { result, snapshot };
diff --git a/extensions/chrome-profile-bridge/index.ts b/extensions/chrome-profile-bridge/index.ts
index 6f2739a..5c72da6 100644
--- a/extensions/chrome-profile-bridge/index.ts
+++ b/extensions/chrome-profile-bridge/index.ts
@@ -8,9 +8,8 @@ import { dirname, join, resolve } from "node:path";
/**
* Existing-profile Chrome bridge for pi.
*
- * This is intentionally not a Chrome DevTools Protocol integration. CDP cannot attach to
- * already-running normal Chrome windows and recent Chrome builds block default-profile
- * remote debugging. Instead, install the companion Chrome extension from the
+ * This is intentionally not a remote-debugging-port integration. Chrome blocks default-profile
+ * remote debugging in many normal launches, so pi-chrome uses a companion extension from the
* browser-extension folder bundled next to this Pi extension.
*
* The companion extension runs inside the user's real Chrome profile and polls this local
@@ -496,18 +495,18 @@ export default function (pi: ExtensionAPI): void {
pi.on("before_agent_start", (event) => {
const primer = `
<chrome-profile-bridge>
-Chrome control is available through the chrome_* tools via a companion Chrome extension installed in the user's normal Chrome profile. Tools target the existing signed-in profile, no CDP, no throwaway profile.
+Chrome control is available through the chrome_* tools via a companion Chrome extension installed in the user's normal Chrome profile. Tools target the existing signed-in profile: no remote-debug port, no throwaway profile.
Capability model (important):
- Interactive controls (click/type/fill/key/hover/drag/scroll/tap) use Chrome's real input layer via chrome.debugger / CDP. Events satisfy normal user-activation gates.
- Input bypasses page CSP because it is injected at browser input layer, not page JavaScript. Chrome may show the “Pi Chrome Connector started debugging this browser” banner while attached.
- \`chrome_evaluate\` and \`chrome_snapshot\` run in MAIN world via the **Function constructor**, which requires \`'unsafe-eval'\` in the page CSP. Pages with strict CSP (e.g. github.com, many bank/SaaS apps) will throw \`EvalError: ... 'unsafe-eval' is not an allowed source of script\` and chrome_snapshot will return empty. On those pages, drive the page with \`chrome_screenshot\` + viewport-coordinate \`chrome_click\`/\`chrome_type\`/\`chrome_key\`. \`chrome_navigate\`, \`chrome_screenshot\`, \`chrome_tab\`, and Chrome input all keep working under any CSP.
-- Tool results include \`pageMutated\`, \`defaultPrevented\`, \`elementVisible\`, \`occludedBy\`, and (for type/fill) \`valueMatches\`. If an action result indicates no page change or occlusion, inspect current page state instead of repeating blindly.
+- Input tools return structured details and support \`includeSnapshot=true\` on click/type/fill/key. Use the fresh snapshot to verify state instead of repeating blindly.
Usage rules:
1. If a chrome_* tool says Chrome control is locked, ask the user to run \`/chrome authorize\` before retrying.
2. \`chrome_snapshot\` before clicking/typing; pass \`uid\` over \`selector\`.
-3. \`includeSnapshot=true\` on click/type/fill to verify in one round trip.
+3. \`includeSnapshot=true\` on click/type/fill/key to verify in one round trip.
4. If \`chrome_evaluate\` returns null when you expected a value, the expression evaluated to null/undefined in the page; surface the value via \`JSON.stringify\` to confirm.
5. \`chrome_navigate\` supports an optional \`initScript\` that runs at document_start in MAIN world for the next navigation (good for seeding localStorage or stubbing Date.now).
6. By default chrome_* tools focus Chrome so the user can watch; pass \`background=true\` or run /chrome background on for session-wide background execution.
diff --git a/package.json b/package.json
index 594c9a5..faa04d6 100644
--- a/package.json
+++ b/package.json
@@ -1,11 +1,11 @@
{
"name": "pi-chrome",
- "version": "0.15.16",
+ "version": "0.15.17",
"scripts": {
"version": "node scripts/sync-manifest-version.js",
"prepublishOnly": "node scripts/sync-manifest-version.js"
},
- "description": "Give a Pi agent your real, signed-in Chrome. No CDP, no throwaway profile, no re-login. 20+ tools (click, type, navigate, screenshot, network capture, file upload, drag, touch) with honest result envelopes — and a built-in browser-control benchmark suite.",
+ "description": "Give a Pi agent your real, signed-in Chrome. No remote-debug port, no throwaway profile, no re-login. 19 tools for click, type, navigate, screenshot, network capture, file upload, drag, and touch.",
"keywords": [
"pi",
"pi-package",