diff options
| author | Tianren Dong <tianrendong.dtr@gmail.com> | 2026-05-13 08:05:16 -0400 |
|---|---|---|
| committer | Tianren Dong <tianrendong.dtr@gmail.com> | 2026-05-13 08:05:16 -0400 |
| commit | 150d92fce35caccd81856f99fc3a62a9731a1e47 (patch) | |
| tree | 0ed2e23f69b267bc71a77e1a489c95c701e0b2e7 | |
| parent | e88e8a3a7a2258c5e7152da192effd8c788ae677 (diff) | |
pi-chrome 0.15.0: README rewrite — top-3 recipes, sharper comparison table, section reorder
| -rw-r--r-- | CHANGELOG.md | 6 | ||||
| -rw-r--r-- | README.md | 230 | ||||
| -rw-r--r-- | extensions/chrome-profile-bridge/browser-extension/manifest.json | 2 | ||||
| -rw-r--r-- | package.json | 5 |
4 files changed, 137 insertions, 106 deletions
diff --git a/CHANGELOG.md b/CHANGELOG.md index d384e79..70f064f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,12 @@ All notable user-facing changes to `pi-chrome`. +## 0.15.0 — 2026-05-13 + +- **README rewrite — top-3 recipes as terminal mockups.** PR triage, Linear standup, and Bug-repro-with-evidence each get a copy-pasteable prompt → tool trace → result block modeled on the hero example. The other six recipes (form auto-fill, admin cross-check, visual diff, auth-only data pull, network forensics, file upload) collapsed into a `<details>` block so the section sells before it catalogs. +- **Comparison table rewritten.** Dropped the all-✅ "Works on strict-CSP pages" row (zero signal). New table leads with "Time from `pi install` → first useful action on your real account" (~60s vs. hours) and "Survives MFA / SSO without code" (✅ already logged in). Multi-session row reframed as the bolded "Multiple agents drive the same Chrome at once". Footnote ² rewritten to highlight mode-aware scoring + open invitation for competing tools to PR their scores. +- **Section reorder: sells before catalogs.** New flow: hero → 60-second install → 30-second try-this → killer recipes → comparison → honest results → tool catalog → click/watch modes (with Diagnostics folded in) → architecture → benchmark suite → security model & why unpacked (combined) → composes-with → roadmap → contributing → license. Hero blockquote now precedes shields badges so pi.dev no longer scrapes a broken-image badge as the description. Package `description` shortened to 255 chars so pi.dev hero stops truncating mid-word. `author` set to `"tianrendong (Earendil Inc.)"`. + ## 0.14.9 - Primer (agent system prompt) now teaches the **trusted-mode escape hatch** explicitly. Previously the bridge would hit a CSP-locked page (github.com, banks, many SaaS apps), `chrome_evaluate`/`chrome_snapshot` would throw `EvalError: 'unsafe-eval' is not an allowed source of script`, and the agent would conclude *"bridge can't drive this page"* and ask the user for a fallback. New primer makes three things self-discoverable: (1) `trusted: true` on click/type/key/fill/hover/drag/scroll dispatches through chrome.debugger / CDP and bypasses page CSP entirely, (2) the recipe for strict-CSP pages is `chrome_screenshot` + trusted input at viewport coordinates, (3) when synthetic input produces no `pageMutated` or you see a CSP/eval error, **escalate to `trusted: true` yourself instead of asking the user**. Also corrects the old claim that `chrome_evaluate` works without `'unsafe-eval'` (it does not — Function constructor is gated by `script-src`). @@ -1,12 +1,10 @@ # pi-chrome -[](https://www.npmjs.com/package/pi-chrome) -[](https://www.npmjs.com/package/pi-chrome) -[](./LICENSE) - > **The fastest way to give a [Pi](https://pi.dev) agent your real Chrome.** > No CDP. No throwaway profile. No re-login. Watch it work — or run silent. +**MIT · 0 runtime deps · loopback-only bridge (`127.0.0.1:17318`) · inspect [`extensions/chrome-profile-bridge/browser-extension/`](./extensions/chrome-profile-bridge/browser-extension) before loading.** Verify connectivity in one command: `/chrome doctor`. + ```text You: "Find my open GitHub PR tab, summarize review state, and screenshot the failing CI." Agent: chrome_tab(list) → chrome_snapshot(uid:…) → chrome_screenshot(...) @@ -14,46 +12,11 @@ Agent: chrome_tab(list) → chrome_snapshot(uid:…) → chrome_screenshot(...) You: [keeps coding — agent never asked you to log in] ``` -`pi-chrome` ships **20+ browser tools** for Pi agents, backed by a small MIT-licensed Chrome extension that runs inside the Chrome profile **you already use** — including every site you're already signed into. - ---- - -## Why pi-chrome vs. everything else - -> Short version: **pi-chrome is primitives — "Playwright for the Chrome you're already signed into."** Not an agent loop. Plug it under any agent framework (Browser Use, Stagehand, LangGraph) or call its tools directly from a Pi agent. See [docs/COMPARISON.md](./docs/COMPARISON.md) for the full three-axis landscape (drivers, agents, cloud providers). - -| | **pi-chrome** | Playwright / Puppeteer | CDP-based agents | Selenium / WebDriver | -| ------------------------------ | --------------------------------- | ----------------------------- | ----------------------------- | ----------------------------- | -| Uses your real signed-in Chrome | ✅ yes (extension in your profile) | ❌ throwaway profile | ⚠️ requires `--remote-debug` | ❌ throwaway profile | -| Re-login required | **Never** | Every run | Sometimes | Every run | -| Watch agent work, live | ✅ default; toggle quiet | ❌ headless or new window | ⚠️ debugger banner always | ❌ new window | -| Works on strict-CSP pages | ✅ `new Function` MAIN-world | ✅ | ✅ | ✅ | -| Real browser-trusted clicks | ✅ opt-in (`chrome clicks on`) | ✅ | ✅ | ✅ | -| Multi-session safe | ✅ shared local bridge | ❌ port collisions | ❌ | ❌ | -| Network/console capture | ✅ built-in | ✅ | ✅ | ⚠️ via extensions | -| Honest result envelopes¹ | ✅ | ⚠️ | ❌ | ❌ | -| Built-in benchmark suite² | ✅ 38 primitives + 4 long-horizon | n/a | n/a | n/a | - -¹ Every action returns `pageMutated`, `defaultPrevented`, `elementVisible`, `occludedBy`, and `valueMatches` so the agent knows when a click didn't take effect — instead of looping blindly. -² See [`test-suite/`](./test-suite) — 38 primitive challenges plus 4 hermetic BrowserGym-style tasks. Scoring is expected-outcome-by-mode (`synthetic` / `trusted` / `manual`), not raw PASS count. Pages grade any browser-control tool on trusted clicks, pointer humanization, keyboard fidelity, drag/drop, clipboard, Shadow DOM, iframes, file uploads, network capture, and fingerprint leaks. - ---- - -## What an agent gets - -**20 tools**, grouped by job. Every one runs against your already-open tabs. - -| Category | Tools | -| --------------- | ---------------------------------------------------------------------------------------------- | -| **Tabs** | `chrome_tab` (list/new/activate/close/version), `chrome_launch` | -| **Inspect** | `chrome_snapshot` (uids + selectors + text + viewport), `chrome_screenshot`, `chrome_evaluate` | -| **Navigate** | `chrome_navigate` (with optional `initScript` at `document_start`), `chrome_wait_for` | -| **Interact** | `chrome_click`, `chrome_type`, `chrome_fill`, `chrome_key`, `chrome_hover` | -| **Gesture** | `chrome_drag` (HTML5 DataTransfer), `chrome_scroll` (wheel + momentum), `chrome_tap` (touch) | -| **Files** | `chrome_upload_file` (no native picker; works with React/Vue/Angular file inputs) | -| **Observe** | `chrome_list_console_messages`, `chrome_list_network_requests`, `chrome_get_network_request` (with response body) | +[](https://www.npmjs.com/package/pi-chrome) +[](https://www.npmjs.com/package/pi-chrome) +[](./LICENSE) -Each tool is documented inline in Pi — agents see the parameters and the gotchas (synthetic vs. trusted, autoplay gates, file picker limits) without trial-and-error. +`pi-chrome` ships **20+ browser tools** for Pi agents, backed by a small MIT-licensed Chrome extension that runs inside the Chrome profile **you already use** — including every site you're already signed into. --- @@ -101,16 +64,43 @@ You'll watch the agent jump to your GitHub tab and read the page — using **you ## Killer recipes (copy-paste into Pi) -Each one assumes tabs you already have open + accounts you're already signed into. +Each recipe assumes the relevant tab is already open in the Chrome you control. **PR triage** -> Use chrome_tab list to find my GitHub notifications tab, snapshot it, summarize PRs needing my review. + +```text +You: "Use chrome_tab list to find my GitHub notifications tab, then summarize PRs needing my review today, sorted by staleness." +Agent: chrome_tab(list) → chrome_snapshot(uid:el-notifications) → chrome_evaluate(...) + ✓ 7 PRs waiting on you. 2 stale >3d (storage-rewrite, billing-v2). + 1 just turned CI-green (api-keys-prune). Full sorted list below. +You: [pastes the list straight into Linear] +``` **Linear standup** -> Open my Linear current cycle in the active tab, snapshot it, write a 5-bullet standup. + +```text +You: "Open my Linear current cycle in the active tab and write a 5-bullet standup." +Agent: chrome_tab(activate, urlIncludes:"linear.app") → chrome_snapshot(uid:el-cycle) → chrome_evaluate(...) + ✓ 5 in-progress, 2 blocked. Standup draft: + • Shipped: bridge auto-recover. • In flight: trusted-mode retry path. + • Blocked: vendor portal CSP (waiting on infra). + • Next: benchmark v2. • Risk: none today. +You: [drops it into #standup] +``` **Bug repro with evidence** -> Open the staging app I'm already signed into, reproduce \<bug>, save a screenshot of each step under `./repro/`. + +```text +You: "Repro the checkout 500 on staging, save a screenshot at each step under ./repro/." +Agent: chrome_navigate(staging) → chrome_click(uid:el-add-to-cart) → chrome_screenshot(./repro/01-cart.png) + → chrome_click(uid:el-checkout) → chrome_list_network_requests() → chrome_screenshot(./repro/02-500.png) + ✓ POST /api/checkout → 500. Response body saved → ./repro/checkout-500.json + 3 screenshots in ./repro/. Trigger: missing tax_id when cart contains digital goods. +You: [files the ticket with the folder attached] +``` + +<details> +<summary><strong>More recipes</strong> (form auto-fill, admin cross-check, visual diff, auth-only data pull, network forensics, file upload)</summary> **Form auto-fill (no submit)** > Open the vendor portal, fill the new-vendor form from this JSON, stop before submit. @@ -130,23 +120,65 @@ Each one assumes tabs you already have open + accounts you're already signed int **File upload through React** > Open the photo uploader, `chrome_upload_file` with `./fixtures/sample.png`, confirm preview rendered. +</details> + --- -## Architecture +## Why pi-chrome vs. Playwright / CDP / Selenium + +> Short version: **pi-chrome is primitives — "Playwright for the Chrome you're already signed into."** Not an agent loop. Plug it under any agent framework (Browser Use, Stagehand, LangGraph) or call its tools directly from a Pi agent. See [docs/COMPARISON.md](./docs/COMPARISON.md) for the full three-axis landscape (drivers, agents, cloud providers). + +| | **pi-chrome** | Playwright / Puppeteer | CDP-based agents | Selenium / WebDriver | +| ------------------------------ | --------------------------------- | ----------------------------- | ----------------------------- | ----------------------------- | +| **Time from `pi install` → first useful action on your real account** | ~60s (load unpacked, `/chrome doctor`) | hours (script login, store creds, debug headless) | 30+ min (`--remote-debug` setup, attach) | hours (driver + login script) | +| **Survives MFA / SSO without code** | ✅ already logged in | ❌ | ⚠️ if you re-auth | ❌ | +| Uses your real signed-in Chrome | ✅ extension in your profile | ❌ throwaway profile | ⚠️ requires `--remote-debug` | ❌ throwaway profile | +| Re-login required | **Never** | Every run | Sometimes | Every run | +| **Multiple agents drive the same Chrome at once** | ✅ shared bridge | ❌ port collisions | ❌ | ❌ | +| Watch agent work, live | ✅ default; toggle quiet | ❌ headless or new window | ⚠️ debugger banner always | ❌ new window | +| Real browser-trusted clicks | ✅ opt-in (`chrome clicks on`) | ✅ | ✅ | ✅ | +| Network/console capture | ✅ built-in | ✅ | ✅ | ⚠️ via extensions | +| **Honest result envelopes¹** | ✅ | ⚠️ | ❌ | ❌ | +| Self-graded by built-in benchmark² | ✅ 38 primitives + 4 long-horizon | n/a | n/a | n/a | + +¹ Every action returns `pageMutated`, `defaultPrevented`, `elementVisible`, `occludedBy`, and `valueMatches` so the agent knows when a click didn't take effect — instead of looping blindly. +² [`test-suite/`](./test-suite) is mode-aware: a synthetic-events tool is *expected* to fail clipboard. If you build a competing tool, send a PR with your scores. We benchmark in public. + +--- + +## Honest results +Most browser-automation libraries return `void` or a generic ack. `pi-chrome` returns a structured envelope on every interaction: + +```text +chrome_click(occluded-button) → + "Clicked el-3 — pageMutated=false; occluded by <div#overlay>" ``` - ┌──────────────────────┐ ┌──────────────────────────┐ - │ Pi agent (terminal) │ ─── http://127.0.0.1:17318 ─→ │ Chrome extension │ - │ chrome_* tools │ │ (your real profile) │ - └──────────┬───────────┘ └─────────┬────────────────┘ - │ same machine │ - ▼ ▼ - Other Pi sessions Tabs you already have open - share the same bridge (signed in to GitHub, - automatically Linear, Stripe, etc.) + +```text +chrome_type(react-input, "hello") → + "Typed into el-7 — valueMatches=true; pageMutated=true" ``` -Multiple Pi sessions (planner / worker / audit) can all drive the same Chrome at once. The first session opens the local bridge; later sessions detect it and pipe their commands through. +This is why agents using pi-chrome don't get stuck in retry loops on broken sites. They get the **reason** the action didn't land and can fix course in one turn. + +--- + +## What an agent gets + +**20 tools**, grouped by job. Every one runs against your already-open tabs. + +| Category | Tools | +| --------------- | ---------------------------------------------------------------------------------------------- | +| **Tabs** | `chrome_tab` (list/new/activate/close/version), `chrome_launch` | +| **Inspect** | `chrome_snapshot` (uids + selectors + text + viewport), `chrome_screenshot`, `chrome_evaluate` | +| **Navigate** | `chrome_navigate` (with optional `initScript` at `document_start`), `chrome_wait_for` | +| **Interact** | `chrome_click`, `chrome_type`, `chrome_fill`, `chrome_key`, `chrome_hover` | +| **Gesture** | `chrome_drag` (HTML5 DataTransfer), `chrome_scroll` (wheel + momentum), `chrome_tap` (touch) | +| **Files** | `chrome_upload_file` (no native picker; works with React/Vue/Angular file inputs) | +| **Observe** | `chrome_list_console_messages`, `chrome_list_network_requests`, `chrome_get_network_request` (with response body) | + +Each tool is documented inline in Pi — agents see the parameters and the gotchas (synthetic vs. trusted, autoplay gates, file picker limits) without trial-and-error. --- @@ -166,9 +198,7 @@ Multiple Pi sessions (planner / worker / audit) can all drive the same Chrome at Per-call `trusted: true / false` on any input tool wins over the global mode. ---- - -## Background / watch modes +### Background / watch modes By default, every `chrome_*` call focuses Chrome and activates the target tab so you can **watch the agent work** — invaluable for demos, debugging, and first-time confidence. @@ -180,51 +210,56 @@ By default, every `chrome_*` call focuses Chrome and activates the target tab so Per-call `background: true` wins over the session toggle. ---- +### Diagnostics -## Honest results +- `/chrome doctor` — single command: connectivity, extension version, bridge owner, version drift, MAIN-world helper injection, `chrome_evaluate("1+1") === 2`, fingerprint flags. +- `/chrome onboard` — guided first-time setup. +- `/chrome quiet status`, `/chrome clicks status` — current modes. -Most browser-automation libraries return `void` or a generic ack. `pi-chrome` returns a structured envelope on every interaction: +If the loaded Chrome extension is older than `pi-chrome` on disk, `/chrome doctor` tells you to reload it from `chrome://extensions`. -```text -chrome_click(occluded-button) → - "Clicked el-3 — pageMutated=false; occluded by <div#overlay>" -``` +--- -```text -chrome_type(react-input, "hello") → - "Typed into el-7 — valueMatches=true; pageMutated=true" +## Architecture + +``` + ┌──────────────────────┐ ┌──────────────────────────┐ + │ Pi agent (terminal) │ ─── http://127.0.0.1:17318 ─→ │ Chrome extension │ + │ chrome_* tools │ │ (your real profile) │ + └──────────┬───────────┘ └─────────┬────────────────┘ + │ same machine │ + ▼ ▼ + Other Pi sessions Tabs you already have open + share the same bridge (signed in to GitHub, + automatically Linear, Stripe, etc.) ``` -This is why agents using pi-chrome don't get stuck in retry loops on broken sites. They get the **reason** the action didn't land and can fix course in one turn. +Multiple Pi sessions (planner / worker / audit) can all drive the same Chrome at once. The first session opens the local bridge; later sessions detect it and pipe their commands through. --- -## Diagnostics - -- `/chrome doctor` — single command: connectivity, extension version, bridge owner, version drift, MAIN-world helper injection, `chrome_evaluate("1+1") === 2`, fingerprint flags. -- `/chrome onboard` — guided first-time setup. -- `/chrome quiet status`, `/chrome clicks status` — current modes. - -If the loaded Chrome extension is older than `pi-chrome` on disk, `/chrome doctor` tells you to reload it from `chrome://extensions`. +## Built-in benchmark suite ---- +[`test-suite/`](./test-suite) is a benchmark for **any** browser-control agent (not just pi-chrome). It includes **38 primitive challenges** plus **4 hermetic BrowserGym-style long-horizon tasks**. -## Composes with +Scoring is **expected-outcome-by-mode**, not raw PASS count: each challenge has an expected verdict per mode (`synthetic`, `trusted`, `manual`) and a tool grades itself by whether its actual outcome matches the expected one. This avoids false equivalence between modes — a synthetic-events tool isn't supposed to satisfy a clipboard user-activation gate; matching that expectation is the pass. -- **[pi-qq](https://www.npmjs.com/package/pi-qq)** — `/qq summarize what the active GitHub tab shows` without polluting the main transcript. -- **[pi-bar](https://www.npmjs.com/package/pi-bar)** — when the agent scrapes large pages, watch the context-usage segment turn yellow → red as a signal to `/qq` for a recap. -- **PR demo skills** — screenshots write to `.pi/chrome-screenshots/` so you can attach them to PR descriptions or demo bundles. +Each challenge exposes `window.__verdict` / `window.__reason` / `window.__events` and a manifest entry with expected results per mode. ---- +```bash +cd test-suite && python3 -m http.server 8765 +# open http://127.0.0.1:8765/ in the Chrome window pi-chrome controls +``` -## Why an unpacked Chrome extension? +Categories: `trusted-input`, `pointer-humanization`, `keyboard`, `activation-gates`, `scroll`, `drag-drop`, `clipboard`, `native-controls`, `frameworks`, `editing`, `dom-complexity`, `frames`, `files`, `observability`, `fingerprint`, `agent-safety`. -`pi-chrome` cannot ship through the Chrome Web Store — a Web Store extension cannot talk to a local bridge controlled by another tool on the same machine. So it ships as a small MIT-licensed unpacked extension in [`extensions/chrome-profile-bridge/browser-extension/`](./extensions/chrome-profile-bridge/browser-extension). **Read the source before loading.** `/chrome doctor` reports the extension version and warns when it drifts from your installed `pi-chrome`. +If you build a competing tool, please open a PR with your scores. We benchmark in public. --- -## Security model +## Security model & why unpacked + +**Unpacked on purpose.** A Web Store extension cannot talk to a local bridge controlled by another tool on the same machine — so pi-chrome ships its bridge as an inspectable, MIT-licensed folder you load once with Developer Mode. Every line is yours to read in [`extensions/chrome-profile-bridge/browser-extension/`](./extensions/chrome-profile-bridge/browser-extension). `/chrome doctor` reports the loaded extension version and warns when it drifts from your installed `pi-chrome`. The companion extension runs in the Chrome profile where you install it and has broad tab/scripting permissions. Only install it from a package source you trust. @@ -238,22 +273,11 @@ There is no network exposure; the bridge binds to loopback only. --- -## Built-in benchmark suite - -[`test-suite/`](./test-suite) is a benchmark for **any** browser-control agent (not just pi-chrome). It includes **38 primitive challenges** plus **4 hermetic BrowserGym-style long-horizon tasks**. - -Scoring is **expected-outcome-by-mode**, not raw PASS count: each challenge has an expected verdict per mode (`synthetic`, `trusted`, `manual`) and a tool grades itself by whether its actual outcome matches the expected one. This avoids false equivalence between modes — a synthetic-events tool isn't supposed to satisfy a clipboard user-activation gate; matching that expectation is the pass. - -Each challenge exposes `window.__verdict` / `window.__reason` / `window.__events` and a manifest entry with expected results per mode. - -```bash -cd test-suite && python3 -m http.server 8765 -# open http://127.0.0.1:8765/ in the Chrome window pi-chrome controls -``` - -Categories: `trusted-input`, `pointer-humanization`, `keyboard`, `activation-gates`, `scroll`, `drag-drop`, `clipboard`, `native-controls`, `frameworks`, `editing`, `dom-complexity`, `frames`, `files`, `observability`, `fingerprint`, `agent-safety`. +## Composes with -If you build a competing tool, please open a PR with your scores. We benchmark in public. +- **[pi-qq](https://www.npmjs.com/package/pi-qq)** — `/qq summarize what the active GitHub tab shows` without polluting the main transcript. +- **[pi-bar](https://www.npmjs.com/package/pi-bar)** — when the agent scrapes large pages, watch the context-usage segment turn yellow → red as a signal to `/qq` for a recap. +- **PR demo skills** — screenshots write to `.pi/chrome-screenshots/` so you can attach them to PR descriptions or demo bundles. --- diff --git a/extensions/chrome-profile-bridge/browser-extension/manifest.json b/extensions/chrome-profile-bridge/browser-extension/manifest.json index 58525e4..53f5418 100644 --- a/extensions/chrome-profile-bridge/browser-extension/manifest.json +++ b/extensions/chrome-profile-bridge/browser-extension/manifest.json @@ -1,7 +1,7 @@ { "manifest_version": 3, "name": "Pi Chrome Connector", - "version": "0.14.9", + "version": "0.15.0", "description": "Lets Pi control tabs in Chrome via a local connector at 127.0.0.1.", "permissions": [ "tabs", diff --git a/package.json b/package.json index d1f861f..b2d42e0 100644 --- a/package.json +++ b/package.json @@ -1,11 +1,11 @@ { "name": "pi-chrome", - "version": "0.14.9", + "version": "0.15.0", "scripts": { "version": "node scripts/sync-manifest-version.js", "prepublishOnly": "node scripts/sync-manifest-version.js" }, - "description": "The de-facto browser automation toolkit for Pi agents. Drive your existing logged-in Chrome — no re-login, no throwaway profile, no CDP. 20+ tools (click, type, navigate, screenshot, network capture, file upload, drag, scroll, touch) + honest result envelopes + a built-in benchmark suite.", + "description": "Give a Pi agent your real, signed-in Chrome. No CDP, no throwaway profile, no re-login. 20+ tools (click, type, navigate, screenshot, network capture, file upload, drag, touch) with honest result envelopes — and a built-in browser-control benchmark suite.", "keywords": [ "pi", "pi-package", @@ -36,6 +36,7 @@ "stagehand-alternative" ], "license": "MIT", + "author": "tianrendong (Earendil Inc.)", "homepage": "https://github.com/tianrendong/pi-chrome#readme", "repository": { "type": "git", |
