diff options
| author | Tianren Dong <tianrendong.dtr@gmail.com> | 2026-09-10 10:19:54 -0400 |
|---|---|---|
| committer | Tianren Dong <tianrendong.dtr@gmail.com> | 2026-09-10 10:19:54 -0400 |
| commit | 502bcc818572bf2f9df906e472ad00ab60dd977e (patch) | |
| tree | 467d20b860e5673aea049081ade1c0a2e0e1e578 /README.md | |
| parent | 304c40a7d42b3c8ad8e93627012c53b086c72065 (diff) | |
docs: improve pi-chrome README and onboarding
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 177 |
1 files changed, 83 insertions, 94 deletions
@@ -1,156 +1,145 @@ # pi-chrome -> Let [Pi](https://pi.dev) use your existing signed-in Chrome profile after explicit authorization. +**Give Pi the Chrome you're already signed into.** -**MIT · 0 runtime deps · loopback-only bridge (`127.0.0.1:17318`) · inspectable unpacked Chrome extension.** Review [`extensions/chrome-profile-bridge/browser-extension/`](./extensions/chrome-profile-bridge/browser-extension) before loading. Verify setup with `/chrome doctor`. +Debug your app, inspect signed-in dashboards, and capture screenshots using your existing Chrome profile—without setting up a separate automation browser. -```text -You: "Find my open GitHub PR tab, summarize review state, and screenshot failing CI." -Agent: chrome_tab(list) → chrome_snapshot(uid:…) → chrome_screenshot(...) - ✓ 3 reviewers, 1 change requested, CI red on iOS. Saved → .pi/chrome-screenshots/ci.png -You: [keeps coding — agent never asked you to log in] -``` +Built for the [Pi coding agent](https://pi.dev). Pi's Chrome tools stay locked until you authorize the current session. -`pi-chrome` runs through a small Chrome extension inside the Chrome profile **you already use** — including sites where you're already signed in. Agents can inspect or control Chrome only after you run `/chrome authorize` in current Pi session. +[Quick start](#quick-start) · [Examples](./docs/EXAMPLES.md) · [Safety](#safety) · [Contributing](./CONTRIBUTING.md) ---- +[](https://www.npmjs.com/package/pi-chrome) +[](./LICENSE) -## Install +## What you can do -```bash -pi install npm:pi-chrome -``` +Try prompts like these after setup: -In Pi: +| Use case | Ask Pi | +| --- | --- | +| **Debug a signed-in app** | “Reproduce the filter bug in my staging app. Inspect captured console and network errors, then save a screenshot.” | +| **Understand an existing page** | “Find my open dashboard tab and summarize what's on the page. Don't change anything.” | +| **Create evidence for a PR** | “On my local app, capture the empty, loading, and populated states of this feature for my PR.” | -```text -/chrome onboard -``` +Pi gets tools to inspect pages, click, type, fill forms, scroll, upload files, capture screenshots, and inspect captured console logs and `fetch`/`XMLHttpRequest` responses. You describe the task; Pi handles the agent loop. -This opens `chrome://extensions` and copies bundled extension path. In Chrome Extensions: +**Best fit:** interactive workflows in the Chrome profile you already use. For deterministic CI tests, consider a test framework such as Playwright; for fleets of isolated browsers, consider a hosted browser service. See [more workflows](./docs/EXAMPLES.md) and [browser-tool comparisons](./docs/COMPARISON.md). -1. Turn on **Developer mode**. -2. Click **Load unpacked**. -3. Open path field with **Cmd+Shift+G** on macOS or **Ctrl+L** on Windows/Linux. -4. Paste copied path. -5. Press Enter. +## Quick start -Reload Pi so installed package loads: +**Requirements:** [Pi](https://pi.dev) and Google Chrome. Setup includes a one-time manual installation of the bundled Chrome companion extension. -```text -/reload -``` +> **Trust and privacy:** The companion has broad browser permissions and runs in your real Chrome profile. Review [its source](./extensions/chrome-profile-bridge/browser-extension/) before loading it, and authorize only tasks you trust. The browser bridge is local, but page content returned to Pi may be sent to your configured model provider. -Check bridge: +### 1. Install and load the Pi package -```text -/chrome doctor +In your terminal: + +```bash +pi install npm:pi-chrome ``` -You should see: +Start Pi with `pi`. If Pi is already running, run `/reload` in that session **before** using the `/chrome` commands. -```text -✓ Chrome is connected (...) -``` +### 2. Connect Chrome -Authorize current session: +In Pi: ```text -/chrome authorize -/chrome doctor +/chrome onboard ``` -Second doctor run should show all checks passing. +The setup dialog shows the companion extension's folder path. ---- +- **macOS:** after confirmation, Pi opens `chrome://extensions`, reveals the companion folder in Finder, and copies its path to your clipboard. +- **Windows/Linux:** copy the folder path shown in the dialog and open `chrome://extensions` manually. Automatic desktop opening and clipboard setup are currently macOS-only. -## What it can do +In Chrome: -- Read and summarize pages you're already signed into. -- Click, type, fill forms, scroll, drag, tap, and upload files. -- Capture screenshots for bugs, PRs, and demos. -- Inspect console logs and captured `fetch`/`XMLHttpRequest` responses. -- Manage tabs without taking over your active window. - -Tool parameters and gotchas are documented inline in Pi. - -### Typing into rich editors +1. Turn on **Developer mode**. +2. Click **Load unpacked**. +3. Select the companion folder shown by `/chrome onboard`. On macOS, press **Cmd+Shift+G** in the folder picker and paste the copied path. -`chrome_type` and `chrome_fill` use one native CDP `Input.insertText` operation for focused contenteditables. This avoids per-character delays for long text and preserves Unicode/newlines. Ordinary inputs and textareas retain individual key events. +### 3. Authorize and verify -For an editor that needs a `keydown` for every character, pass `perCharacter:true`. Bulk insertion still uses Chrome's input system, but does not emit per-character key events or a clipboard `paste` event. Use `includeSnapshot:true` to verify the result; `chrome_fill` still honors `domFallback:false` when synthetic fallback is unwanted. +In Pi: ---- +```text +/chrome authorize +/chrome doctor +``` -## Safety model +Approve the authorization prompt for a task you trust. The default authorization lasts **15 minutes**. Doctor should report `✓ Chrome is connected (...)`; follow its instructions if any checks fail. -Chrome control is locked by default. Authorize per Pi session: +Then try a read-only first task: ```text -/chrome authorize # 15 minutes -/chrome authorize 30m # custom duration -/chrome authorize indefinite -/chrome revoke # lock again -/chrome status +List my open Chrome tabs without navigating, clicking, or changing anything. ``` -Safety properties: +Run `/chrome revoke` when finished. Use `/chrome authorize` again whenever you want to grant access for another task or session. + +## Safety -- Extension runs in your real Chrome profile and has broad tab/scripting permissions. Install only from trusted package source. -- Pi side binds to `127.0.0.1:17318` only; no default network exposure. -- Bridge rejects browser-origin command requests, so ordinary web pages cannot drive it through CORS. -- Each Pi session gets its own automation target; user tabs/windows are not closed by cleanup. -- `/chrome revoke` closes only calling session's automation target. +- **Per-session approval.** Pi's Chrome tools require `/chrome authorize`. `/chrome revoke` locks them and requests cleanup of that session's owned automation tabs. Cleanup preserves existing user tabs. +- **Separate targets by default.** Page actions without an explicit target use a session-owned automation window or tab. The agent can deliberately target an existing tab when your task calls for it. +- **Local transport, not a sandbox.** The bridge binds to `127.0.0.1:17318` and rejects browser-origin command requests. It does not authenticate arbitrary non-browser local callers; it is not protection against hostile processes on your machine. +- **Background mode by default.** Tools cannot override it to explicitly focus Chrome or activate tabs. This is not a zero-focus guarantee: page scripts, trusted input, native prompts, and Chrome/OS behavior can still affect focus. Chrome may also show its debugger banner while attached. -Security details: [`SECURITY.md`](./SECURITY.md). Architecture details: [`docs/ARCHITECTURE.md`](./docs/ARCHITECTURE.md). +Read the [security policy](./SECURITY.md) and [background-mode scope and risks](./docs/ARCHITECTURE.md#scope-and-risks) before using it with sensitive accounts. ---- +### Limits + +This is browser automation, not full OS control. Native Chrome/OS dialogs, password-manager prompts, passkeys/security keys/biometrics, CAPTCHA challenges, cross-origin iframe DOM access, rich multitouch/stylus gestures, and arbitrary desktop apps are outside its reliable tool surface. Some workflows need human assistance. + +If page inspection or evaluation is blocked, use screenshots and coordinate input where possible. Background pages can throttle rendering or reject focus-gated actions. See the [FAQ](./docs/FAQ.md) for details. ## Commands ```text -/chrome onboard # guided setup -/chrome doctor # connectivity + version + eval checks -/chrome status # connection + auth + background state -/chrome authorize [duration] -/chrome revoke -/chrome background on # default: hard background policy -/chrome background off # foreground/watch mode +/chrome onboard # one-time companion setup +/chrome authorize # authorize this Pi session for 15 minutes +/chrome authorize 30m # choose a duration +/chrome authorize indefinite # no time limit; revoke when finished +/chrome revoke # lock tools and request session cleanup +/chrome doctor # connection, version, and page checks +/chrome status # connection, authorization, and background state +/chrome background on # default: block explicit focus/tab activation +/chrome background off # allow foreground/watch mode /chrome background status ``` -If loaded extension is older than installed `pi-chrome`, `/chrome doctor` tells you to reload it from `chrome://extensions`. +Tool parameters are documented inline in Pi. See [architecture](./docs/ARCHITECTURE.md) for target ownership, screenshot behavior, and background-policy details. -### Background policy +### Updating and troubleshooting -`/chrome background on` is enforced, not an overridable default. Per-call `background:false` cannot bring Chrome forward, new tabs stay inactive, and `chrome_tab activate` is blocked. Use the existing `/chrome background off` for foreground/watch mode; per-call `background:true` still works when that mode is off. +After `pi update npm:pi-chrome`, run `/reload` in Pi and reload **Pi Chrome Connector** in `chrome://extensions`. Run `/chrome doctor` to check the connection and companion version. -Screenshots use CDP without activating background tabs. Debugger/capture failures return errors, never an activation fallback. Reload both Pi and the Chrome companion after upgrading; old companions reject background tab creation/screenshots rather than silently switching tabs. +If Chrome is not responding, confirm the companion is enabled and keep Chrome open. If page checks fail on a Chrome internal page, try a regular web page and run Doctor again. Follow its version-mismatch or session-restart instructions when shown. -This prevents explicit pi-chrome focus/activation, not every Chrome/OS side effect. Trusted input, page popups, native prompts, debugger banners, and macOS Spaces can still affect focus. Inactive pages may throttle rendering or reject focus-gated actions. See [scope and risks](./docs/ARCHITECTURE.md#scope-and-risks). +### Typing into rich editors ---- +`chrome_type` and `chrome_fill` use one native CDP `Input.insertText` operation for focused contenteditables. This avoids per-character delays for long text and preserves Unicode/newlines. Ordinary inputs and textareas retain individual key events. -## Limits +For an editor that needs a `keydown` for every character, pass `perCharacter:true`. Bulk insertion still uses Chrome's input system, but does not emit per-character key events or a clipboard `paste` event. Use `includeSnapshot:true` to verify the result; `chrome_fill` still honors `domFallback:false` when synthetic fallback is unwanted. -`pi-chrome` works best on web-page workflows exposed through DOM, screenshots, tabs, network, console, and Chrome input. It is not full OS automation. +## Tests and documentation -Current limits include native Chrome/OS surfaces, print/save dialogs, permission bubbles, password-manager prompts, cross-origin iframe DOM access, CAPTCHA/bot challenges, passkeys/security keys/biometrics, rich multitouch/pinch/stylus gestures, and arbitrary desktop apps. +The [benchmark suite](./test-suite/README.md) includes **44 browser challenges**, hermetic multi-step tasks, and mocked regression tests for input, screenshots, session cleanup, and background policy. These are reproducible test cases—not a guarantee that every website or OS interaction works. -For strict-CSP pages, use screenshots + coordinate input when snapshot/evaluate paths are blocked. +Run mocked regressions with `npm test` (Node.js 22.13+; no live Chrome required). Follow the benchmark guide for live browser checks. ---- +- [Examples](./docs/EXAMPLES.md) — prompts and workflows to try. +- [FAQ](./docs/FAQ.md) — compatibility, setup questions, and limitations. +- [Architecture](./docs/ARCHITECTURE.md) — bridge, session lifecycle, and background policy. +- [Comparison](./docs/COMPARISON.md) — where pi-chrome fits among browser tools. +- [Changelog](./CHANGELOG.md) — release history. -## Docs +## Contributing -- Examples: [`docs/EXAMPLES.md`](./docs/EXAMPLES.md) -- FAQ: [`docs/FAQ.md`](./docs/FAQ.md) -- Comparison: [`docs/COMPARISON.md`](./docs/COMPARISON.md) -- Security: [`SECURITY.md`](./SECURITY.md) -- Benchmark suite: [`test-suite/README.md`](./test-suite/README.md) -- Architecture: [`docs/ARCHITECTURE.md`](./docs/ARCHITECTURE.md) +Bug reports, reproducible browser challenges, and real workflow demos are welcome. Include `/chrome doctor` output and reproduction steps; redact private URLs and page content. See [CONTRIBUTING.md](./CONTRIBUTING.md). ---- +If pi-chrome is useful, consider starring the repo or sharing a workflow that worked for you. ## License |
