aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorTianren Dong <tianrendong.dtr@gmail.com>2026-09-10 10:22:43 -0400
committerGitHub <noreply@github.com>2026-09-10 10:22:43 -0400
commit38475fd32113a13c3a4929b458079498a644b5fc (patch)
tree467d20b860e5673aea049081ade1c0a2e0e1e578
parent304c40a7d42b3c8ad8e93627012c53b086c72065 (diff)
parent502bcc818572bf2f9df906e472ad00ab60dd977e (diff)
Merge pull request #15 from tianrendong/docs/readme-onboarding
docs: improve pi-chrome README and onboarding
-rw-r--r--README.md177
1 files changed, 83 insertions, 94 deletions
diff --git a/README.md b/README.md
index a6774bc..3244fae 100644
--- a/README.md
+++ b/README.md
@@ -1,156 +1,145 @@
# pi-chrome
-> Let [Pi](https://pi.dev) use your existing signed-in Chrome profile after explicit authorization.
+**Give Pi the Chrome you're already signed into.**
-**MIT · 0 runtime deps · loopback-only bridge (`127.0.0.1:17318`) · inspectable unpacked Chrome extension.** Review [`extensions/chrome-profile-bridge/browser-extension/`](./extensions/chrome-profile-bridge/browser-extension) before loading. Verify setup with `/chrome doctor`.
+Debug your app, inspect signed-in dashboards, and capture screenshots using your existing Chrome profile—without setting up a separate automation browser.
-```text
-You: "Find my open GitHub PR tab, summarize review state, and screenshot failing CI."
-Agent: chrome_tab(list) → chrome_snapshot(uid:…) → chrome_screenshot(...)
- ✓ 3 reviewers, 1 change requested, CI red on iOS. Saved → .pi/chrome-screenshots/ci.png
-You: [keeps coding — agent never asked you to log in]
-```
+Built for the [Pi coding agent](https://pi.dev). Pi's Chrome tools stay locked until you authorize the current session.
-`pi-chrome` runs through a small Chrome extension inside the Chrome profile **you already use** — including sites where you're already signed in. Agents can inspect or control Chrome only after you run `/chrome authorize` in current Pi session.
+[Quick start](#quick-start) · [Examples](./docs/EXAMPLES.md) · [Safety](#safety) · [Contributing](./CONTRIBUTING.md)
----
+[![npm version](https://img.shields.io/npm/v/pi-chrome.svg)](https://www.npmjs.com/package/pi-chrome)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](./LICENSE)
-## Install
+## What you can do
-```bash
-pi install npm:pi-chrome
-```
+Try prompts like these after setup:
-In Pi:
+| Use case | Ask Pi |
+| --- | --- |
+| **Debug a signed-in app** | “Reproduce the filter bug in my staging app. Inspect captured console and network errors, then save a screenshot.” |
+| **Understand an existing page** | “Find my open dashboard tab and summarize what's on the page. Don't change anything.” |
+| **Create evidence for a PR** | “On my local app, capture the empty, loading, and populated states of this feature for my PR.” |
-```text
-/chrome onboard
-```
+Pi gets tools to inspect pages, click, type, fill forms, scroll, upload files, capture screenshots, and inspect captured console logs and `fetch`/`XMLHttpRequest` responses. You describe the task; Pi handles the agent loop.
-This opens `chrome://extensions` and copies bundled extension path. In Chrome Extensions:
+**Best fit:** interactive workflows in the Chrome profile you already use. For deterministic CI tests, consider a test framework such as Playwright; for fleets of isolated browsers, consider a hosted browser service. See [more workflows](./docs/EXAMPLES.md) and [browser-tool comparisons](./docs/COMPARISON.md).
-1. Turn on **Developer mode**.
-2. Click **Load unpacked**.
-3. Open path field with **Cmd+Shift+G** on macOS or **Ctrl+L** on Windows/Linux.
-4. Paste copied path.
-5. Press Enter.
+## Quick start
-Reload Pi so installed package loads:
+**Requirements:** [Pi](https://pi.dev) and Google Chrome. Setup includes a one-time manual installation of the bundled Chrome companion extension.
-```text
-/reload
-```
+> **Trust and privacy:** The companion has broad browser permissions and runs in your real Chrome profile. Review [its source](./extensions/chrome-profile-bridge/browser-extension/) before loading it, and authorize only tasks you trust. The browser bridge is local, but page content returned to Pi may be sent to your configured model provider.
-Check bridge:
+### 1. Install and load the Pi package
-```text
-/chrome doctor
+In your terminal:
+
+```bash
+pi install npm:pi-chrome
```
-You should see:
+Start Pi with `pi`. If Pi is already running, run `/reload` in that session **before** using the `/chrome` commands.
-```text
-✓ Chrome is connected (...)
-```
+### 2. Connect Chrome
-Authorize current session:
+In Pi:
```text
-/chrome authorize
-/chrome doctor
+/chrome onboard
```
-Second doctor run should show all checks passing.
+The setup dialog shows the companion extension's folder path.
----
+- **macOS:** after confirmation, Pi opens `chrome://extensions`, reveals the companion folder in Finder, and copies its path to your clipboard.
+- **Windows/Linux:** copy the folder path shown in the dialog and open `chrome://extensions` manually. Automatic desktop opening and clipboard setup are currently macOS-only.
-## What it can do
+In Chrome:
-- Read and summarize pages you're already signed into.
-- Click, type, fill forms, scroll, drag, tap, and upload files.
-- Capture screenshots for bugs, PRs, and demos.
-- Inspect console logs and captured `fetch`/`XMLHttpRequest` responses.
-- Manage tabs without taking over your active window.
-
-Tool parameters and gotchas are documented inline in Pi.
-
-### Typing into rich editors
+1. Turn on **Developer mode**.
+2. Click **Load unpacked**.
+3. Select the companion folder shown by `/chrome onboard`. On macOS, press **Cmd+Shift+G** in the folder picker and paste the copied path.
-`chrome_type` and `chrome_fill` use one native CDP `Input.insertText` operation for focused contenteditables. This avoids per-character delays for long text and preserves Unicode/newlines. Ordinary inputs and textareas retain individual key events.
+### 3. Authorize and verify
-For an editor that needs a `keydown` for every character, pass `perCharacter:true`. Bulk insertion still uses Chrome's input system, but does not emit per-character key events or a clipboard `paste` event. Use `includeSnapshot:true` to verify the result; `chrome_fill` still honors `domFallback:false` when synthetic fallback is unwanted.
+In Pi:
----
+```text
+/chrome authorize
+/chrome doctor
+```
-## Safety model
+Approve the authorization prompt for a task you trust. The default authorization lasts **15 minutes**. Doctor should report `✓ Chrome is connected (...)`; follow its instructions if any checks fail.
-Chrome control is locked by default. Authorize per Pi session:
+Then try a read-only first task:
```text
-/chrome authorize # 15 minutes
-/chrome authorize 30m # custom duration
-/chrome authorize indefinite
-/chrome revoke # lock again
-/chrome status
+List my open Chrome tabs without navigating, clicking, or changing anything.
```
-Safety properties:
+Run `/chrome revoke` when finished. Use `/chrome authorize` again whenever you want to grant access for another task or session.
+
+## Safety
-- Extension runs in your real Chrome profile and has broad tab/scripting permissions. Install only from trusted package source.
-- Pi side binds to `127.0.0.1:17318` only; no default network exposure.
-- Bridge rejects browser-origin command requests, so ordinary web pages cannot drive it through CORS.
-- Each Pi session gets its own automation target; user tabs/windows are not closed by cleanup.
-- `/chrome revoke` closes only calling session's automation target.
+- **Per-session approval.** Pi's Chrome tools require `/chrome authorize`. `/chrome revoke` locks them and requests cleanup of that session's owned automation tabs. Cleanup preserves existing user tabs.
+- **Separate targets by default.** Page actions without an explicit target use a session-owned automation window or tab. The agent can deliberately target an existing tab when your task calls for it.
+- **Local transport, not a sandbox.** The bridge binds to `127.0.0.1:17318` and rejects browser-origin command requests. It does not authenticate arbitrary non-browser local callers; it is not protection against hostile processes on your machine.
+- **Background mode by default.** Tools cannot override it to explicitly focus Chrome or activate tabs. This is not a zero-focus guarantee: page scripts, trusted input, native prompts, and Chrome/OS behavior can still affect focus. Chrome may also show its debugger banner while attached.
-Security details: [`SECURITY.md`](./SECURITY.md). Architecture details: [`docs/ARCHITECTURE.md`](./docs/ARCHITECTURE.md).
+Read the [security policy](./SECURITY.md) and [background-mode scope and risks](./docs/ARCHITECTURE.md#scope-and-risks) before using it with sensitive accounts.
----
+### Limits
+
+This is browser automation, not full OS control. Native Chrome/OS dialogs, password-manager prompts, passkeys/security keys/biometrics, CAPTCHA challenges, cross-origin iframe DOM access, rich multitouch/stylus gestures, and arbitrary desktop apps are outside its reliable tool surface. Some workflows need human assistance.
+
+If page inspection or evaluation is blocked, use screenshots and coordinate input where possible. Background pages can throttle rendering or reject focus-gated actions. See the [FAQ](./docs/FAQ.md) for details.
## Commands
```text
-/chrome onboard # guided setup
-/chrome doctor # connectivity + version + eval checks
-/chrome status # connection + auth + background state
-/chrome authorize [duration]
-/chrome revoke
-/chrome background on # default: hard background policy
-/chrome background off # foreground/watch mode
+/chrome onboard # one-time companion setup
+/chrome authorize # authorize this Pi session for 15 minutes
+/chrome authorize 30m # choose a duration
+/chrome authorize indefinite # no time limit; revoke when finished
+/chrome revoke # lock tools and request session cleanup
+/chrome doctor # connection, version, and page checks
+/chrome status # connection, authorization, and background state
+/chrome background on # default: block explicit focus/tab activation
+/chrome background off # allow foreground/watch mode
/chrome background status
```
-If loaded extension is older than installed `pi-chrome`, `/chrome doctor` tells you to reload it from `chrome://extensions`.
+Tool parameters are documented inline in Pi. See [architecture](./docs/ARCHITECTURE.md) for target ownership, screenshot behavior, and background-policy details.
-### Background policy
+### Updating and troubleshooting
-`/chrome background on` is enforced, not an overridable default. Per-call `background:false` cannot bring Chrome forward, new tabs stay inactive, and `chrome_tab activate` is blocked. Use the existing `/chrome background off` for foreground/watch mode; per-call `background:true` still works when that mode is off.
+After `pi update npm:pi-chrome`, run `/reload` in Pi and reload **Pi Chrome Connector** in `chrome://extensions`. Run `/chrome doctor` to check the connection and companion version.
-Screenshots use CDP without activating background tabs. Debugger/capture failures return errors, never an activation fallback. Reload both Pi and the Chrome companion after upgrading; old companions reject background tab creation/screenshots rather than silently switching tabs.
+If Chrome is not responding, confirm the companion is enabled and keep Chrome open. If page checks fail on a Chrome internal page, try a regular web page and run Doctor again. Follow its version-mismatch or session-restart instructions when shown.
-This prevents explicit pi-chrome focus/activation, not every Chrome/OS side effect. Trusted input, page popups, native prompts, debugger banners, and macOS Spaces can still affect focus. Inactive pages may throttle rendering or reject focus-gated actions. See [scope and risks](./docs/ARCHITECTURE.md#scope-and-risks).
+### Typing into rich editors
----
+`chrome_type` and `chrome_fill` use one native CDP `Input.insertText` operation for focused contenteditables. This avoids per-character delays for long text and preserves Unicode/newlines. Ordinary inputs and textareas retain individual key events.
-## Limits
+For an editor that needs a `keydown` for every character, pass `perCharacter:true`. Bulk insertion still uses Chrome's input system, but does not emit per-character key events or a clipboard `paste` event. Use `includeSnapshot:true` to verify the result; `chrome_fill` still honors `domFallback:false` when synthetic fallback is unwanted.
-`pi-chrome` works best on web-page workflows exposed through DOM, screenshots, tabs, network, console, and Chrome input. It is not full OS automation.
+## Tests and documentation
-Current limits include native Chrome/OS surfaces, print/save dialogs, permission bubbles, password-manager prompts, cross-origin iframe DOM access, CAPTCHA/bot challenges, passkeys/security keys/biometrics, rich multitouch/pinch/stylus gestures, and arbitrary desktop apps.
+The [benchmark suite](./test-suite/README.md) includes **44 browser challenges**, hermetic multi-step tasks, and mocked regression tests for input, screenshots, session cleanup, and background policy. These are reproducible test cases—not a guarantee that every website or OS interaction works.
-For strict-CSP pages, use screenshots + coordinate input when snapshot/evaluate paths are blocked.
+Run mocked regressions with `npm test` (Node.js 22.13+; no live Chrome required). Follow the benchmark guide for live browser checks.
----
+- [Examples](./docs/EXAMPLES.md) — prompts and workflows to try.
+- [FAQ](./docs/FAQ.md) — compatibility, setup questions, and limitations.
+- [Architecture](./docs/ARCHITECTURE.md) — bridge, session lifecycle, and background policy.
+- [Comparison](./docs/COMPARISON.md) — where pi-chrome fits among browser tools.
+- [Changelog](./CHANGELOG.md) — release history.
-## Docs
+## Contributing
-- Examples: [`docs/EXAMPLES.md`](./docs/EXAMPLES.md)
-- FAQ: [`docs/FAQ.md`](./docs/FAQ.md)
-- Comparison: [`docs/COMPARISON.md`](./docs/COMPARISON.md)
-- Security: [`SECURITY.md`](./SECURITY.md)
-- Benchmark suite: [`test-suite/README.md`](./test-suite/README.md)
-- Architecture: [`docs/ARCHITECTURE.md`](./docs/ARCHITECTURE.md)
+Bug reports, reproducible browser challenges, and real workflow demos are welcome. Include `/chrome doctor` output and reproduction steps; redact private URLs and page content. See [CONTRIBUTING.md](./CONTRIBUTING.md).
----
+If pi-chrome is useful, consider starring the repo or sharing a workflow that worked for you.
## License