aboutsummaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
authorTianren Dong <tianrendong.dtr@gmail.com>2026-05-14 22:09:07 -0400
committerTianren Dong <tianrendong.dtr@gmail.com>2026-05-14 22:09:07 -0400
commita13b6e4b51daf687e0cb16ef8fbc7040fde415a9 (patch)
tree56309e6cf89c98b428fa173bd785ef97bab5b9be /docs
parent815fd19c72f7a5a57f87eb0debe50194588e0202 (diff)
Restore terminal Chrome authorization
Diffstat (limited to 'docs')
-rw-r--r--docs/COMPARISON.md14
-rw-r--r--docs/EXAMPLES.md22
-rw-r--r--docs/FAQ.md24
3 files changed, 30 insertions, 30 deletions
diff --git a/docs/COMPARISON.md b/docs/COMPARISON.md
index 4f51291..d30cb6a 100644
--- a/docs/COMPARISON.md
+++ b/docs/COMPARISON.md
@@ -36,21 +36,21 @@ We benchmark in public — see [`../test-suite/`](../test-suite). Where exact sc
## Axis 1 — drivers (where pi-chrome lives)
-| Tool | Transport | Profile | Trusted events | Banner when controlling | Default detectable as bot |
+| Tool | Transport | Profile | Browser input | Banner when controlling | Default detectable as bot |
| --------------------------------- | ------------------------------------------ | ---------------------------------- | -------------------- | ----------------------------------- | ------------------------- |
| Playwright | CDP (own driver) | throwaway by default | always | always ("controlled by test software") | yes (webdriver flag, automation flags) |
| Puppeteer | CDP | throwaway by default | always | always | yes |
| Selenium | WebDriver / BiDi | throwaway | partial (BiDi improves) | always | most detectable |
| puppeteer-stealth / playwright-extra | CDP + patches | throwaway | always | always | medium (patches flags) |
| Raw CDP | direct devtools protocol | either (needs `--remote-debugging-port`) | always | always | yes |
-| **pi-chrome** | **Chrome extension bridge → local loopback** | **your real Chrome profile, signed-in cookies, extensions, history** | **opt-in** (`/chrome clicks on` or `trusted: true`) | **only when trusted mode is active** | **synthetic mode bypasses common detection signals**¹ |
+| **pi-chrome** | **Chrome extension bridge → local loopback** | **your real Chrome profile, signed-in cookies, extensions, history** | **always for input tools** | **while Chrome input is attached** | **low (real profile + Chrome input)¹** |
-¹ pi-chrome synthetic mode dispatches DOM events with `isTrusted=false` — most sites don't check; some anti-bot defenses do. The [`test-suite/`](../test-suite) grades both modes against common detection signals. Trusted mode uses `chrome.debugger` and shows Chrome's banner like every other CDP-based tool.
+¹ pi-chrome uses `chrome.debugger` for browser input and shows Chrome's banner like other CDP-based tools. The [`test-suite/`](../test-suite) grades browser-control behavior against common detection signals.
### What makes pi-chrome different on this axis
1. **Profile attach, not driver launch.** Every other driver fights cookie persistence, login walls, MFA, and extension state. pi-chrome inherits all of it because it *is* your Chrome.
-2. **Synthetic-first, trusted-on-demand.** Two-tier event model — agents pick the right tradeoff per call. Competitors are all-trusted (CDP) and always show the banner. pi-chrome avoids it by default; you opt in when a site needs it (autoplay gate, clipboard, file picker).
+2. **Chrome input against your real profile.** Interactive tools use CDP input for reliability while still controlling the Chrome profile you already use.
3. **Extension bridge transport.** No `--remote-debugging-port`, no throwaway Chromium. Survives Chrome auto-updates. Works alongside your normal Chrome usage.
4. **Honest result envelopes.** Every action returns `pageMutated`, `defaultPrevented`, `elementVisible`, `occludedBy`, `valueMatches`. Competitors return `void` or generic acks; agents loop blindly on broken clicks.
5. **Multi-session shared bridge.** Planner + worker + audit Pi sessions all drive the same Chrome concurrently.
@@ -73,7 +73,7 @@ These wrap a driver with an LLM loop. They are **higher-level than pi-chrome** a
| **OpenAI Operator** | proprietary | OpenAI's own VLM + browser; ChatGPT-integrated. | closed, hosted |
| **Project Mariner** (Google) | proprietary Chrome integration | Google's own VLM Chrome experiment. | closed |
| **Surfer 2 / Surfer-H** (H Company) | proprietary | Hosted proprietary agent stack. | closed, hosted |
-| **Anthropic Computer Use** | OS-level screenshots + mouse/keyboard | Broader than browser; trusted events at OS level. | closed (API) |
+| **Anthropic Computer Use** | OS-level screenshots + mouse/keyboard | Broader than browser; OS-level events. | closed (API) |
**Why pi-chrome is not on this list:** it's intentionally **not an agent**. There's no LLM loop, no `.act("click the blue button")`. Pi handles the loop; pi-chrome provides the primitives. This means:
@@ -126,7 +126,7 @@ Yes — you can export cookies and replay them, or point Playwright at your exis
Different security boundary, not strictly safer.
- **CDP-based tools** require `chrome --remote-debugging-port=...`. That port is unauthenticated and exposes the whole browser to any local process. Easy to misconfigure.
-- **pi-chrome** runs through an extension you install yourself with broad permissions (tabs, scripting, debugger, webNavigation). The bridge listens on `127.0.0.1:17318` loopback only. **Only install the bundled extension if you trust the source you got the npm package from.**
+- **pi-chrome** runs through an extension you install yourself with broad permissions (tabs, scripting, debugger, webNavigation). The bridge listens on `127.0.0.1:17318` loopback only, rejects browser-origin command requests, and keeps chrome_* tools locked until `/chrome authorize` is run in the current Pi session. **Only install the bundled extension if you trust the source you got the npm package from.**
If your threat model excludes extensions with broad permissions, neither approach is a fit — you want a sandboxed CI runner.
@@ -134,7 +134,7 @@ If your threat model excludes extensions with broad permissions, neither approac
## Public benchmarks worth knowing (for axis 2 / axis 3 comparison)
-Pi-chrome itself ships a benchmark suite ([`../test-suite/`](../test-suite)) of **38 primitive challenges** plus **4 hermetic BrowserGym-style long-horizon tasks** covering trusted-input, pointer humanization, keyboard fidelity, drag/drop, Shadow DOM, file uploads, network observability, fingerprint leaks, and agent-safety honeypots. Scoring is **expected-outcome-by-mode** (not raw PASS count): each challenge has expected verdicts per mode (`synthetic` / `trusted` / `manual`) and a tool grades itself by whether its actual outcome matches expectations. That's **driver-level** grading.
+Pi-chrome itself ships a benchmark suite ([`../test-suite/`](../test-suite)) of **38 primitive challenges** plus **4 hermetic BrowserGym-style long-horizon tasks** covering real input, pointer humanization, keyboard fidelity, drag/drop, Shadow DOM, file uploads, network observability, fingerprint leaks, and agent-safety honeypots. Scoring tracks expected outcomes per challenge instead of raw PASS count. That's **driver-level** grading.
For **agent-level** comparison (axis 2), the public benchmarks worth citing:
diff --git a/docs/EXAMPLES.md b/docs/EXAMPLES.md
index 905d278..a5cccf4 100644
--- a/docs/EXAMPLES.md
+++ b/docs/EXAMPLES.md
@@ -1,6 +1,6 @@
# pi-chrome examples
-Real, useful agent prompts. Drop any of these into Pi after running `/chrome onboard`. Each one uses Chrome tabs and accounts you already have.
+Real, useful agent prompts. Drop any of these into Pi after running `/chrome onboard`, then `/chrome authorize`. Each one uses Chrome tabs and accounts you already have.
## Daily workflow
@@ -129,14 +129,14 @@ component re-rendered with the new value.
```text
chrome_upload_file paths=[./fixtures/avatar.png] selector="input[type=file]"
-# No native file picker opens. Works with React/Vue/Angular controlled inputs.
+# Uses Chrome file-input control. No native file picker opens.
```
### Drag-to-reorder lists
```text
chrome_drag fromUid=row-3 toUid=row-1
-# Fires real HTML5 dragstart/dragover/drop with a shared DataTransfer.
+# Uses Chrome pointer drag through its input layer.
```
## Multi-session patterns
@@ -153,14 +153,14 @@ chrome_drag fromUid=row-3 toUid=row-1
A third Pi session can run `chrome_snapshot` periodically in `background: true` mode and post summaries via `pi-qq` — handy for long-running flows.
-## When to prefer trusted clicks
+## Chrome input
-Pass `trusted: true` on `chrome_click` (or run `/chrome clicks on`) when:
+Interactive tools use Chrome's real input layer by default: clicks, typing, fill, keys, hover, drag, scroll, and touch. This is reliable for:
-- the click should open a file picker
-- the click should write to the clipboard or read it
-- the click should start an audio/video play
-- the click should request fullscreen / push permission
-- the page is wrapped in a strict user-activation guard (some paywalls / login flows)
+- sign-in flows
+- guarded buttons
+- audio/video controls
+- fullscreen / permission prompts
+- pages with strict CSP or user-activation checks
-Everything else is faster and quieter without it.
+Chrome may show its debugger banner while pi-chrome is attached.
diff --git a/docs/FAQ.md b/docs/FAQ.md
index 2839f8f..33902d3 100644
--- a/docs/FAQ.md
+++ b/docs/FAQ.md
@@ -14,23 +14,23 @@ By default no — extensions need explicit "Allow in incognito" permission. Togg
## Will sites detect that I'm automating?
-For **synthetic** (quiet) input: yes, technically. `event.isTrusted` is `false`. Most sites don't check; some anti-bot defenses do.
+Interactive controls use Chrome's real input layer via CDP: pointer paths are humanized, key cadence has variance, and normal user-activation gates are satisfied. Some detectors check for the `chrome.debugger` API attached and Chrome will show the "Chrome is being debugged" banner.
-For **trusted** (CDP) input: events are `isTrusted=true`, pointer paths are humanized, key cadence has variance. Most fingerprint-based detectors don't fire. Some specifically check for the `chrome.debugger` API attached and will show the "Chrome is being debugged" banner. That banner is the visible cost of trusted mode.
-
-The [`test-suite/`](../test-suite) grades both modes against common detection signals.
+The [`test-suite/`](../test-suite) grades browser-control behavior against common detection signals.
## Why do I see a banner saying "Pi Chrome Connector started debugging this browser"?
-That's Chrome's built-in warning when an extension uses `chrome.debugger`. pi-chrome uses it only in trusted-input mode. If you don't want to see it, run `/chrome clicks off` and accept that some sign-in flows / file pickers / clipboard ops won't work.
+That's Chrome's built-in warning when an extension uses `chrome.debugger`. pi-chrome uses Chrome's input layer for interactive controls, so the banner appears while attached.
## Can a malicious page escape and access my other tabs?
-No — pages cannot directly talk to extensions. Commands flow agent → local bridge (`127.0.0.1:17318`) → extension → tab. The bridge binds to loopback only. The risk surface is **other local processes** that could connect to `127.0.0.1:17318` and impersonate Pi. If that's in your threat model, run pi-chrome on a separate user account.
+No — pages cannot directly talk to extensions. Commands flow agent → local bridge (`127.0.0.1:17318`) → extension → tab. The bridge binds to loopback only and rejects browser-origin command requests, so ordinary web pages cannot use CORS to drive it.
+
+Chrome control is also locked per Pi session until you run `/chrome authorize`; `/chrome revoke` locks it again. The remaining risk surface is **other local processes running as you** that can connect to loopback and imitate Pi. If that's in your threat model, run pi-chrome in a separate OS user account.
## Can multiple Pi sessions use it at once?
-Yes. The first session opens the local bridge; later sessions detect it and pipe their commands through the same bridge. Planner + worker + audit can all drive the same Chrome concurrently.
+Yes. The first session opens the local bridge; later sessions detect it and pipe their commands through the same bridge. Each Pi session must be authorized with `/chrome authorize` before its chrome_* tools work.
## Why can't this be on the Chrome Web Store?
@@ -38,7 +38,7 @@ Web Store extensions cannot communicate with a local process bridge controlled b
## What happens when I update pi-chrome?
-`/chrome doctor` will warn you if the loaded extension is older than the installed `pi-chrome`. Reload it from `chrome://extensions` to pick up the new version. Trusted-input mode in particular requires re-approving the `debugger` permission once.
+`/chrome doctor` will warn you if the loaded extension is older than the installed `pi-chrome`. Reload it from `chrome://extensions` to pick up the new version. Updates that add Chrome permissions may require re-approval once.
## What's the install footprint?
@@ -51,24 +51,24 @@ The Pi-facing tools are thin wrappers around an HTTP bridge at `127.0.0.1:17318`
## Does `chrome_evaluate` work on strict-CSP pages?
-Yes. The handler compiles with `new Function(...)` in the MAIN world, which works under `script-src 'self'` without `'unsafe-eval'`. Multi-statement bodies are supported via a statement-mode fallback. Exceptions are surfaced to the agent.
+Not always. `chrome_evaluate` and `chrome_snapshot` run in the page's MAIN world through the Function constructor, so pages whose CSP blocks `'unsafe-eval'` can reject them. `chrome_screenshot`, `chrome_navigate`, tab tools, and real Chrome input still work because they use extension/browser APIs rather than page JavaScript.
## Why does my click return `pageMutated=false`?
Either:
- The element was occluded (look for `occludedBy: <selector>` in the envelope).
- The click handler called `event.preventDefault()` and the page intentionally ignored it.
-- The site rejects synthetic events. Try `trusted: true` or `/chrome clicks on`.
+- The target changed after your snapshot; take a fresh snapshot or screenshot.
The result envelope tells you which one. **Don't blind-retry.**
## Why does `chrome_type` return `valueMatches=false`?
-The editor rejected the synthetic input. Common culprits: contenteditable rich-text editors, native date pickers, masked-input libraries. Try `chrome_fill` (uses framework-aware native setters) or `trusted: true`.
+The field rejected or transformed the typed value. Common culprits: contenteditable rich-text editors, native date pickers, masked-input libraries, or masks. Try `chrome_fill`, then verify with `includeSnapshot=true`.
## How do I attach a file to a React file input?
-`chrome_upload_file` — populates `input.files` via a real `DataTransfer` and fires `input` + `change` events. It does **not** open the native file picker (no synthetic event can; that's a user-activation gate). Works with React/Vue/Angular controlled inputs.
+`chrome_upload_file` — uses Chrome DevTools file-input control and fires `input` + `change` events. It does **not** open the native file picker. Works with React/Vue/Angular controlled inputs.
## Can it record videos?