1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
|
# pi-chrome
[](https://www.npmjs.com/package/pi-chrome)
Control the Chrome profile you already use from Pi.
`pi-chrome` gives Pi agents browser tools for your **real Chrome windows, tabs, and authenticated sessions**. It uses a companion Chrome extension instead of the Chrome DevTools Protocol (CDP), so it does not launch a throwaway debug browser profile and does not require re-signing into the apps you already have open.
Multiple Pi sessions can use Chrome at the same time. The first Pi session starts the local bridge; later sessions automatically detect that bridge and submit their Chrome commands through it.
## Why try it?
- **Uses your existing Chrome profile** — works with the Chrome windows/tabs you are already using, including logged-in GitHub, admin dashboards, local apps, and internal tools.
- **Watch your authenticated Chrome work** — by default, `chrome_*` tool calls focus Chrome and activate the target tab so you can see the agent inspect, navigate, click, and type in real time. Switch to silent/background mode for the whole session with `/chrome-background`, or pass `background: true` on a single tool call when you want quiet.
- **Full browser automation toolkit for Pi** — list/create/activate/close tabs, snapshot pages with usable CSS selectors, navigate, evaluate JavaScript, click, type, press keys, wait for page state, and capture screenshots.
- **Built-in setup and agent guidance** — `/chrome-onboard` walks users through installing the companion extension, `/chrome-doctor` checks connectivity and version drift, screenshots save to disk, and the prompt primer tells agents to inspect with `chrome_snapshot` before acting and avoid destructive actions unless explicitly requested.
## Install
```bash
pi install npm:pi-chrome
```
For local development from a checkout:
```bash
pi install ./pi-chrome
```
### Why an unpacked Chrome extension?
`pi-chrome` cannot ship through the Chrome Web Store: a Web Store extension is not allowed to talk to a local bridge controlled by another tool. Instead it ships as a small, MIT-licensed unpacked extension in `extensions/chrome-profile-bridge/browser-extension/` — read the source before loading. `/chrome-doctor` reports the loaded extension version and warns when it drifts from the installed `pi-chrome`.
## First-time setup
In Pi, run:
```text
/chrome-onboard
```
Pi first shows setup instructions and waits for confirmation. Press Enter to continue. On macOS it will:
- open `chrome://extensions`
- reveal the bundled `browser-extension` folder in Finder
- copy the extension folder path to your clipboard
Then in Chrome:
1. Enable **Developer mode**.
2. Click **Load unpacked**.
3. Select the revealed/copied `browser-extension` folder.
4. Return to Pi and run:
```text
/chrome-doctor
```
Expected output:
```text
Performing Chrome bridge health check
pi-chrome v<version>
• Local bridge: mode=server, url=http://127.0.0.1:17318
✓ Companion Chrome extension responding (ID: <chrome-extension-id>, ext v<version>)
```
## Trusted-input mode (CDP)
By default, `chrome_*` clicks and keystrokes are **synthetic** DOM events (`event.isTrusted === false`). They drive React/Vue/Angular state correctly but **do not** satisfy Chrome's user-activation gates: clipboard write, fullscreen, file picker, and autoplay all need a real user gesture.
pi-chrome can optionally route input through `chrome.debugger` (CDP `Input.dispatchMouseEvent` / `Input.dispatchKeyEvent`) so each event arrives as `isTrusted=true`, satisfies user-activation, and bypasses site bot-detection that filters synthetic events. The tradeoff: Chrome pins a yellow *"Pi Chrome Connector started debugging this browser"* banner to the top of any debugged tab.
Usage:
```text
/chrome-trusted on # all chrome_* tools dispatch via CDP
/chrome-trusted off # synthetic events only (default)
/chrome-trusted auto # CDP only when a tool passes trusted=true
/chrome-trusted status # show current mode and attached tabs
```
For a single call, pass `trusted: true` on `chrome_click`, `chrome_type`, `chrome_fill`, `chrome_key`, `chrome_hover`, `chrome_drag`, or `chrome_scroll`. The per-call value always wins over the session toggle.
First time you upgrade to a pi-chrome that uses trusted input, Chrome will prompt to re-approve the extension's new `debugger` permission — do that once in `chrome://extensions`.
## Background mode
By default, `chrome_*` tools focus Chrome and activate the target tab so you can watch the agent work — great for demos, pair-driving, debugging, and first-time confidence that things are happening.
When you want quiet (planner / audit / worker sessions running alongside your editor), turn background mode on for the whole Pi session:
```text
/chrome-background # toggle
/chrome-background on # explicit
/chrome-background off # explicit
```
For a single tool call, the agent can pass `background: true` directly. The per-call value always wins over the session toggle.
## Quick demo prompts
After setup, try one of these in Pi:
Silent inspection (no Chrome interruption):
```text
Inspect my active GitHub tab with chrome_snapshot using background:true and summarize the PR state without focusing Chrome.
```
Existing authenticated tab:
```text
Use chrome_tab list to find my existing GitHub tab, chrome_snapshot it, then summarize the visible PR state. Do not click anything yet.
```
Local web app repro with screenshot:
```text
Use chrome_tab list to find my localhost app, inspect it with chrome_snapshot, navigate through the bug repro flow, and save a screenshot when you reach the broken state.
```
## Recipes
Copy-paste these into Pi after setup. Each one uses tabs you already have open and accounts you are already signed into.
- **PR triage:** "Use chrome_tab list to find my GitHub notifications tab, snapshot it, and summarize PRs needing my review."
- **Linear standup:** "Open my Linear current cycle in the active tab, snapshot it, and write me a 5-bullet standup."
- **Bug repro with evidence:** "Open the staging app I'm already signed into, reproduce <bug>, and save a screenshot of each step under ./repro/."
- **Form auto-fill (no submit):** "Open <vendor> portal, fill the new-vendor form from this JSON, but stop before submit."
- **Admin cross-check:** "Across my Stripe / Postmark / our admin tabs, find any user where state disagrees."
- **Local dev visual diff:** "Snapshot localhost:3000 and the staging URL of the same page; tell me what's visually different."
- **Auth-only data pull:** "Open my analytics dashboard tab and chrome_evaluate to extract today's KPIs from the page state."
Screenshots save under `.pi/chrome-screenshots/` by default, which composes nicely with PR demo workflows.
## Diagnostics
- `/chrome-doctor` — single command that checks connectivity and reports the loaded Chrome extension ID + version, plus a one-line fix for common setup failures (extension not loaded, bridge owner stale after `pi update`, version mismatch between pi-chrome and the loaded Chrome extension).
If the Chrome extension you have loaded is older than `pi-chrome` on disk, `/chrome-doctor` will tell you to reload it from `chrome://extensions`.
## Compose with
- **pi-qq** — ask side questions about what the agent saw in Chrome without polluting the main transcript: `/qq summarize what the active GitHub tab shows`.
- **pi-bar** — watch context pressure as the agent scrapes large pages; the footer's red threshold is a clean signal to `/qq` for a recap before context overflows.
- **PR demo skills** (such as `ios-pr-agent` / `ios-demo-record` workflows) — `chrome_screenshot` writes to `.pi/chrome-screenshots/` so you can attach images to PR descriptions or demo bundles.
## Tools
The package registers these Pi tools:
- `chrome_launch` — setup/help entry point; opens a URL if the bridge is connected
- `chrome_tab` — list, create, activate, close, or inspect tabs
- `chrome_snapshot` — inspect title, URL, visible text, viewport, and clickable/focusable selectors
- `chrome_navigate` — navigate an existing tab
- `chrome_evaluate` — evaluate JavaScript in a tab
- `chrome_click` — click by selector or coordinates
- `chrome_type` — type text, optionally focusing a selector first
- `chrome_key` — send keyboard keys
- `chrome_wait_for` — wait for a selector or expression
- `chrome_screenshot` — capture viewport screenshots to disk
These tools are especially useful for authenticated web app debugging, repro flows, admin workflows, visual checks, and inspecting local development pages without rebuilding login state.
<details>
<summary><strong>How it works (technical details)</strong></summary>
Pi starts a local bridge on `127.0.0.1:17318`. The companion Chrome extension, installed in your normal Chrome profile, polls that local bridge for commands and executes them using Chrome extension APIs.
If another Pi session is already running the bridge, additional Pi sessions automatically act as clients of that shared bridge. This lets planner/audit/worker sessions all use the same authenticated Chrome profile concurrently without fighting over the port.
This is intentionally different from CDP-based tools: the browser extension lives inside the profile you already use, so Pi can interact with existing tabs and authenticated page state.
</details>
## Security model
The companion Chrome extension runs in the Chrome profile where you install it and has broad tab/scripting permissions. Only install it from a package source you trust.
The Pi side listens on `127.0.0.1:17318` by default. Override before starting Pi with:
```bash
PI_CHROME_BRIDGE_PORT=17319 pi
```
|
