pi-chrome vs. the rest of the browser-automation landscape

This is the honest "which tool when" page. The browser-automation space has three different layers and people often compare across them — let's not do that.

We benchmark in public — see ../test-suite/. Where exact scores matter (WebVoyager, WorkArena++, BrowseComp, Mind2Web 2), check the live leaderboards; they shift monthly.

TL;DR

pi-chrome is primitives, not an opinionated agent loop. Think of it as "Playwright for the Chrome you're already signed into" — and pluggable under any agent framework above the line.

The three axes you should compare on

pi-chrome is axis 1. You can wrap it with any axis-2 agent framework, or run it directly from a Pi agent's chrome_* tool calls. It explicitly does NOT compete with axis 3 — it runs locally inside your Chrome.

Axis 1 — drivers (where pi-chrome lives)

¹ pi-chrome uses chrome.debugger for browser input and shows Chrome's banner like other CDP-based tools. The test-suite/ grades browser-control behavior against common detection signals.

What makes pi-chrome different on this axis

1. Profile attach, not driver launch. Every other driver fights cookie persistence, login walls, MFA, and extension state. pi-chrome inherits all of it because it is your Chrome.
2. Chrome input against your real profile. Interactive tools use CDP input for reliability while still controlling the Chrome profile you already use.
3. Extension bridge transport. No --remote-debugging-port, no throwaway Chromium. Survives Chrome auto-updates. Works alongside your normal Chrome usage.
4. Structured action results. Input tools return target coordinates/tags and can include a fresh snapshot (includeSnapshot) so agents can verify state instead of blindly retrying.
5. Multi-session shared bridge. Planner + worker + audit Pi sessions all drive the same Chrome concurrently.
6. Stable element uids. chrome_snapshot returns deterministic uids you can pass to subsequent actions — similar to BrowserGym's bid, but built into the snapshot tool itself.

Axis 2 — agent frameworks (built on top of axis 1)

These wrap a driver with an LLM loop. They are higher-level than pi-chrome and complementary, not competitors.

Why pi-chrome is not on this list: it's intentionally not an agent. There's no LLM loop, no .act("click the blue button"). Pi handles the loop; pi-chrome provides the primitives. This means:

• You can use pi-chrome under Browser Use, Stagehand, LangGraph, or any other agent framework (see Interop).
• You don't pay for an opinion you don't want.
• Agent improvements compound across all your workflows because the primitives stay stable.

Axis 3 — cloud browser providers (orthogonal)

pi-chrome doesn't compete here. It runs locally in your Chrome. The right framing if someone asks: "no cloud cost, no session-handoff, no rate limits — and the agent runs against your real logged-in state." If you need fleets of isolated Chromes in CI, you want one of these.

Interop

pi-chrome exposes tools that any Pi agent can call. If you want to use it from outside Pi:

1. The local bridge speaks HTTP JSON over 127.0.0.1:17318 (default). The API is internal; use the Pi tool surface unless you are building an adapter.
2. Tool surface mirrors Playwright closely (click/type/navigate/snapshot/screenshot/evaluate/wait_for) so adapter code is short.
3. includeSnapshot on input tools lets agent harnesses verify state after actions.

If you want a first-class pi-chrome adapter for Browser Use / Stagehand / LangGraph, file an issue with your use case.

"But Playwright has storageState / Puppeteer has user-data-dir"

Yes — you can export cookies and replay them, or point Playwright at your existing profile directory. In practice for agent workflows that breaks down fast:

1. OAuth + SSO providers (Okta, Google, GitHub) often pin sessions to TLS fingerprints, device IDs, and browser-extension state that doesn't survive replay or a parallel Chrome instance.
2. MFA tokens expire mid-run.
3. Internal admin tools hard-pin to your real device.
4. Pointing Playwright at your real user-data-dir requires closing your normal Chrome (Chrome won't share the profile lock). pi-chrome doesn't fight you for the profile because it lives inside it.
5. Watching the agent work in your real window is a different UX than a hidden parallel Chrome. Demos, pair-driving, and confidence-building all want axis-1-with-attach.

"Is this safer than CDP?"

Different security boundary, not strictly safer.

• CDP-based tools require chrome --remote-debugging-port=.... That port is unauthenticated and exposes the whole browser to any local process. Easy to misconfigure.
• pi-chrome runs through an extension you install yourself with broad permissions (tabs, scripting, debugger, webNavigation). The bridge listens on 127.0.0.1:17318 loopback only, rejects browser-origin command requests, and keeps chrome_ tools locked until /chrome authorize is run in the current Pi session. Only install the bundled extension if you trust the source you got the npm package from.*

If your threat model excludes extensions with broad permissions, neither approach is a fit — you want a sandboxed CI runner.

Public benchmarks worth knowing (for axis 2 / axis 3 comparison)

Pi-chrome itself ships a benchmark suite (../test-suite/) of 43 primitive challenges plus 4 hermetic BrowserGym-style long-horizon tasks covering trusted input, pointer humanization, keyboard fidelity, drag/drop, Shadow DOM, iframes, file uploads, strict-CSP screenshot fallback and CDP eval/snapshot bypass, dynamic waits, tab lifecycle, network observability, fingerprint leaks, and agent-safety honeypots. Scoring tracks expected outcomes per challenge instead of raw PASS count, with core, conditional, and quality gate buckets. That's driver-level grading.

For agent-level comparison (axis 2), the public benchmarks worth citing:

Cite live leaderboards rather than hard-coded numbers; agent scores shift monthly.

Reproducing pi-chrome's driver-level claims

Run ../test-suite/ against any browser-control tool. Each challenge exposes window.__verdict / window.__reason / window.__events, so any tool (Playwright, Puppeteer, Selenium, Stagehand, pi-chrome) can grade itself deterministically. Headline release scoring should use the core gate; conditional depends on declared environment capabilities, and quality tracks adversarial/humanization regressions.

cd test-suite && python3 -m http.server 8765
# open http://127.0.0.1:8765/ in the Chrome window the tool controls

If you build a competing tool, please open a PR with your scores.