# pi-castle-cdp-search

Web search for [pi](https://pi.dev) through a Chrome that is **already running**
on the wire network, driven over the Chrome DevTools Protocol.

It never launches a browser. It attaches to one a person or a launchd job
started — by default castle's shared Chrome on `10.88.0.25:9223` — opens a
background tab, reads the results page, and closes the tab again.

Because the search runs in a real browser with a real profile, it reaches pages
that reject datacentre traffic, and it costs nothing per query.

## The tool

One tool, `castle_cdp_search`. Deliberately **not** named `web_search`, so it
coexists with `pi-web-access` rather than shadowing it — both stay registered and
the model picks.

| Parameter | Type | Notes |
|---|---|---|
| `query` | string, required | As you would type it into a search box |
| `numResults` | integer, optional | 1–20, default 10 |
| `recency` | `day` \| `week` \| `month` \| `year`, optional | Omit for no time limit |

It returns titles, URLs and snippets — **search results only**. It does not fetch
the linked pages; follow up with a read/fetch tool for full content.

## Which browser it drives

Exactly the same configuration as [`pi-browser-harness`](../pi-browser-harness),
on purpose: one decision governs both packages, so they can never end up driving
different browsers in the same session.

| `BU_CDP_HTTP` | Behaviour |
|---|---|
| unset | the `/browser-target` choice stored by pi-browser-harness, else castle (`10.88.0.25:9223`) |
| `host:port` | that endpoint |
| `local` (also `localhost`, `off`, `none`, `0`, `no`) | this machine's own browser on `127.0.0.1:9222` |

Precedence is `BU_CDP_HTTP` > `/browser-target` > built-in default, matching the
harness. The `/browser-target` file (`~/.pi/agent/browser-target.json`) is read
only, never written — the harness owns that setting.

**One deliberate difference from the harness.** When the configured endpoint is
unreachable, this extension reports that and stops. The harness falls back from
its built-in default to a local browser; here that fallback would mean silently
running searches through the operator's own signed-in Chrome without saying so.
Say `local` explicitly if that is what you want.

The endpoint is re-resolved from `/json/version` on every connect, never cached:
castle recycles Chrome hourly and Chrome re-mints its browser UUID each launch,
so a pinned `ws://` URL breaks on the first recycle. A dropped socket reconnects
transparently on the next search.

## Behaviour worth knowing

- **Nothing is dialled at load.** pi runs extension factories in invocations that
  never start a session, so the connection is made on first search and closed by
  an idempotent `session_shutdown` handler.
- **One fresh background tab per search**, closed on every path including errors
  and aborts. `background: true` so it does not steal focus from whoever is
  looking at that screen.
- **Two concurrent searches**, queued beyond that.
- **20-second deadline** per search, covering connect, navigate and extract
  together. Esc aborts an in-flight navigation.
- **Every failure throws.** pi only sets `isError: true` when `execute()` throws;
  a returned `{ error }` object would read to the model as a search that simply
  found nothing.

## What it touches

The browser is shared and belongs to a person. Worth being aware of:

- Queries the agent runs land in that browser profile's history and cookies, and
  in the Google account's search history if that profile is signed in.
- Tabs open and close on someone's screen. `background: true` keeps them from
  stealing focus, but they are visible.
- Searching hard trips Google's rate limiter for the whole host — a handful of
  queries in a few seconds is enough to earn a `/sorry/` page that affects the
  human using that browser too. The cap of two concurrent searches limits this;
  it does not eliminate it. Measured once: roughly 30 queries in a few minutes
  cost about 90 minutes of blocking.
- **A decaying block does not look like a block.** Once the `/sorry/` page stops
  being served, Google returns an *empty results page* for a while instead —
  which is indistinguishable from a query that genuinely has no hits. It
  surfaces as `NoResultsError`, not `SearchChallengeError`. If several unrelated
  queries all come back empty, that is rate limiting; wait rather than retrying.

## When a captcha appears

Google will eventually serve a `/sorry/` interstitial, a consent wall, or a
recaptcha — especially if searches come in fast. The extension detects this and
throws a `SearchChallengeError` naming the challenge, the browser, and the URL a
human has to visit.

The agent cannot solve it. The guidelines tell it to stop searching and hand off
to the user, who opens that URL in the browser at the endpoint, clears the
challenge, and lets the agent retry. The cookie is profile-wide, so one pass
unblocks later searches.

The tab is closed rather than left open on the challenge page: leaving it would
let the operator solve it in place, but would also litter a shared browser with
abandoned tabs on every failure. The URL in the error is enough.

## Search parameters, and two surprises

Plain searches use `udm=14` — Google's "Web" tab: no AI overview, no carousels,
just ranked links, which is both cheaper to parse and closer to what was asked
for. Verified against the real browser:

- `tbs=qdr:*`, the parameter Google's own Tools menu writes, renders an **empty
  page** for this profile — `#search` present, no `#rso`, no results. The older
  `as_qdr=*` works and genuinely filters.
- Any date restriction combined with `udm=14` also renders that empty page.

So a time-limited search drops `udm` and uses `as_qdr`. The extractor handles
both layouts.

## Result extraction

`src/extract.ts` avoids Google's generated class names (`MjjYud`, `kb0PBd`, …)
entirely. It prefers the `data-snhf` / `data-sncf` hooks and otherwise falls back
to a structural walk: from each `<h3>`, take the enclosing link, climb until the
text grows past the header's, and abandon the climb if a second `<h3>` comes into
scope. Checked against both the classic SERP and the `udm=14` layout.

Google's markup will change anyway. When it does, a search returns
`NoResultsError` whose message distinguishes "results container rendered but
unparseable" (extractor needs updating) from "no results area at all" (query or
parameters).

## Install

```
pi install /Volumes/Sense/src/soarez/pi-castle-cdp-search
```

Or, once it is on castle alongside the harness:

```
pi install ssh://sz@10.88.0.25/Users/sz/repos/pi-castle-cdp-search.git
```

The ssh form is what pi-browser-harness uses, and it is the portable one: pi
records local-path installs by resolved absolute path, and `settings.json` is
shared across hosts through the dotfiles symlink.

To try it without installing:

```
pi -e ./src/index.ts
```

## Development

```
npm install
npm run typecheck
npm test          # unit tests, no browser needed
```

The tests cover endpoint resolution, URL building, formatting, the challenge
error, and the concurrency semaphore. The semaphore tests point at an
unreachable endpoint on purpose — the questions there are about slot
bookkeeping, and hitting a real browser to test a counter would be slow, flaky,
and rude to whoever is using it.

Requires Node 22+, for the global `WebSocket`. There are no runtime
dependencies; pi supplies `@earendil-works/pi-coding-agent`,
`@earendil-works/pi-ai` and `typebox` to extensions through its own loader.