| Age | Commit message (Collapse) | Author |
|---|
|
Concrete numbers from development rather than a vague warning: Google
tolerates a lot then blocks for ~90 minutes, Brave challenges after one or
two queries and clears in ~15-20, and DuckDuckGo and Bing never challenged
at all. That last fact is the useful one — it says which engine to reach
for when several searches are needed in a row.
|
|
Brave's source name was leaking into every snippet ("Medium March 27,
2025 - ..."), because the .sitename/.netloc selectors guessed for it match
nothing. Rather than guess again, use the ordering: an engine rendering
"source / breadcrumb / title / description" puts all its metadata before
the title, so everything after the title line is the description. Class
names churn; that ordering does not. The named selectors stay as the
fallback for when the title is not on a line of its own.
Verified against Brave, and Google/DuckDuckGo/Bing re-checked for
regressions.
|
|
ctx.ui.notify is a no-op when hasUI is false, so `pi -p /search-engine`
changed the stored engine and printed nothing — the command looked broken.
Print mode now falls back to stdout. JSON mode deliberately does not: its
stdout is a structured event stream and a stray line would corrupt it.
|
|
Engine resolution mirrors the browser target: CCS_SEARCH_ENGINE, then a
/search-engine choice persisted machine-wide, then Google. A per-call
`engine` parameter sits above both so the agent can fall back when one
engine starts serving captchas — the one case where the model, not the
operator, has to make the call. Unlike the CDP target there is no safety
argument for the environment winning: driving the wrong browser means
automating someone's signed-in Chrome, choosing a different index does not.
An unsupported recency window is refused, naming the engines that support
it, rather than dropped. Silently returning unfiltered results is
indistinguishable from success, which is the failure this whole design is
trying to avoid.
Extraction grows a second mode. DuckDuckGo, Bing and Brave have clean
per-result containers; Google does not, so its heading-walk stays as its
own path rather than being bent into the item shape. DuckDuckGo and Bing
route links through redirectors, unwrapped in the page.
Third silent-failure trap found, alongside Google's two: on Bing, `count`
cancels `filters`. With ex1:"ez1" alone every result is hours old; add
count in either order and months-old results return, looking perfectly
ordinary. Bing now drops count whenever a date filter is present.
Challenge detection widened to Brave's "Verifying you're not a bot" and
"Quick check before you continue searching", which the previous Google-
shaped matcher missed entirely — found by tripping it.
|
|
Attaches to a browser that is already running — never launches one —
using the same endpoint configuration as pi-browser-harness, so a single
/browser-target choice governs both packages.
One tool, castle_cdp_search, deliberately not named web_search so it
coexists with pi-web-access rather than shadowing it.
Notes from validating against castle's Chrome:
- tbs=qdr:*, the parameter Google's own Tools menu writes, renders an
empty page on this profile; the older as_qdr=* works. Any date filter
combined with udm=14 is also empty, so recency drops udm.
- Target.createTarget must not be raced against the abort signal:
raceAbort abandons the promise but cannot cancel the command, and the
command's side effect is a tab nothing is left holding.
- A search cancelled while queued has to be removed from the semaphore
queue, or the slot handed to it later is never counted back.
- A decaying rate-limit block stops serving /sorry/ and returns an empty
results page instead, indistinguishable from a genuine zero-hit search.
|
