diff options
| author | amankumarsingh77 <amankumarsingh7702@gmail.com> | 2026-07-08 21:13:15 +0530 |
|---|---|---|
| committer | amankumarsingh77 <amankumarsingh7702@gmail.com> | 2026-07-08 21:13:15 +0530 |
| commit | c5c42df5976ed63faac2adcf3b04a4ad2f519b32 (patch) | |
| tree | 77d389a590864eaf5c2408c9bc99ed07f55343d8 /src/domains/search/extract.ts | |
| parent | 7826c950adee89512162185d07fee565ab0e96cc (diff) | |
feat: add browser_web_search tool (real-Chrome SERP scrape, no API key)
Thin query→ranked-links primitive. Isolated-attach tab lifecycle mirroring
browser_open_urls (never touches the user's current tab); pure Google SERP
parser with semantic (a h3) extraction and redirect unwrapping; CAPTCHA and
no-results surfaced as invalid_state + details.reason. Fixture-based parser
tests (9 scenarios). Establishes the isolated-tab lifecycle and tsx+fixture
test convention that browser_read_page and deep-research build on.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Diffstat (limited to 'src/domains/search/extract.ts')
| -rw-r--r-- | src/domains/search/extract.ts | 44 |
1 files changed, 44 insertions, 0 deletions
diff --git a/src/domains/search/extract.ts b/src/domains/search/extract.ts new file mode 100644 index 0000000..52f7f86 --- /dev/null +++ b/src/domains/search/extract.ts @@ -0,0 +1,44 @@ +/** + * The JavaScript source evaluated inside the Google SERP page to capture raw + * anchors + page text. This string crosses the CDP boundary via + * Runtime.evaluate (returnByValue) and returns a JSON string that + * `parseGoogleSerp` / `classifySerp` consume on the harness side. + * + * ADR-0001: anchors are selected structurally (a link carrying an <h3>), not by + * Google CSS class names, so cosmetic markup churn does not break extraction. + */ + +/** Max bytes of page text captured for CAPTCHA / no-results classification. */ +const PAGE_TEXT_LIMIT = 4000; + +/** + * Build the extraction expression. Takes no interpolated values, so it is a + * fixed source string (no injection surface). Returns + * `JSON.stringify({ anchors, pageText })`. + */ +export const buildSerpExtractionExpr = (): string => ` + (() => { + const main = document.querySelector('#search, #rso, #main') || document.body; + const headings = Array.from(main.querySelectorAll('a h3')); + const seen = new Set(); + const anchors = []; + for (const h3 of headings) { + const a = h3.closest('a[href]'); + if (!a) continue; + const href = a.href; + if (!href || seen.has(href)) continue; + seen.add(href); + const container = + a.closest('div.g') || + a.closest('div[data-hveid]') || + a.parentElement?.parentElement || + a.parentElement; + const containerText = container ? container.innerText || '' : ''; + const heading = h3.innerText || h3.textContent || ''; + const snippet = containerText.replace(heading, '').replace(/\\s+/g, ' ').trim().slice(0, 500); + anchors.push({ href, heading, snippet }); + } + const pageText = (document.body.innerText || '').slice(0, ${PAGE_TEXT_LIMIT}); + return JSON.stringify({ anchors, pageText }); + })() +`; |
