castle-search: resolve Google links lazily, in Chrome, not eagerly over plain HTTPHEADmaster

The Node-side resolver sent one curl-like request per result with no
cookies, JS or browser; Google answers that with its 'Verifying your
request' countdown after a handful, and the testing itself triggered it.
A click or direct navigation in Chrome redirects at once (measured).

Search now leaves Google links wrapped and prints the breadcrumb Google
shows for each. castle_cdp_resolve follows one wrapped link in a Chrome tab
when the agent picks it, accepts only google.com/goto?url=<token>, and
reports the verification page instead of waiting it out.

1 files changed, 15 insertions, 0 deletions

diff --git a/castle-search/src/google-links.ts b/castle-search/src/google-links.ts
new file mode 100644
index 0000000..7500e2d
--- /dev/null
+++ b/castle-search/src/google-links.ts
@@ -0,0 +1,15 @@
+/**
+ * Google wraps every outbound result in `https://www.google.com/goto?url=<opaque
+ * token>`. The token is a protobuf blob with no URL in it and the target appears
+ * nowhere in the page, so the only way to learn it is to follow the link.
+ *
+ * It is followed in Chrome, lazily, one link at a time, when the caller asks
+ * (see `resolveLink` in search.ts) — not here, and not for every result. A
+ * plain HTTP client gets Google's "Verifying your request" countdown after a
+ * handful of requests; the browser, with its own profile, gets the redirect.
+ */
+
+const GOTO = /^https?:\/\/(www\.)?google\.(?:com?\.)?[a-z]{2,3}\/goto\?url=[A-Za-z0-9_-]+$/;
+
+/** Only these may be opened by castle_cdp_resolve — it is not a general navigator. */
+export const isGoogleGoto = (url: string): boolean => GOTO.test(url);