Write up what AI answer blocks would cost before building one

Google, Bing and Brave all put a generated answer above their results now,
and none of it is being read. Investigated against castle's Chrome; nothing
in src/ changed, because the cost turned out to be worth deciding on rather
than absorbing.

Two things make it expensive. Google's AI Overview exists only on the
classic layout — udm=14 is the "Web" tab, and having no overview is part of
what that tab is — so enabling it means giving back the clean page the
extractor was built around. And all three stream in after the results are
already parseable: Google settles ~2.3s in, Brave was still growing at 5s,
against an extractor that returns at ~0.5s. The wait is often wasted, since
the blocks are query-dependent — no Google overview for "how does postgres
MVCC vacuum work" after 15s, no Bing answer for "who founded anthropic".

Selectors, the traps behind them (#m-x-content briefly holds ~29k
characters of unrelated page; Google's "Show more" hides only the
disclaimer), the measured timings, three possible designs and their
tradeoffs are all in the doc, so this does not have to be re-measured if
the answer turns out to be yes.

1 files changed, 54 insertions, 0 deletions

diff --git a/castle-search/src/format.ts b/castle-search/src/format.ts
new file mode 100644
index 0000000..aef6c3b
--- /dev/null
+++ b/castle-search/src/format.ts
@@ -0,0 +1,54 @@
+/**
+ * Turning results into the text the model reads.
+ *
+ * Twenty results with 600-character snippets is roughly 15KB, comfortably under
+ * pi's 50KB / 2000-line budget — but the truncation is applied anyway rather
+ * than argued about, because a pathological page could blow past it and an
+ * overflowing tool result costs the whole session, not just this call.
+ */
+
+import type { SearchResult } from "./extract.ts";
+
+export type FormatInput = {
+  query: string;
+  searchUrl: string;
+  finalUrl: string;
+  endpoint: string;
+  /** Engine display name, e.g. "DuckDuckGo". */
+  engine: string;
+  /** Where the engine choice came from, e.g. "/search-engine". */
+  engineSource: string;
+  results: SearchResult[];
+};
+
+export const formatResults = (input: FormatInput): string => {
+  const lines: string[] = [
+    // Naming the engine on every result set matters more than it looks: the
+    // agent may have switched engines mid-conversation to get around a block,
+    // and results from different indexes are not interchangeable evidence.
+    `Search results for "${input.query}" — ${input.results.length} hits from ${input.engine} ` +
+      `(${input.engineSource}), via Chrome at ${input.endpoint}`,
+    `Query URL: ${input.searchUrl}`,
+  ];
+  if (input.finalUrl !== input.searchUrl) lines.push(`Landed on: ${input.finalUrl}`);
+  lines.push("");
+
+  input.results.forEach((result, index) => {
+    lines.push(`${index + 1}. ${result.title}`);
+    lines.push(`   ${result.url}`);
+    if (result.snippet) lines.push(`   ${result.snippet}`);
+    lines.push("");
+  });
+
+  return lines.join("\n").trimEnd();
+};
+
+/**
+ * The note appended when pi's truncation utilities cut the output. Says what was
+ * lost and what to do about it — a bare "[truncated]" tells the model nothing it
+ * can act on, and there is no temp file to point at because the full text only
+ * ever existed in memory.
+ */
+export const truncationNote = (shown: number, total: number, bytesShown: string, bytesTotal: string): string =>
+  `\n\n[Output truncated: ${shown} of ${total} lines (${bytesShown} of ${bytesTotal}). ` +
+  `Re-run castle_cdp_search with a smaller numResults to see the rest.]`;