summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorIgor Soarez <igor@soarez.org>2026-10-21 12:23:28 +0100
committerIgor Soarez <igor@soarez.org>2026-10-21 12:23:28 +0100
commit32123f167dd5a7cdcbdaf6e5c144f40654d236fc (patch)
treeeb2b2585d2f7cb14b03d5a895f17a962821ade52
parent485aec099b56f2e24fa04600e85d7aba8d66f3fd (diff)
skills: pi-subagent, extracted from pi-review-panel, which now runs on it
The panel's mechanics -- headless pi runs that don't hang, answer extraction, billed cost, retries, verifying what comes back -- are not review-specific. pi-subagent covers one delegated task, one brief fanned out to several models, or a job split into parallel parts; pi-review-panel keeps only what a review adds. run-agent.sh replaces run-member.sh and fixes what it got wrong: - `timeout` is not on a Mac without coreutils, which failed every member with exit 127; it falls back to gtimeout, then perl's alarm (142 -> 124). - pi exits 0 when the provider refuses the request, or the answer is empty; both are now exit 1, with stopReason and error in the cost file. - a retry moves the earlier attempt to <name>.attempt<N>.* instead of overwriting its cost file. New options: --tools (an allowlist; read,grep,find,ls is read-only by construction), --cwd, --context-files, --brief, --deliver. status.sh replaces the `pgrep -f run-member.sh` loop, which counted every panel on the machine, and prints the cost table. From dot request 20261021-114826-work-laptop-2b868c.
-rw-r--r--skills/pi-review-panel/SKILL.md170
-rwxr-xr-xskills/pi-review-panel/run-member.sh88
-rw-r--r--skills/pi-subagent/SKILL.md209
-rwxr-xr-xskills/pi-subagent/run-agent.sh189
-rwxr-xr-xskills/pi-subagent/status.sh85
5 files changed, 530 insertions, 211 deletions
diff --git a/skills/pi-review-panel/SKILL.md b/skills/pi-review-panel/SKILL.md
index 1320ad4..f850ebb 100644
--- a/skills/pi-review-panel/SKILL.md
+++ b/skills/pi-review-panel/SKILL.md
@@ -16,17 +16,16 @@ description: >-
Review something with several external models via `pi`, each run
independently, then verify their claims and synthesize.
-The fiddly parts (hang-avoidance, cost accounting, output extraction) are
-already solved in `run-member.sh` next to this file. Use it. Don't hand-roll
-`pi` invocations — the flag set and the redirection shape are load-bearing and
-the failure modes are silent hangs. If you're curious why, the reasons are
-documented at the top of that script.
+This is a **fan-out** in the sense of the pi-subagent skill, which does the
+running: read its SKILL.md first and follow it for the workdir, launching,
+monitoring, retries, cost table and verification. `$SA` below is that skill's
+directory. What follows is only what a review panel adds.
## 0. Panel composition
The default panel is eight runs:
-| provider | model | output name |
+| provider | model | name |
|---|---|---|
| fireworks | `accounts/fireworks/models/kimi-k3` | fw-kimi-k3 |
| openrouter | `moonshotai/kimi-k3` | or-kimi-k3 |
@@ -38,145 +37,70 @@ The default panel is eight runs:
| openrouter | `z-ai/glm-5.3` | or-glm-5.3 |
If the user names specific providers/models, use exactly those instead — don't
-silently add the defaults back. Credentials are already configured in `pi`; no
-setup needed.
+silently add the defaults back. A provider this machine lacks
+(`pi --offline --list-models <model>` shows nothing for it) is dropped and
+named in the report, not substituted.
## 1. Establish what is under review
**Review what the user specified.** It is whatever they point at — a document,
a proposal, a single file, a design question, a diff. Take their framing as
-given; the panel machinery is indifferent to what is in the brief.
+given.
Only when they didn't specify, default to the diff: uncommitted changes if any
(`git status`, `git diff`), otherwise the current branch against its merge-base
with the default branch. Ask only if genuinely ambiguous.
State at the top of the final report exactly what was reviewed (commit range,
-"working tree", or the file path).
-
-**Snapshot it.** If the target is a file in a repo someone else may be editing,
-record its mtime and content hash now, and re-check before you synthesize. A
-target that changes mid-run invalidates parts of the panel's reasoning and you
-need to know that happened.
-
-## 2. Write a review brief
-
-Create a scratch directory `$P` with `$P/out/`. Write `$P/brief.md`:
-
-- One paragraph of project context (what the project is, from its README).
-- The material under review, inline in a fenced block. Inline it even if you
- also give the path — reviewers should not have to find it.
-- Absolute paths of the relevant files, so reviewers can read the surrounding
- context with their tools.
-- The **question the user is actually asking**, up front. "Find bugs in this
- diff" and "can you beat this design" produce very different reviews; the
- brief must not leave reviewers to guess which one this is.
-- The **criteria**, stated as hard requirements the reviewer must judge
- against. If the user gave criteria, use their words verbatim.
-- Reviewer instructions, matched to that question: act as a strict senior
- reviewer; look for errors, incorrect claims (especially documented numbers,
- or behavior the artifact does not actually produce), missing cases,
- unnecessary complexity; for each finding give location, severity (blocker /
- major / minor / nit), and a one-paragraph justification; **verify
- empirically before asserting** — say which interpreter/version/command
- produced each claim; end with a single verdict line; deliver the full
- markdown review as the final answer, asking no questions and offering no
- follow-up work.
-- Tell them everything they can reach is **read-only**. `pi` has edit and write
- tools and will use them; a review never needs files mutated.
-
-A useful verdict line is a fixed pair, e.g. `VERDICT: SHIP` /
-`VERDICT: DO NOT SHIP`, or for a design review `VERDICT: PROPOSAL IS BEST` /
-`VERDICT: BETTER IDEA FOUND — <name>`. Fixed strings make the tally mechanical.
+"working tree", or the file path), and snapshot it as pi-subagent §1 says.
+
+## 2. Write the review brief
+
+`$P/brief.md`, following pi-subagent §2, with these specifics:
+
+- The question up front: "find bugs in this diff" and "can you beat this
+ design" are different reviews.
+- The reviewer stance: a strict senior reviewer, looking for errors, incorrect
+ claims (especially documented numbers, or behavior the artifact does not
+ actually produce), missing cases, unnecessary complexity.
+- For each finding: location, severity (blocker / major / minor / nit), and a
+ one-paragraph justification.
+- Everything they can reach is **read-only**. Reviewers need to run things to
+ verify, so this is read-only by instruction (pi-subagent §3, level 2), not
+ `--tools`.
+- A fixed verdict pair as the last line: `VERDICT: SHIP` /
+ `VERDICT: DO NOT SHIP`, or for a design review `VERDICT: PROPOSAL IS BEST` /
+ `VERDICT: BETTER IDEA FOUND — <name>`.
## 3. Run the panel
-Launch all members concurrently. `$SK` is this skill's directory:
+All members at once, one line per row of the table:
```bash
-"$SK/run-member.sh" "$P" fireworks "accounts/fireworks/models/kimi-k3" fw-kimi-k3 > "$P/out/_fw-kimi-k3.log" 2>&1 &
-"$SK/run-member.sh" "$P" openrouter "moonshotai/kimi-k3" or-kimi-k3 > "$P/out/_or-kimi-k3.log" 2>&1 &
-"$SK/run-member.sh" "$P" fireworks "accounts/fireworks/models/glm-5p2" fw-glm-5p2 > "$P/out/_fw-glm-5p2.log" 2>&1 &
-"$SK/run-member.sh" "$P" openrouter "z-ai/glm-5.2" or-glm-5.2 > "$P/out/_or-glm-5.2.log" 2>&1 &
-"$SK/run-member.sh" "$P" fireworks "accounts/fireworks/models/deepseek-v4-flash-0731" fw-deepseek-v4 > "$P/out/_fw-deepseek-v4.log" 2>&1 &
-"$SK/run-member.sh" "$P" openrouter "deepseek/deepseek-v4-flash-0731" or-deepseek-v4 > "$P/out/_or-deepseek-v4.log" 2>&1 &
-"$SK/run-member.sh" "$P" openrouter "google/gemini-3.7-flash" or-gemini-3.7-flash > "$P/out/_or-gemini-3.7-flash.log" 2>&1 &
-"$SK/run-member.sh" "$P" openrouter "z-ai/glm-5.3" or-glm-5.3 > "$P/out/_or-glm-5.3.log" 2>&1 &
+"$SA/run-agent.sh" --deliver "the full markdown review" "$P" fireworks "accounts/fireworks/models/kimi-k3" fw-kimi-k3 > "$P/out/_fw-kimi-k3.log" 2>&1 &
+# ...and so on for each member
```
-Expect **5–30 minutes**. Poll with a loop that also watches disk, since the raw
-streams are large while runs are in flight:
+Expect 5–30 minutes; use them as pi-subagent §5 says. A member succeeded if
+its review ends with a verdict line. A provider that fails twice doesn't sink
+the panel: the same model on the other provider usually keeps the coverage.
-```bash
-while pgrep -f run-member.sh > /dev/null; do
- echo "alive=$(pgrep -f run-member.sh | wc -l) freeGB=$(df -g "$P" | tail -1 | awk '{print $4}')"
- sleep 60
-done
-```
-
-**Do the work you can do while they run.** This is the whole reason to launch
-early: don't sit on the poll loop. Verify the target yourself, build the
-measurement harness, establish the baseline. You will need your own evidence in
-step 5 regardless, and by the time the reviews land you want to be ready to
-adjudicate them, not starting from scratch.
-
-A run is good if `$P/out/<name>.md` is non-empty and ends with a verdict line.
-Retry a failure once. If it fails twice, note it in the report with its cost and
-continue — one dead provider must not sink the panel, and a duplicated model on
-the other provider means coverage usually survives.
-
-## 4. Cost accounting
-
-`run-member.sh` writes `$P/out/<name>.cost.json` per run, summed from the
-provider's own per-message `usage.cost` — actual billed figures, not estimates.
-Report them by default; the user should never have to ask what a panel cost.
-
-```bash
-jq -s -r 'sort_by(.cost)|reverse[]
- | [.name,.provider,(.exit|tostring),(.seconds|tostring),(.turns|tostring),
- (.input|tostring),(.output|tostring),(.cost*10000|round/10000|tostring)] | @tsv' \
- "$P"/out/*.cost.json | column -t
-jq -s '{runs:length, total_cost:(map(.cost)|add)}' "$P"/out/*.cost.json
-```
+## 4. Synthesize
-Include failed attempts as their own rows — a timed-out run bills for every
-token it generated and can easily be the most expensive line in the table. If a
-retry ran, keep both attempts (rename the first `<name>.attempt1.cost.json`
-before retrying, or it is overwritten).
-
-Cost per useful finding varies by two orders of magnitude across these models.
-Worth a sentence in the report when the spread is large — it informs whether
-the panel should shrink next time.
-
-## 5. Verify, then synthesize
-
-The panel's output is raw material, not findings. Reviewers state confident,
-specific, mutually contradictory things, and the disagreements are where the
-real answer usually is.
-
-- **Reproduce every substantive claim yourself** before it goes in the report.
- Models hallucinate line numbers, APIs and measurements. Where reviewers
- disagree, run the experiment that decides it and name who was right — that
- adjudication is often the most valuable part of the report.
-- Consensus is weak evidence, not proof: on a version-specific question, four
- reviewers agreeing can all be testing one version. A finding raised by one
- reviewer and confirmed by you outranks one raised by four and confirmed by
- nobody.
-- Watch for claims that are true of a variant but not of the thing actually
- written — a reviewer "verifying" a snippet they silently corrected.
-- Beware your own harness. If you extract code to test it standalone, a missing
- import can surface as a swallowed exception that looks exactly like the bug
- you're hunting. Sanity-check that the baseline still passes the cases it
- should.
+Verify first (pi-subagent §7): the reviews are raw material, and the places
+where reviewers contradict each other are where the real answer usually is.
The report:
+- What was reviewed.
- Findings deduplicated, ranked by severity, each tagged with how many members
raised it and whether **you** confirmed it, on what.
-- A "Dropped" section naming what you could not confirm and why. This is not
- optional — it's what makes the confirmed findings trustworthy.
-- The verdict tally and any disagreement worth the author's attention.
-- The cost table from step 4.
-- Paths to the per-model raw outputs. `$P` is session scratch and will not
- survive; if the findings matter, copy the reviews and any verification
- scripts somewhere durable in the project and link those instead.
+- A "Dropped" section naming what you could not confirm and why. Not optional —
+ it's what makes the confirmed findings trustworthy.
+- The verdict tally and any disagreement worth the author's attention, with who
+ was right where you settled it.
+- The cost table (`"$SA/status.sh" "$P"`), and a sentence on cost per useful
+ finding when the spread is large — it informs whether the panel should
+ shrink next time.
+- Paths to the per-model reviews, copied somewhere durable if the findings
+ matter (pi-subagent §9).
diff --git a/skills/pi-review-panel/run-member.sh b/skills/pi-review-panel/run-member.sh
deleted file mode 100755
index 7578c1e..0000000
--- a/skills/pi-review-panel/run-member.sh
+++ /dev/null
@@ -1,88 +0,0 @@
-#!/bin/bash
-# Run one pi-review-panel member.
-#
-# run-member.sh <workdir> <provider> <model> <name> [timeout_seconds]
-#
-# Reads <workdir>/brief.md, writes:
-# <workdir>/out/<name>.md the review markdown
-# <workdir>/out/<name>.cost.json tokens + USD cost, from the provider
-# <workdir>/out/<name>.jsonl filtered event stream (kept for debugging)
-# <workdir>/out/<name>.err stderr
-#
-# Exit status is pi's. 124 means the timeout fired — the run is a write-off,
-# but it still cost money; the cost file is written either way.
-#
-# Six mechanics here were each established the hard way. Do not "simplify"
-# them away:
-#
-# 1. `--offline` is required. pi's startup model-catalog fetch intermittently
-# hangs forever; --offline skips it. Symptom without it: the process
-# produces no output at all, not even the session line, and never exits.
-#
-# 2. pi's stdout must be redirected to a FILE, never piped. A pipeline
-# deadlocks: `timeout` kills pi, but the downstream reader never sees EOF
-# and hangs past its own timeout. This is why message_update filtering
-# happens after the run, not inline.
-#
-# 3. `--mode json` re-emits the entire message on every token delta, so the
-# raw stream grows quadratically — 790 MB and 3.1 GB on real runs. It is
-# filtered to the events that matter and the raw file is deleted at once.
-# Budget ~5 GB of scratch per concurrent member while runs are in flight.
-#
-# 4. Costs must be summed over every assistant message, not read off the last
-# one. Each message carries only its own usage.
-#
-# 5. pi's stdin must be /dev/null. In -p mode pi also reads stdin when it is
-# not a terminal, and waits for EOF. Started from a shell whose stdin never
-# closes — an agent's tool shell, a backgrounded job — it hangs forever
-# with no output at all, which looks exactly like the hang in (1). Found
-# 2026-09-14: two members timed out after 30 minutes having never called
-# the API.
-#
-# 6. Byte counts are stripped of whitespace before jq reads them. macOS `wc -c`
-# pads its output (" 6"), jq's tonumber rejects that, and the whole
-# cost file silently came out empty.
-
-set -u
-
-P="${1:?workdir}"; provider="${2:?provider}"; model="${3:?model}"; name="${4:?name}"
-limit="${5:-1800}"
-
-mkdir -p "$P/out"
-start=$SECONDS
-
-timeout "$limit" pi --provider "$provider" --model "$model" \
- -p --no-session -nc --no-extensions -ns --offline \
- --thinking high --mode json \
- "Read the file $P/brief.md and do exactly what it says. Deliver the full markdown review as your final answer." \
- < /dev/null > "$P/out/$name.raw" 2> "$P/out/$name.err"
-rc=$?
-elapsed=$((SECONDS - start))
-
-raw_bytes=$(wc -c < "$P/out/$name.raw" | tr -d '[:space:]')
-grep -v '"type":"message_update"' "$P/out/$name.raw" > "$P/out/$name.jsonl"
-rm -f "$P/out/$name.raw"
-
-# The review: text blocks of the last assistant message.
-jq -rs '
- [.[] | select(.type=="agent_end")] | last | .messages
- | [.[] | select(.role=="assistant")] | last
- | [.content[] | select(.type=="text") | .text] | join("")
-' "$P/out/$name.jsonl" > "$P/out/$name.md" 2>/dev/null
-
-jq -rs --arg name "$name" --arg model "$model" --arg provider "$provider" \
- --arg rc "$rc" --arg elapsed "$elapsed" --arg raw "$raw_bytes" '
- [.[] | select(.type=="message_end" and .message.role=="assistant") | .message.usage]
- | {name: $name, provider: $provider, model: $model, exit: ($rc|tonumber),
- seconds: ($elapsed|tonumber), turns: length,
- input: (map(.input) | add // 0), output: (map(.output) | add // 0),
- cacheRead: (map(.cacheRead) | add // 0),
- cacheWrite: (map(.cacheWrite) | add // 0),
- rawStreamBytes: ($raw|tonumber),
- cost: (map(.cost.total) | add // 0)}
-' "$P/out/$name.jsonl" > "$P/out/$name.cost.json" 2>/dev/null
-
-printf '%-16s exit:%-4s %5ss review:%7s bytes cost:$%s\n' \
- "$name" "$rc" "$elapsed" "$(wc -c < "$P/out/$name.md" | tr -d '[:space:]')" \
- "$(jq -r '.cost' "$P/out/$name.cost.json" 2>/dev/null)"
-exit $rc
diff --git a/skills/pi-subagent/SKILL.md b/skills/pi-subagent/SKILL.md
new file mode 100644
index 0000000..c5f1bfb
--- /dev/null
+++ b/skills/pi-subagent/SKILL.md
@@ -0,0 +1,209 @@
+---
+name: pi-subagent
+description: >-
+ Delegates work to one or more headless `pi` subagents on any provider/model,
+ run in the background, and collects each one's final answer plus its actual
+ billed cost. Covers a single delegated task, the same task fanned out to
+ several models for independent answers, or a job split into separate tasks
+ run in parallel, followed by verifying what came back. Use when the user
+ asks for subagents, to hand something to another model, to get several
+ models' takes, or to parallelize work across pi runs — and when another
+ skill (such as pi-review-panel) builds on it. Spawning costs money: never
+ start subagents on your own initiative without the user asking.
+---
+
+# pi subagents
+
+Run `pi` headless as a subagent: a fresh process, a written brief, no memory of
+this conversation, one final answer. You stay the orchestrator. Subagents
+produce raw material; you check it before it reaches the user.
+
+The fiddly parts (hang avoidance, output extraction, cost accounting, keeping
+failed attempts) are solved in `run-agent.sh` next to this file. Use it. Don't
+hand-roll `pi -p` invocations — the flag set and the redirection shape are
+load-bearing, and the failure modes are silent hangs. The reasons are at the
+top of the script. `$SK` below is this skill's directory.
+
+## 0. Shape the job
+
+Pick one of three shapes, and say which to the user if it isn't obvious:
+
+| shape | what | when |
+|---|---|---|
+| **delegate** | one task, one model | a self-contained job worth running elsewhere: a long investigation, a second opinion from a specific model, work you want off your own context |
+| **fan-out** | one brief, N models | you want independent answers to compare: reviews, designs, hard questions. Disagreement is the signal |
+| **split** | N briefs, one model each | a job that divides cleanly into parts that don't depend on each other's results |
+
+A subagent knows nothing you don't write down, can't ask you anything, and
+can't see your other subagents. If the parts of a split need each other's
+output, it's a sequence, not a split — run it in stages.
+
+**Models.** Use exactly the provider/model pairs the user names; don't add
+others. If they named none, ask — or, for a fan-out, offer the default panel in
+the pi-review-panel skill. Check a name resolves first:
+`pi --offline --list-models <search>`. Credentials are already configured in
+`pi`.
+
+## 1. Set up a workdir
+
+```bash
+P="$(mktemp -d -t pi-subagent)"; mkdir -p "$P/out"
+```
+
+`$P` is scratch. One workdir per job; several briefs can live in it
+(`$P/brief-<part>.md`), and every run writes `$P/out/<name>.*`.
+
+**Snapshot the inputs.** If the subagents will read files someone else may be
+editing — including you — record their content hash now and re-check before you
+use the answers. Input that changes mid-run invalidates parts of their
+reasoning, and you need to know that happened.
+
+## 2. Write the brief
+
+The brief is the whole interface. Write it as if for a capable contractor who
+has never seen the project and will not get to ask questions:
+
+- **The task, first sentence.** What you actually want. "Find bugs in this
+ diff" and "propose a better design than this one" produce very different
+ work; don't leave it to be guessed.
+- **Context.** One paragraph on the project (from its README) and on why this
+ task exists.
+- **The material, inline** in fenced blocks, even if you also give paths — the
+ subagent should not have to go looking for the thing it's about.
+- **Absolute paths** of relevant files and directories, so it can read the
+ surroundings with its tools.
+- **Criteria**, as hard requirements. If the user gave criteria, quote them
+ verbatim.
+- **Evidence rules.** Verify empirically before asserting, and say which
+ command, interpreter, or version produced each claim. Distinguish what was
+ run from what was reasoned.
+- **What it may touch** — see §3 — stated plainly. `pi` has edit and write
+ tools and will use them unless told, or prevented.
+- **The deliverable**: format, sections, length. End with a **fixed final
+ line** where answers will be compared — `VERDICT: SHIP` /
+ `VERDICT: DO NOT SHIP`, `ANSWER: <x>`, `STATUS: DONE` /
+ `STATUS: BLOCKED — <why>`. Fixed strings make the tally and the success check
+ mechanical.
+- **No questions, no follow-ups.** Deliver the full result as the final answer;
+ don't ask for confirmation, don't offer further work.
+
+## 3. Decide what it may touch
+
+Subagents share this machine and its filesystem. Constrain them in this order
+of preference:
+
+1. **Read-only by construction**: `--tools read,grep,find,ls`. No shell, so it
+ can't run or change anything. Right for pure reading and reasoning.
+2. **Read-only by instruction**: default tools (bash included), with the brief
+ saying everything is read-only. Needed when the subagent must run things to
+ verify claims — tests, interpreters, measurements. Tell it to put scratch
+ files under its own `$P/scratch-<name>/` and nowhere else.
+3. **Writes, isolated**: give each writing subagent its own git worktree and
+ pass it as `--cwd`; never let two writers share a tree, and never point one
+ at the user's working copy. Worktrees go under
+ `~/x/worktrees/<repo>/<name>`. You review and bring over the result; the
+ subagent doesn't commit, push or post anything — say so in the brief.
+
+Context files (AGENTS.md/CLAUDE.md) are off by default, so the brief is the
+only instruction a subagent gets. Pass `--context-files` when it works inside
+a repo whose conventions it must follow.
+
+## 4. Launch
+
+Always in the background, so you can keep working:
+
+```bash
+"$SK/run-agent.sh" "$P" <provider> "<model>" <name> > "$P/out/_<name>.log" 2>&1 &
+```
+
+Options go before the positionals: `--brief FILE` (default `$P/brief.md`),
+`--timeout SECS` (default 1800), `--thinking LEVEL` (default `high`),
+`--tools LIST`, `--cwd DIR`, `--context-files`, `--deliver TEXT`. Names must be
+unique within a workdir; a natural scheme is `<provider-abbrev>-<model>` for a
+fan-out and `<part>` for a split.
+
+A fan-out of one brief to three models:
+
+```bash
+"$SK/run-agent.sh" "$P" fireworks "accounts/fireworks/models/kimi-k3" fw-kimi-k3 > "$P/out/_fw-kimi-k3.log" 2>&1 &
+"$SK/run-agent.sh" "$P" openrouter "z-ai/glm-5.3" or-glm-5.3 > "$P/out/_or-glm-5.3.log" 2>&1 &
+"$SK/run-agent.sh" "$P" openrouter "google/gemini-3.7-flash" or-gemini > "$P/out/_or-gemini.log" 2>&1 &
+```
+
+A split, each part with its own brief:
+
+```bash
+for part in parser cli docs; do
+ "$SK/run-agent.sh" --brief "$P/brief-$part.md" "$P" openrouter "z-ai/glm-5.3" "$part" \
+ > "$P/out/_$part.log" 2>&1 &
+done
+```
+
+## 5. Monitor — and work meanwhile
+
+Expect minutes to tens of minutes per run. Check progress with:
+
+```bash
+"$SK/status.sh" "$P" # once: running names, then the table so far
+"$SK/status.sh" --wait 60 "$P" # block until all are done (exit 0)
+```
+
+`status.sh` exits 3 while anything is still running, and also prints free disk
+space — the event streams are big while runs are in flight.
+
+**Don't sit on the wait loop.** Launching early is only worth it if you use the
+time: do your own investigation of the same question, build whatever harness
+you'll need to check the answers, establish baselines. You need your own
+evidence in §7 regardless. When you do block, give the shell call a timeout
+and loop, rather than one unbounded wait.
+
+## 6. Handle failures
+
+A run succeeded if `$P/out/<name>.md` is non-empty and (when you asked for one)
+ends with the fixed final line. Exit 124 is the timeout firing. `<name>.err`
+and `<name>.jsonl` say what happened otherwise.
+
+Retry a failure once, under the same name: `run-agent.sh` moves the failed
+run's files to `<name>.attempt<N>.*` first, cost included. If it fails twice,
+report it with its cost and carry on — one dead provider must not sink the job.
+For a split, a failed part is a gap you either fill yourself or report.
+
+## 7. Verify, then use
+
+What comes back is raw material, not results:
+
+- **Reproduce every substantive claim** before passing it on. Models
+ hallucinate line numbers, APIs, file contents and measurements, with
+ confidence.
+- **Disagreement is where the answer is.** When answers conflict, run the
+ experiment that decides it and say who was right.
+- **Consensus is weak evidence.** Several models agreeing can all be making the
+ same mistake, or testing the same wrong version. One claim you confirmed
+ outranks four you didn't.
+- **Check what was actually tested.** Watch for a claim true of a variant but
+ not of the thing written — a subagent "verifying" code it silently fixed
+ first.
+- **Distrust your own harness too.** A test extracted to run standalone can
+ fail for reasons of its own (a missing import, a swallowed exception) that
+ look exactly like the bug under discussion. Check the baseline passes what it
+ should.
+- **Written work**, from a worktree: read the whole diff and run the tests
+ yourself before bringing any of it over. Never merge it unseen.
+
+Say what you dropped and why, alongside what you kept — that's what makes the
+kept part trustworthy.
+
+## 8. Report cost
+
+Every report that used subagents ends with the cost table; the user should
+never have to ask. `status.sh` prints it: per run, from the provider's own
+per-message `usage.cost` — billed figures, not estimates — failed attempts as
+their own rows, then the total. A timed-out run bills for every token it
+generated and can be the most expensive line. When cost per useful result
+varies a lot across models, say so: it informs who to use next time.
+
+## 9. Keep what matters
+
+`$P` won't survive the session. If the answers matter, copy the `.md` outputs,
+the cost files, and any verification scripts somewhere durable in the project,
+and link those instead of `$P`.
diff --git a/skills/pi-subagent/run-agent.sh b/skills/pi-subagent/run-agent.sh
new file mode 100755
index 0000000..6312eb3
--- /dev/null
+++ b/skills/pi-subagent/run-agent.sh
@@ -0,0 +1,189 @@
+#!/bin/bash
+# Run one pi subagent, headless, to completion.
+#
+# run-agent.sh [options] <workdir> <provider> <model> <name>
+#
+# Options:
+# --brief FILE task brief to hand the subagent (default: <workdir>/brief.md)
+# --timeout SECS hard wall-clock limit (default: 1800)
+# --thinking LEVEL off|minimal|low|medium|high|xhigh|max (default: high)
+# --tools LIST pi tool allowlist, e.g. read,grep,find,ls for a subagent
+# that must not run or change anything (default: pi's own)
+# --cwd DIR directory the subagent runs in (default: the current one)
+# --context-files let pi load AGENTS.md/CLAUDE.md from --cwd (default: off)
+# --deliver TEXT what the final answer must be (default: "your complete
+# result, as markdown")
+#
+# Writes, under <workdir>/out/:
+# <name>.md the subagent's final answer: text of its last message
+# <name>.cost.json tokens, seconds, exit status, USD cost from the provider
+# <name>.jsonl the event stream without per-token deltas, for debugging
+# <name>.err pi's stderr
+# <name>.running exists only while the run is in flight; holds the PID
+#
+# A previous run under the same name is never overwritten: its files are moved
+# to <name>.attempt<N>.* first, so a retry keeps the bill of the failure.
+#
+# Exit status is pi's, except that a run pi calls a success is turned into 1
+# when its answer is empty or its last message ended in a provider error: pi
+# exits 0 for both (an invalid API key, for one), and the cost file records
+# the stop reason and error. 124 means the timeout fired -- the run is a
+# write-off, but it still cost money; the cost file is written either way.
+#
+# Six mechanics here were each established the hard way. Do not "simplify"
+# them away:
+#
+# 1. `--offline` is required. pi's startup model-catalog fetch intermittently
+# hangs forever; --offline skips it. Symptom without it: the process
+# produces no output at all, not even the session line, and never exits.
+#
+# 2. pi's stdout must be redirected to a FILE, never piped. A pipeline
+# deadlocks: `timeout` kills pi, but the downstream reader never sees EOF
+# and hangs past its own timeout. This is why filtering happens after the
+# run, not inline.
+#
+# 3. The raw stream is filtered and deleted at once. Older pi re-emitted the
+# entire message on every token delta, so it grew quadratically -- 790 MB
+# and 3.1 GB on real runs. pi 1.x documents message_update as delta-only,
+# but the stream is still mostly deltas nobody reads. Budget generous
+# scratch per concurrent subagent until that is confirmed on the version
+# in use.
+#
+# 4. Costs must be summed over every assistant message, not read off the last
+# one. Each message carries only its own usage.
+#
+# 5. pi's stdin must be /dev/null. In -p mode pi also reads stdin when it is
+# not a terminal, and waits for EOF. Started from a shell whose stdin never
+# closes -- an agent's tool shell, a backgrounded job -- it hangs forever
+# with no output at all, which looks exactly like the hang in (1).
+#
+# 6. Byte counts are stripped of whitespace before jq reads them. macOS `wc -c`
+# pads its output (" 6"), jq's tonumber rejects that, and the whole
+# cost file silently came out empty.
+#
+# `timeout` is GNU coreutils and macOS has none unless coreutils is installed,
+# where it may be `gtimeout`. Without either, perl's alarm does the same job: it
+# survives exec, and SIGALRM ends pi. That exit (142) is reported as 124.
+#
+# Extensions and skills are always off (-ne -ns): an extension can block on UI
+# that a headless run has no way to answer, and the brief should say everything
+# the subagent needs to know.
+
+set -u
+
+brief=""; limit=1800; thinking=high; tools=""; cwd=""; context=""
+deliver="your complete result, as markdown"
+while [[ $# -gt 0 ]]
+do
+ case "$1" in
+ --brief) brief="$2"; shift 2 ;;
+ --timeout) limit="$2"; shift 2 ;;
+ --thinking) thinking="$2"; shift 2 ;;
+ --tools) tools="$2"; shift 2 ;;
+ --cwd) cwd="$2"; shift 2 ;;
+ --context-files) context=y; shift ;;
+ --deliver) deliver="$2"; shift 2 ;;
+ -h|--help) sed -n '2,/^$/p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
+ --) shift; break ;;
+ -*) echo "run-agent.sh: unknown option $1" >&2; exit 2 ;;
+ *) break ;;
+ esac
+done
+
+P="${1:?workdir}"; provider="${2:?provider}"; model="${3:?model}"; name="${4:?name}"
+P="$(cd "$P" && pwd)" || exit 2
+brief="${brief:-$P/brief.md}"
+[[ "$brief" == /* ]] || brief="$PWD/$brief"
+[[ -s "$brief" ]] || { echo "run-agent.sh: no brief at $brief" >&2; exit 2; }
+
+out="$P/out"
+mkdir -p "$out"
+
+# Keep any earlier attempt, cost file included.
+if [[ -e "$out/$name.cost.json" || -e "$out/$name.md" || -e "$out/$name.err" ]]
+then
+ n=1
+ while [[ -e "$out/$name.attempt$n.cost.json" || -e "$out/$name.attempt$n.err" ]]
+ do
+ n=$((n + 1))
+ done
+ for ext in md cost.json jsonl err
+ do
+ [[ -e "$out/$name.$ext" ]] && mv "$out/$name.$ext" "$out/$name.attempt$n.$ext"
+ done
+ if [[ -e "$out/$name.attempt$n.cost.json" ]]
+ then
+ tmp="$(mktemp)"
+ jq --arg a "$name.attempt$n" '.name = $a' "$out/$name.attempt$n.cost.json" > "$tmp" \
+ && mv "$tmp" "$out/$name.attempt$n.cost.json"
+ fi
+fi
+
+args=(--provider "$provider" --model "$model"
+ -p --no-session -ne -ns --offline
+ --thinking "$thinking" --mode json)
+[[ -n "$context" ]] || args+=(-nc)
+[[ -n "$tools" ]] && args+=(--tools "$tools")
+
+if command -v timeout > /dev/null; then limiter=(timeout "$limit")
+elif command -v gtimeout > /dev/null; then limiter=(gtimeout "$limit")
+else limiter=(perl -e 'alarm shift; exec @ARGV or die "exec: $!\n"' "$limit")
+fi
+
+echo $$ > "$out/$name.running"
+trap 'rm -f "$out/$name.running"' EXIT
+start=$SECONDS
+
+(
+ if [[ -n "$cwd" ]]; then cd "$cwd" || exit 2; fi
+ exec "${limiter[@]}" pi "${args[@]}" \
+ "Read the file $brief and do exactly what it says. Deliver $deliver as your final answer." \
+ < /dev/null > "$out/$name.raw" 2> "$out/$name.err"
+)
+rc=$?
+[[ $rc -eq 142 && "${limiter[0]}" == perl ]] && rc=124
+elapsed=$((SECONDS - start))
+
+raw_bytes=$(wc -c < "$out/$name.raw" | tr -d '[:space:]')
+grep -v '"type":"message_update"' "$out/$name.raw" > "$out/$name.jsonl"
+rm -f "$out/$name.raw"
+
+# The deliverable: text blocks of the last assistant message.
+jq -rs '
+ [.[] | select(.type=="agent_end")] | last | .messages
+ | [.[] | select(.role=="assistant")] | last
+ | [.content[] | select(.type=="text") | .text] | join("")
+' "$out/$name.jsonl" > "$out/$name.md" 2>/dev/null
+
+# pi exits 0 when the provider refused the request, or the model said nothing.
+stop=$(jq -rs '[.[] | select(.type=="message_end" and .message.role=="assistant")]
+ | last | .message.stopReason // "none"' "$out/$name.jsonl" 2>/dev/null)
+if [[ $rc -eq 0 ]] && { [[ "$stop" == error || "$stop" == none ]] || ! grep -q '[^[:space:]]' "$out/$name.md"; }
+then
+ rc=1
+fi
+
+jq -rs --arg name "$name" --arg model "$model" --arg provider "$provider" \
+ --arg rc "$rc" --arg elapsed "$elapsed" --arg raw "$raw_bytes" '
+ [.[] | select(.type=="message_end" and .message.role=="assistant") | .message] as $msgs
+ | ($msgs | last) as $last
+ | ($msgs | map(.usage))
+ | {name: $name, provider: $provider, model: $model, exit: ($rc|tonumber),
+ stopReason: ($last.stopReason // null),
+ error: ($last.errorMessage // null | if . then .[0:500] else . end),
+ seconds: ($elapsed|tonumber), turns: length,
+ input: (map(.input) | add // 0), output: (map(.output) | add // 0),
+ cacheRead: (map(.cacheRead) | add // 0),
+ cacheWrite: (map(.cacheWrite) | add // 0),
+ rawStreamBytes: ($raw|tonumber),
+ cost: (map(.cost.total) | add // 0)}
+' "$out/$name.jsonl" > "$out/$name.cost.json" 2>/dev/null
+
+printf '%-20s exit:%-4s stop:%-8s %5ss answer:%7s bytes cost:$%s\n' \
+ "$name" "$rc" "$stop" "$elapsed" "$(wc -c < "$out/$name.md" | tr -d '[:space:]')" \
+ "$(jq -r '.cost' "$out/$name.cost.json" 2>/dev/null)"
+if [[ $rc -ne 0 ]]
+then
+ jq -r '.error // empty' "$out/$name.cost.json" 2>/dev/null | head -c 300; echo
+fi
+exit $rc
diff --git a/skills/pi-subagent/status.sh b/skills/pi-subagent/status.sh
new file mode 100755
index 0000000..15a75de
--- /dev/null
+++ b/skills/pi-subagent/status.sh
@@ -0,0 +1,85 @@
+#!/bin/bash
+# Report on the subagents of one workdir: what is still running, and the
+# cost table of everything that finished, failed attempts included.
+#
+# status.sh <workdir> report once
+# status.sh --wait [secs] <workdir> report every <secs> (default 60) until
+# nothing is running, then the table
+#
+# Exit status: 0 when nothing is running, 3 while something still is.
+#
+# "Running" means <workdir>/out/<name>.running exists and its PID is alive. A
+# marker whose process is gone (run-agent.sh was killed) is reported as stale
+# and removed.
+
+set -u
+
+wait=""; interval=60
+if [[ "${1:-}" == "--wait" ]]
+then
+ wait=y; shift
+ if [[ "${1:-}" =~ ^[0-9]+$ ]]; then interval="$1"; shift; fi
+fi
+P="${1:?workdir}"
+out="$P/out"
+
+running() {
+ local marker pid alive=()
+ for marker in "$out"/*.running
+ do
+ [[ -e "$marker" ]] || continue
+ pid="$(cat "$marker" 2>/dev/null)"
+ if [[ -n "$pid" ]] && kill -0 "$pid" 2>/dev/null
+ then
+ alive+=("$(basename "$marker" .running)")
+ else
+ echo "stale: $(basename "$marker" .running) (process gone)" >&2
+ rm -f "$marker"
+ fi
+ done
+ echo "${alive[*]:-}"
+}
+
+free_gb() {
+ df -k "$P" | awk 'NR==2 {printf "%.1f", $4/1048576}'
+}
+
+if [[ -n "$wait" ]]
+then
+ while :
+ do
+ names="$(running)"
+ [[ -z "$names" ]] && break
+ echo "$(date +%H:%M:%S) running=$(wc -w <<< "$names" | tr -d ' ') freeGB=$(free_gb): $names"
+ sleep "$interval"
+ done
+fi
+
+names="$(running)"
+if [[ -n "$names" ]]
+then
+ echo "running: $names"
+fi
+
+shopt -s nullglob
+costs=("$out"/*.cost.json)
+if [[ ${#costs[@]} -gt 0 ]]
+then
+ {
+ echo -e "name\tprovider\tmodel\texit\tstop\tsecs\tturns\tin\tout\tanswer_bytes\tcost"
+ for f in "${costs[@]}"
+ do
+ n="$(basename "$f" .cost.json)"
+ bytes=0
+ [[ -e "$out/$n.md" ]] && bytes="$(wc -c < "$out/$n.md" | tr -d '[:space:]')"
+ jq -r --arg b "$bytes" '[.name,.provider,.model,(.exit|tostring),(.stopReason // "-"),(.seconds|tostring),
+ (.turns|tostring),(.input|tostring),(.output|tostring),$b,
+ (.cost*10000|round/10000|tostring)] | @tsv' "$f" 2>/dev/null \
+ || echo -e "$n\t?\t?\t?\t?\t?\t?\t?\t?\t$bytes\t(unreadable cost file)"
+ done | sort -t$'\t' -k11 -rn
+ } | column -t -s$'\t'
+ jq -s -r '"runs: \(length) failed: \(map(select(.exit != 0)) | length) total cost: $\(map(.cost) | add | .*10000 | round/10000)"' \
+ "${costs[@]}" 2>/dev/null
+fi
+
+[[ -z "$names" ]] || exit 3