diff options
| author | Igor Soarez <igor@soarez.org> | 2026-10-21 12:39:32 +0100 |
|---|---|---|
| committer | Igor Soarez <igor@soarez.org> | 2026-10-21 12:39:32 +0100 |
| commit | 9faef45275e9ab24e10bfed878f6890ba76a1bb0 (patch) | |
| tree | 0bbeb699d94186ca545e782808f09a00611a9742 | |
| parent | 32123f167dd5a7cdcbdaf6e5c144f40654d236fc (diff) | |
pi-subagent: opt-in sessions, to follow up, resume after a timeout, or fork
run-agent.sh still runs with --no-session by default. --session saves the
run as session <name> under <workdir>/sessions/; --followup TEXT sends the
next message in it, from any provider and model; --fork FROM starts <name>
as a copy of FROM. A follow-up or fork inherits --tools, --thinking, --cwd
and --context-files, so a read-only subagent stays read-only, and the
earlier round's files move to <name>.round<N>.*.
Sessions are opened by file path: by id, pi looks only among sessions
started in the current directory and otherwise silently creates an empty
one, exit 0. A fresh retry moves the old session aside, since pi would
resume it given the same id.
pi-review-panel launches members with --session, so a contradiction can be
put to both reviewers before it is settled; verdicts are still tallied from
the original reviews.
From dot request 20261021-123436-work-laptop-d0adf2.
| -rw-r--r-- | skills/pi-review-panel/SKILL.md | 21 | ||||
| -rw-r--r-- | skills/pi-subagent/SKILL.md | 64 | ||||
| -rwxr-xr-x | skills/pi-subagent/run-agent.sh | 165 |
3 files changed, 218 insertions, 32 deletions
diff --git a/skills/pi-review-panel/SKILL.md b/skills/pi-review-panel/SKILL.md index f850ebb..f50abac 100644 --- a/skills/pi-review-panel/SKILL.md +++ b/skills/pi-review-panel/SKILL.md @@ -74,10 +74,11 @@ State at the top of the final report exactly what was reviewed (commit range, ## 3. Run the panel -All members at once, one line per row of the table: +All members at once, one line per row of the table, with `--session` so that +contradictions can be put back to the reviewers in §4: ```bash -"$SA/run-agent.sh" --deliver "the full markdown review" "$P" fireworks "accounts/fireworks/models/kimi-k3" fw-kimi-k3 > "$P/out/_fw-kimi-k3.log" 2>&1 & +"$SA/run-agent.sh" --session --deliver "the full markdown review" "$P" fireworks "accounts/fireworks/models/kimi-k3" fw-kimi-k3 > "$P/out/_fw-kimi-k3.log" 2>&1 & # ...and so on for each member ``` @@ -90,6 +91,22 @@ the panel: the same model on the other provider usually keeps the coverage. Verify first (pi-subagent §7): the reviews are raw material, and the places where reviewers contradict each other are where the real answer usually is. +Where two reviewers contradict each other on something that matters, follow +each one up with the other's finding (pi-subagent §10) before you settle it: + +```bash +"$SA/run-agent.sh" --followup "Another reviewer says: <finding, with its evidence>. \ +Check it against the code and say who is right, with the command and output \ +that decides it. End with: VERDICT: <same pair as before>" \ + "$P" fireworks "accounts/fireworks/models/kimi-k3" fw-kimi-k3 +``` + +Their replies sharpen the question; they don't answer it. A reviewer giving +way is not proof — run the deciding experiment yourself either way, and tally +verdicts from the original reviews, noting any that changed. Once a member has +been followed up, its original review is `<name>.round1.md` and `<name>.md` is +the reply. + The report: - What was reviewed. diff --git a/skills/pi-subagent/SKILL.md b/skills/pi-subagent/SKILL.md index c5f1bfb..7b17e50 100644 --- a/skills/pi-subagent/SKILL.md +++ b/skills/pi-subagent/SKILL.md @@ -5,8 +5,9 @@ description: >- run in the background, and collects each one's final answer plus its actual billed cost. Covers a single delegated task, the same task fanned out to several models for independent answers, or a job split into separate tasks - run in parallel, followed by verifying what came back. Use when the user - asks for subagents, to hand something to another model, to get several + run in parallel, followed by verifying what came back; a run saved as a + session can be followed up, resumed after a timeout, or forked. Use when + the user asks for subagents, to hand something to another model, to get several models' takes, or to parallelize work across pi runs — and when another skill (such as pi-review-panel) builds on it. Spawning costs money: never start subagents on your own initiative without the user asking. @@ -118,7 +119,8 @@ Always in the background, so you can keep working: Options go before the positionals: `--brief FILE` (default `$P/brief.md`), `--timeout SECS` (default 1800), `--thinking LEVEL` (default `high`), -`--tools LIST`, `--cwd DIR`, `--context-files`, `--deliver TEXT`. Names must be +`--tools LIST`, `--cwd DIR`, `--context-files`, `--deliver TEXT`, and +`--session` to keep the conversation for a follow-up (§10). Names must be unique within a workdir; a natural scheme is `<provider-abbrev>-<model>` for a fan-out and `<part>` for a split. @@ -164,8 +166,11 @@ ends with the fixed final line. Exit 124 is the timeout firing. `<name>.err` and `<name>.jsonl` say what happened otherwise. Retry a failure once, under the same name: `run-agent.sh` moves the failed -run's files to `<name>.attempt<N>.*` first, cost included. If it fails twice, -report it with its cost and carry on — one dead provider must not sink the job. +run's files to `<name>.attempt<N>.*` first, cost included. A run started with +`--session` that timed out after real progress is often better continued than +retried: `--followup` it (§10), and it picks up with everything it had read +and run. If it fails twice, report it with its cost and carry on — one dead +provider must not sink the job. For a split, a failed part is a gap you either fill yourself or report. ## 7. Verify, then use @@ -176,7 +181,11 @@ What comes back is raw material, not results: hallucinate line numbers, APIs, file contents and measurements, with confidence. - **Disagreement is where the answer is.** When answers conflict, run the - experiment that decides it and say who was right. + experiment that decides it and say who was right. With sessions (§10) you + can also put each side's claim to the other first — but a subagent + conceding or holding firm is not evidence either; the experiment is. +- **Ask for the evidence.** With a session, a vague or surprising claim can be + followed up: "show the exact command and output behind X". - **Consensus is weak evidence.** Several models agreeing can all be making the same mistake, or testing the same wrong version. One claim you confirmed outranks four you didn't. @@ -207,3 +216,46 @@ varies a lot across models, say so: it informs who to use next time. `$P` won't survive the session. If the answers matter, copy the `.md` outputs, the cost files, and any verification scripts somewhere durable in the project, and link those instead of `$P`. + +## 10. Follow up, resume, fork + +Off by default: a run saves nothing, and once it ends it can't be talked to +again. Pass `--session` at launch when you expect to come back to it — a long +run that may time out, a fan-out whose disagreements you'll want to put back +to the subagents, anything you may need to question. It costs nothing extra +until used. The session is kept as `$P/sessions/<timestamp>_<name>.jsonl`. + +Then, once that run has finished: + +```bash +# Continue the same conversation. Provider and model may differ from before. +"$SK/run-agent.sh" --followup "<message>" "$P" <provider> "<model>" <name> + +# Branch it into a new name; the original session is left as it was. +"$SK/run-agent.sh" --fork <name> --followup "<message>" "$P" <provider> "<model>" <new-name> +``` + +- **Uses.** Continue a run that timed out ("You were cut off. Carry on from + where you stopped and deliver the full result"). Ask for the evidence behind + a claim. Put another subagent's contradicting finding to it ("Another + reviewer says X, because Y. Check it and say who is right, with evidence"). + Fork to ask several different questions of the same context without them + seeing each other, or to hand a whole investigation to a different model — + it reads the full history, whoever produced it. +- **The message is sent as written.** No brief, no `--deliver` wrapper, so say + what to deliver and end with the same fixed final line if you'll tally it. +- **What carries over.** A follow-up or fork inherits `--tools`, `--thinking`, + `--cwd` and `--context-files` from the run it continues, so a read-only + subagent stays read-only; pass one again to change it. +- **Outputs.** The earlier round's files move to `<name>.round<N>.*`, so every + answer is kept and every round is its own row in the cost table. Each + follow-up resends the whole conversation as input: a short question to a long + session is not cheap. +- **Limits.** A run killed mid-step loses that step; everything before it is + in the session. `run-agent.sh` refuses to follow up or fork a run still in + flight. A fresh retry under the same name (no `--followup`) starts a new + conversation and moves the old session aside. +- **Don't hand-roll the resume.** By id, pi looks only among sessions started + in the current directory, and where it finds none it silently starts a new, + empty one under that id — exit 0, and an answer from a model that remembers + nothing. `run-agent.sh` opens the session by its file path instead. diff --git a/skills/pi-subagent/run-agent.sh b/skills/pi-subagent/run-agent.sh index 6312eb3..aaf6770 100755 --- a/skills/pi-subagent/run-agent.sh +++ b/skills/pi-subagent/run-agent.sh @@ -14,6 +14,22 @@ # --deliver TEXT what the final answer must be (default: "your complete # result, as markdown") # +# Sessions -- off by default; a run without --session cannot be continued: +# --session save the conversation as session <name>, under +# <workdir>/sessions/, so it can be followed up or forked +# --followup TEXT send TEXT as the next message in session <name>, instead +# of the brief. <provider> <model> may differ from the +# earlier run's: the new model reads the whole history +# --fork FROM with --followup: start session <name> as a copy of +# session FROM and send TEXT there. FROM is left as it was +# +# A follow-up or fork inherits --tools, --thinking, --cwd and --context-files +# from the run it continues, unless given again: a read-only subagent stays +# read-only. The session is opened by file path, never by id -- by id, pi +# looks only among sessions started in the current directory, and where it +# finds none it starts a new, empty one with that id, exit status 0, and only +# a warning on stderr to say so. +# # Writes, under <workdir>/out/: # <name>.md the subagent's final answer: text of its last message # <name>.cost.json tokens, seconds, exit status, USD cost from the provider @@ -22,7 +38,9 @@ # <name>.running exists only while the run is in flight; holds the PID # # A previous run under the same name is never overwritten: its files are moved -# to <name>.attempt<N>.* first, so a retry keeps the bill of the failure. +# to <name>.attempt<N>.* first, so a retry keeps the bill of the failure -- or +# to <name>.round<N>.* when the new run is a follow-up of it. A retry also +# moves the old session aside, so it starts fresh instead of resuming it. # # Exit status is pi's, except that a run pi calls a success is turned into 1 # when its answer is empty or its last message ended in a provider error: pi @@ -71,18 +89,22 @@ set -u -brief=""; limit=1800; thinking=high; tools=""; cwd=""; context="" +brief=""; limit=""; thinking=""; tools=""; cwd=""; context="" deliver="your complete result, as markdown" +session=""; followup=""; fork_from=""; set_tools=""; set_context="" while [[ $# -gt 0 ]] do case "$1" in --brief) brief="$2"; shift 2 ;; --timeout) limit="$2"; shift 2 ;; --thinking) thinking="$2"; shift 2 ;; - --tools) tools="$2"; shift 2 ;; + --tools) tools="$2"; set_tools=y; shift 2 ;; --cwd) cwd="$2"; shift 2 ;; - --context-files) context=y; shift ;; + --context-files) context=y; set_context=y; shift ;; --deliver) deliver="$2"; shift 2 ;; + --session) session=y; shift ;; + --followup) followup="$2"; shift 2 ;; + --fork) fork_from="$2"; shift 2 ;; -h|--help) sed -n '2,/^$/p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;; --) shift; break ;; -*) echo "run-agent.sh: unknown option $1" >&2; exit 2 ;; @@ -92,36 +114,120 @@ done P="${1:?workdir}"; provider="${2:?provider}"; model="${3:?model}"; name="${4:?name}" P="$(cd "$P" && pwd)" || exit 2 -brief="${brief:-$P/brief.md}" -[[ "$brief" == /* ]] || brief="$PWD/$brief" -[[ -s "$brief" ]] || { echo "run-agent.sh: no brief at $brief" >&2; exit 2; } - out="$P/out" +sessions="$P/sessions" mkdir -p "$out" -# Keep any earlier attempt, cost file included. -if [[ -e "$out/$name.cost.json" || -e "$out/$name.md" || -e "$out/$name.err" ]] -then - n=1 - while [[ -e "$out/$name.attempt$n.cost.json" || -e "$out/$name.attempt$n.err" ]] +die() { echo "run-agent.sh: $*" >&2; exit 2; } + +# The file of session $1 in $sessions, if there is one. pi names it +# <timestamp>_<id>.jsonl; the id in its header line is what decides. +session_file() { + local f + for f in "$sessions"/*_"$1".jsonl + do + [[ -e "$f" ]] || continue + if [[ "$(head -1 "$f" | jq -r '.id // empty' 2>/dev/null)" == "$1" ]] + then + echo "$f"; return 0 + fi + done + return 1 +} + +is_running() { + local pid + pid="$(cat "$out/$1.running" 2>/dev/null)" && [[ -n "$pid" ]] && kill -0 "$pid" 2>/dev/null +} + +# Move run $name's files to $name.$1<N>.*; $1 is "attempt" or "round". +rotate() { + local kind="$1" n=1 ext f + [[ -e "$out/$name.cost.json" || -e "$out/$name.md" || -e "$out/$name.err" ]] || return 0 + while [[ -e "$out/$name.$kind$n.cost.json" || -e "$out/$name.$kind$n.err" ]] do n=$((n + 1)) done for ext in md cost.json jsonl err do - [[ -e "$out/$name.$ext" ]] && mv "$out/$name.$ext" "$out/$name.attempt$n.$ext" + [[ -e "$out/$name.$ext" ]] && mv "$out/$name.$ext" "$out/$name.$kind$n.$ext" done - if [[ -e "$out/$name.attempt$n.cost.json" ]] + if [[ -e "$out/$name.$kind$n.cost.json" ]] then - tmp="$(mktemp)" - jq --arg a "$name.attempt$n" '.name = $a' "$out/$name.attempt$n.cost.json" > "$tmp" \ - && mv "$tmp" "$out/$name.attempt$n.cost.json" + f="$(mktemp)" + jq --arg a "$name.$kind$n" '.name = $a' "$out/$name.$kind$n.cost.json" > "$f" \ + && mv "$f" "$out/$name.$kind$n.cost.json" fi + # A fresh run must not resume the old conversation: pi would, given its id. + if [[ "$kind" == attempt ]] && f="$(session_file "$name")" + then + mv "$f" "$f.attempt$n" + fi +} + +is_running "$name" && die "$name is still running" +[[ -n "$fork_from" && -z "$followup" ]] && die "--fork needs --followup" +if [[ -n "$session$followup" ]] && ! [[ "$name" =~ ^[A-Za-z0-9]([A-Za-z0-9._-]*[A-Za-z0-9])?$ ]] +then + die "$name cannot be a session id: letters, digits, . _ - only, starting and ending with a letter or digit" fi +mode=fresh; src=""; src_name="" +if [[ -n "$followup" ]] +then + if [[ -n "$fork_from" ]] + then + mode=fork; src_name="$fork_from" + is_running "$fork_from" && die "$fork_from is still running; fork it once it is done" + if session_file "$name" > /dev/null || [[ -e "$out/$name.cost.json" ]] + then + die "$name already exists; fork into a new name" + fi + else + mode=followup; src_name="$name" + fi + src="$(session_file "$src_name")" \ + || die "no session $src_name in $sessions -- only a run started with --session can be continued" + # Inherit what the earlier run was given, unless overridden. + prev="$out/$src_name.cost.json" + if [[ -e "$prev" ]] + then + [[ -n "$set_tools" ]] || tools="$(jq -r '.tools // empty' "$prev")" + [[ -n "$set_context" ]] || context="$(jq -r 'if .contextFiles then "y" else empty end' "$prev")" + [[ -n "$thinking" ]] || thinking="$(jq -r '.thinking // empty' "$prev")" + [[ -n "$cwd" ]] || cwd="$(jq -r '.cwd // empty' "$prev")" + fi + [[ -n "$cwd" ]] || cwd="$(head -1 "$src" | jq -r '.cwd // empty')" +else + brief="${brief:-$P/brief.md}" + [[ "$brief" == /* ]] || brief="$PWD/$brief" + [[ -s "$brief" ]] || die "no brief at $brief" +fi +thinking="${thinking:-high}"; limit="${limit:-1800}" +cwd="$(cd "${cwd:-.}" && pwd)" || die "no directory $cwd" + +if [[ $mode == followup ]]; then rotate round; else rotate attempt; fi + args=(--provider "$provider" --model "$model" - -p --no-session -ne -ns --offline + -p -ne -ns --offline --thinking "$thinking" --mode json) +case $mode in + fresh) + if [[ -n "$session" ]] + then + mkdir -p "$sessions" + args+=(--session-dir "$sessions" --session-id "$name") + else + args+=(--no-session) + fi + prompt="Read the file $brief and do exactly what it says. Deliver $deliver as your final answer." ;; + followup) + args+=(--session "$src") + prompt="$followup" ;; + fork) + args+=(--fork "$src" --session-dir "$sessions" --session-id "$name") + prompt="$followup" ;; +esac [[ -n "$context" ]] || args+=(-nc) [[ -n "$tools" ]] && args+=(--tools "$tools") @@ -135,9 +241,8 @@ trap 'rm -f "$out/$name.running"' EXIT start=$SECONDS ( - if [[ -n "$cwd" ]]; then cd "$cwd" || exit 2; fi - exec "${limiter[@]}" pi "${args[@]}" \ - "Read the file $brief and do exactly what it says. Deliver $deliver as your final answer." \ + cd "$cwd" || exit 2 + exec "${limiter[@]}" pi "${args[@]}" "$prompt" \ < /dev/null > "$out/$name.raw" 2> "$out/$name.err" ) rc=$? @@ -163,8 +268,16 @@ then rc=1 fi +session_path="" +if [[ $mode == followup ]]; then session_path="$src" +elif [[ $mode == fork || -n "$session" ]]; then session_path="$(session_file "$name")" +fi + jq -rs --arg name "$name" --arg model "$model" --arg provider "$provider" \ - --arg rc "$rc" --arg elapsed "$elapsed" --arg raw "$raw_bytes" ' + --arg rc "$rc" --arg elapsed "$elapsed" --arg raw "$raw_bytes" \ + --arg mode "$mode" --arg from "$src_name" --arg session "$session_path" \ + --arg tools "$tools" --arg thinking "$thinking" --arg cwd "$cwd" \ + --arg context "$context" ' [.[] | select(.type=="message_end" and .message.role=="assistant") | .message] as $msgs | ($msgs | last) as $last | ($msgs | map(.usage)) @@ -176,7 +289,11 @@ jq -rs --arg name "$name" --arg model "$model" --arg provider "$provider" \ cacheRead: (map(.cacheRead) | add // 0), cacheWrite: (map(.cacheWrite) | add // 0), rawStreamBytes: ($raw|tonumber), - cost: (map(.cost.total) | add // 0)} + cost: (map(.cost.total) | add // 0), + mode: $mode, from: (if $from == "" then null else $from end), + session: (if $session == "" then null else $session end), + tools: (if $tools == "" then null else $tools end), + thinking: $thinking, cwd: $cwd, contextFiles: ($context != "")} ' "$out/$name.jsonl" > "$out/$name.cost.json" 2>/dev/null printf '%-20s exit:%-4s stop:%-8s %5ss answer:%7s bytes cost:$%s\n' \ |
