diff options
| author | Ogulcan Celik <ogulcancelik@gmail.com> | 2026-09-01 14:48:45 +0300 |
|---|---|---|
| committer | Ogulcan Celik <ogulcancelik@gmail.com> | 2026-09-01 14:48:45 +0300 |
| commit | 8a6d697308162874a236c84aac0b0f8c7ac01f83 (patch) | |
| tree | ed7dc7e8c6f50d66bde721fae78576aede929f66 /website/src/content/blog | |
| parent | b54a78aa510b87d1e3b56c619b72eaeb59e4a2a1 (diff) | |
refactor: separate website presentation from runtime
Diffstat (limited to 'website/src/content/blog')
5 files changed, 0 insertions, 589 deletions
diff --git a/website/src/content/blog/coding-agents-are-becoming-runtimes.md b/website/src/content/blog/coding-agents-are-becoming-runtimes.md deleted file mode 100644 index 3628f4ec..00000000 --- a/website/src/content/blog/coding-agents-are-becoming-runtimes.md +++ /dev/null @@ -1,166 +0,0 @@ ---- -title: Coding agents are becoming runtimes -description: Coding agent CLIs are becoming developer infrastructure. Project instructions, lifecycle state, hooks, and terminal runtimes need shared contracts. -date: 2026-06-10 -draft: false ---- - -Coding agents started as chat interfaces that could edit code. Now they live inside repositories, terminals, editors, CI jobs, remote machines, mobile shells, multiplexers, and custom wrappers. They read project instructions. They run commands. They ask for permission. They spawn subagents. They pause, resume, compact, fail, recover, and keep working while the developer does something else. - -That is runtime behavior. - -The model can be proprietary. The hosted API can be proprietary. The subscription can be priced however the lab needs to price it. That part belongs to the companies building the models. - -But the CLI that runs inside a developer's terminal is part of the developer environment. It sits next to shells, language servers, package managers, test runners, terminals, editors, and multiplexers. Developers will compose it with the rest of their tools whether vendors plan for that or not. - -That is where the industry needs to be more careful. - -## We have seen this drift before - -Software already has too many places where the same idea fractured into incompatible files and conventions. - -JavaScript has `package.json`, then layers of package-manager lockfiles and workspace formats. Rust has `Cargo.toml`, which is refreshingly predictable. Python has lived through `requirements.txt`, `setup.py`, `setup.cfg`, `Pipfile`, `poetry.lock`, `pyproject.toml`, and tool-specific sections inside the same TOML file. Editors have their own settings. CI systems have their own YAML. Formatters and linters each add another little contract. - -Some of that is natural. Tools have different jobs. - -But the cost is real. Every repository becomes a map of overlapping conventions. Every new tool needs to learn which file matters, which one wins, and which one is legacy but still active. - -Agent CLIs are young enough that we can avoid repeating the worst version of that. - -Project instructions are the clearest example. [AGENTS.md](https://agents.md/) is trying to give coding agents one predictable place to read repository guidance. It describes itself as "a simple, open format for guiding coding agents" and says it is used by more than 60,000 open-source projects. - -The shape is simple: - -```text -one repository -one shared instruction file -many coding agents -``` - -Claude Code still stays on [`CLAUDE.md`, not `AGENTS.md`](https://docs.anthropic.com/en/docs/claude-code/memory#agents-md). The official workaround is to create a `CLAUDE.md` that imports `AGENTS.md`, or to symlink one file to the other. - -That workaround works, and it is how standards drift begins. - -First there is one extra file. Then every agent has a native file. Then every repository grows a compatibility layer. Then teams spend time keeping duplicated intent in sync. - -Nobody loses a production database because a repo has two instruction files. The cost is smaller and more constant: duplicated intent, stale guidance, and another convention every tool has to learn. - -## The terminal is the shared surface - -There is another debate happening around access and third-party tools. - -Coding agents made per-token pricing feel very different. A chat session can be expensive. A coding agent can run tools, read files, loop, spawn work, and burn through a lot more context. So the labs started selling subscriptions with usage limits: some explicit, some soft, some hourly, some weekly, some hard to reason about from the outside. - -Then developers started doing what developers always do. They put the model behind the tool they preferred. - -That is where the Claude ecosystem became a useful case study. People used Claude subscriptions through third-party tools such as OpenClaw and OpenCode. Anthropic pushed back. Today Anthropic documents that [Claude Code can use Pro and Max subscriptions](https://support.claude.com/en/articles/11145838-use-claude-code-with-your-pro-or-max-plan), while [Claude paid plans and API access are separate products](https://support.claude.com/en/articles/9876003-i-have-a-paid-claude-subscription-pro-max-team-or-enterprise-plans-why-do-i-have-to-pay-separately-to-use-the-claude-api-and-console). Anthropic also introduced a separate [Agent SDK credit](https://support.claude.com/en/articles/15036540-use-the-claude-agent-sdk-with-your-claude-plan) for programmatic and third-party Agent SDK usage. Zed summarized the practical effect well: first-party Claude usage and third-party agent or SDK usage now draw from different pools. - -Anthropic can make that decision. They own the API. They own the subscription terms. Compute is expensive, and a first-party interactive CLI has different economics from arbitrary programmatic tools running through an SDK or protocol adapter. - -The developer-tooling question remains: what does a developer actually own—the subscription, the API spend, the model access, or the runtime where the agent works? - -When an agent runs in a terminal, it is no longer only a hosted product experience. It is a process in a developer environment. Developers will put that process in Herdr, Zed, tmux, SSH sessions, CI jobs, containers, and scripts. They will supervise it. They will route it. They will build workflows around it. - -The terminal is where ownership gets shared. - -That does not mean vendors lose the right to build first-party experiences. It means the CLI surface needs stable contracts because it has become infrastructure other tools depend on. - -## Hooks are a symptom - -Hooks show the same tension. - -Most serious agent CLIs have hooks, plugins, or extensions now. That is good. The problem is that "hooks" can mean almost anything. - -Some hooks are policy gates. Some are notifications. Some modify tool calls. Some inject context. Some observe messages. Some run at session boundaries. Some run around tools but miss interruptions, permission resolutions, or the exact state an external supervisor needs. - -This matters because agent work is no longer a single request and response. A coding agent has a lifecycle. - -For external tools, the minimum lifecycle is small: - -```text -working -> actively doing work -blocked -> needs permission, input, recovery, or attention -idle -> ready for the next instruction -``` - -The concrete shape could be small: - -```json -{ - "session_id": "abc123", - "state": "blocked", - "reason": "permission_request", - "source": "agent" -} -``` - -Permission prompts, failed tools, interrupted generations, automated reviews, subagent handoffs, and waiting for the user all need to land somewhere coherent. - -Claude Code has a serious hook system, and Codex has its own shape too. Both are substantial, but each agent still defines its own boundary. A hook surface can be powerful and still fail to provide a portable lifecycle contract. - -OpenCode is closer to the shape orchestration tools need. Its [plugin documentation](https://opencode.ai/docs/plugins/) exposes session-level events such as `session.idle`, `session.status`, `session.error`, `permission.asked`, and `permission.replied`. - -pi, meanwhile, treats extensibility as a broader system. Its [extension documentation](https://github.com/earendil-works/pi/blob/main/packages/coding-agent/docs/extensions.md) describes TypeScript extensions that can subscribe to lifecycle events, register tools, add commands, intercept behavior, render UI, persist state, and hot-reload from local extension directories. - -Those designs are worth praising because they treat the agent as something other tools can understand. - -## What Herdr sees from below - -Herdr sits below the agent. - -It owns terminal panes. It keeps processes alive. It lets developers switch between agents, inspect them, recover sessions, and operate multiple agent processes as one workspace. - -From that layer, runtime drift becomes very visible. - -Herdr's first useful agent detection path was screen reading. If an agent drew "working", "waiting for approval", or "done" in the terminal, Herdr could read the terminal state and classify the pane. - -That matched what humans saw, which made it useful. It also broke when agent UIs changed. If the fix lived inside the Herdr binary, users needed a full Herdr update just to recognize a new prompt or status line. - -Another tempting signal was terminal activity. - -```text -screen changing -> probably working -screen still -> maybe idle or blocked -``` - -Real agents break that quickly. A spinner can redraw while the meaningful state is blocked. A permission prompt can sit still while it is the most important thing in the pane. A subagent can produce activity while the parent session is waiting. - -Terminal activity is evidence. Lifecycle is stronger. - -Herdr now uses better signals where agents expose them, keeps screen evidence as a fallback, and ships hot-reloadable detection manifests so compatibility fixes do not always require a new Herdr binary. That is the practical bridge for today's ecosystem. The durable contract should come from agents themselves. - -## Runtime support should be visible - -Herdr can support agents in two broad ways. - -Some agents expose reliable lifecycle signals. Those can have verified lifecycle support. - -Other agents require observed detection through terminal output, screen evidence, manifests, and heuristics. That can work well, but it depends on UI behavior staying recognizable. - -Herdr will publish this as a per-agent support matrix: which agents expose verified lifecycle signals, and which agents require observed detection. - -This distinction should be public because it helps everyone. - -Users get honest expectations. Agent authors get a target. Tool builders get a vocabulary. The ecosystem rewards agents that expose their runtime state cleanly. - -Agents can keep their own plugin systems while standardizing the parts that should never have been product differentiation. - -## Standardize the boring parts - -Use one predictable project instruction format. Expose a small lifecycle contract. Make permission state observable. Make interruption state observable. Tie state to the session that is actually running. Document what external tools can rely on. - -Keep competing on models, UX, planning, context management, speed, pricing, safety, and taste. - -Do not compete on whether a repository needs five instruction files. - -Do not compete on whether an external tool can tell if an agent is blocked. - -Do not make every terminal runtime reverse-engineer every other terminal runtime. - -Coding agents are becoming infrastructure. The companies building them still own their products, but developers own their environments. - -Herdr is built from that position. The terminal is where these agents actually run. The runtime surface belongs to the developer workflow, and vendors share that space whether they design for it or not. - -Agents should make their basic contracts visible, durable, and boring. - -Then the rest of us can build. diff --git a/website/src/content/blog/herdr-0-6-3.md b/website/src/content/blog/herdr-0-6-3.md deleted file mode 100644 index 3abb5f5f..00000000 --- a/website/src/content/blog/herdr-0-6-3.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -title: "Herdr 0.6.3: session persistence, safer updates, and workspace navigation" -description: "This one adds new persistence paths for updates, server restarts, and larger sessions." -date: 2026-05-27 -draft: false ---- - -## Persistence for the hard parts - -Keeping a session alive means handling several different cases: a detached client, a stopped server, terminal history replay, agent conversation restore, and an update handoff. Each protects a different part of the workspace. - -Herdr 0.6.3 improves three of those paths. - -### Agent restore after server restart - -When a Herdr server stops, normal pane processes stop with it. Native agent restore is the path for getting supported agent conversations back after that kind of restart. - -With `[session] resume_agents_on_restore = true`, Herdr can use official session references from supported integrations to relaunch Pi, Claude Code, Codex, OpenCode, and Hermes panes into their previous conversations. - -[Read the session restore docs](/docs/session-state/#native-agent-session-restore). - -### Pane screen history - -Pane screen history solves the visual half of restart recovery. When enabled, Herdr saves recent pane output and replays it into restored panes so the workspace does not come back as a set of empty terminals. - -It is useful for shells, logs, dev servers, and agent panes where recent output explains what was happening before the server stopped. - -[Read how pane history differs from live persistence](/docs/session-state/#what-survives). - -### Live handoff for updates - -Live handoff is the update path for supported running sessions. Instead of stopping the old server and killing pane processes, Herdr can ask the old server to transfer live PTY ownership to the updated server. - -It is currently experimental and opt-in with `herdr update --handoff` and `herdr --remote <host> --handoff`. That keeps the default path conservative while the feature gets more real-world coverage. - -[Read the live handoff docs](/docs/session-state/#live-handoff). - -For the technical background, read the deep dive: [Live updates without killing your terminal processes](/blog/live-updates-without-killing-your-terminal-processes/). - -## Session navigator - -Large Herdr sessions need fast movement. The new session navigator opens with `prefix+g` and gives you a searchable workspace, tab, and pane tree. - -At a glance, you can see the full tree of workspaces, tabs, and panes. Press `/` to fuzzy find by text. Press `b`, `w`, `i`, or `d` to filter by blocked, working, idle, or done panes. Press `a` or Backspace to clear the state filter. - -Use it to jump straight to the pane that needs attention without manually scanning the sidebar. - -<figure class="release-media"> - <video src="/assets/releases/v0.6.3/session-navigator.mp4" controls playsinline preload="metadata"></video> - <figcaption> - The session navigator shows workspaces, tabs, panes, and agent states in one searchable tree. - </figcaption> -</figure> - -## More terminal workflow polish - -Direct agent terminal attaches now have scrollback, so PageUp, PageDown, and mouse wheel scrolling work in the attach viewport when the running terminal app has not requested that input for itself. - -Herdr also gained better update and remote-bootstrap behavior for package-manager installs, more reliable event delivery under load, and fixes across agent detection, terminal colors, sound alerts, and integration lifecycle handling. - -## Pane metadata reporting - -The new `pane.report_metadata` socket method and `herdr pane report-metadata` CLI command let integrations and local hooks customize what Herdr shows for a pane. - -Once installed, an integration can report a title, visible agent name, compact status label, state label, and native session metadata without taking over unrelated lifecycle behavior. - -You can also use it from your own scripts. For example, a hook can mark a pane as running a deploy, rename a long-lived task, or ask an agent to report a more specific state for the work it is doing. - -```bash -herdr pane report-metadata \ - --title "api deploy" \ - --agent-name "codex" \ - --state-label "reviewing logs" \ - --compact-status "deploy" -``` - -This gives integrations a supported way to make panes more readable while keeping Herdr's session and agent ownership rules intact. - -[Read the socket API docs](/docs/socket-api/). - -## Full notes - -This page highlights the product changes. The complete release notes remain on GitHub. - -[Open the 0.6.3 GitHub release](https://github.com/herdrdev/herdr/releases/tag/v0.6.3). diff --git a/website/src/content/blog/herdr-is-joining-y-combinator.md b/website/src/content/blog/herdr-is-joining-y-combinator.md deleted file mode 100644 index 3a9cfa3e..00000000 --- a/website/src/content/blog/herdr-is-joining-y-combinator.md +++ /dev/null @@ -1,56 +0,0 @@ ---- -title: Herdr is joining Y Combinator. The runtime stays open. -description: Herdr is joining Y Combinator. The runtime is Apache-2.0 and stays open, and here is what the funding is actually for. -date: 2026-08-06 -draft: false -lede: false -ogImage: /assets/og-blog-yc-v1.png ---- - -Hey everyone, it's Can, the only person behind Herdr. This post is about how I decided to change that. - -Four months ago I was looking for another job. Thinking about where to go career-wise. Thinking about software engineering and its future in general, my hobby projects, applications, whiteboard interviews (I still can't believe we do those). And at some point it hit me: I am the bottleneck. - -This is a management and engineering problem. I don't want to install an app or learn a new tool just to manage my agents and my development style. I want something that fits me. You know how every company releases their own agent these days? No, don't do that. Let my agent integrate with your product, don't offer me another agent. - -Same idea here. I didn't like what I tried, so why not build it? So I built a runtime. But what is a runtime? - -## Runtime - -CLI coding agents run in terminals, and terminals have always been our home: editing code (yes, nvim chads), running servers, navigating projects, maintaining CI, machines, configurations. It's the root. It's the connection. So agents need first-class primitives there. - -A terminal pane belongs to an agent. A pane belongs to a tab. Tabs belong to a project. - -They need to be persistent. We have agents running for hours now, sometimes days. And once you have the runtime, where they run stops mattering. **You should be able to run them anywhere and keep them running.** - -That's where Herdr was born. But a runtime needs an interface, so I built the TUI. - -## TUI - -The TUI is the UI I use every day, and it carried Herdr this far because it shows what I believe Herdr should do: track agents at a glance, divide work between projects instead of getting lost in a pile of agents, and alert you only when an agent actually needs you. - -Its biggest advantage, and the reason it stays first-class forever, is that it's bundled. Install Herdr on your VPS, ssh in, and your UI is already there. Or run `herdr --remote user@host` and it installs itself. Depending on your network speed, you're a couple of seconds away from an agent running. One command and you're in. Nothing beats that. - -Still, the TUI was never meant to be the only client. A runtime means people can build on top of it, and people did! A [Raycast extension](https://x.com/vladscale/status/2080195067088871582), a [Stream Deck with buttons for herdr](https://x.com/timvdhoorn/status/2067907258260795808), an [iOS app](https://x.com/imnotchalk/status/2077647387414384936) that drives a whole session from a phone. More than 500 plugins, one month after the marketplace released. I didn't build any of these. - -The TUI has its limits too. Some of the ideas in my head are hard to build inside a terminal, and I trust terminals, but Herdr will need more clients. More on that later. - -## Where we are now - -So a solo project reached **25k stars and 340k downloads**, and it became more than one person can carry. - -Herdr is joining <span style="color:#fb651e">Y Combinator, F26 batch</span>. I'm excited to make Herdr a company that builds a developer tool for anyone juggling agents all day. I want to build a small team: people who keep the runtime healthy, reliable, fast, easy to run anywhere, and more extensible, so you can do more while Herdr stays small. - -The runtime, what you use right now, stays free. Apache-2.0. That's why I recently switched it from AGPL to Apache: I want everyone to use Herdr freely. - -## Where we're going - -I want to build on top of the open Herdr runtime like everyone else. I'll keep supporting the open source while building the features people really need. The demand is already visible: multiple clients, a laptop, a VPS for the six-hour job, a sandbox for risky code and ephemeral agents. Herdr can run anywhere today, but those machines are disconnected. They should be connected. - -There are many more features in my head, but I don't want to rush. People keep telling me they love that Herdr stays lean, and that's what I want to protect. In an age where adding one more feature costs nothing, choosing what goes in the core is the most important decision, I believe. So the core stays small, and everything else stays possible through extensions: your style, your flows, your company's setup, your themes, your crazy ideas. - -Thank you for getting Herdr this far. Thanks for the support, for sharing the love. You have no idea how grateful I am. - -Anyway, back to work. - - diff --git a/website/src/content/blog/live-updates-without-killing-your-terminal-processes.md b/website/src/content/blog/live-updates-without-killing-your-terminal-processes.md deleted file mode 100644 index 6658c488..00000000 --- a/website/src/content/blog/live-updates-without-killing-your-terminal-processes.md +++ /dev/null @@ -1,125 +0,0 @@ ---- -title: Live updates without killing your terminal processes -description: Herdr can replace its server during an update while keeping pane processes alive. -date: 2026-05-27 ---- - -Live handoff shipped in Herdr 0.6.3 as an experimental, opt-in update path. See the [live handoff docs](/docs/session-state/#live-handoff) for the user-facing behavior. - -Terminal multiplexers are supposed to protect long-running work from interruption. You start a server, run shells and tools inside panes, detach clients, reconnect later, and the work keeps going. - -Updating the multiplexer itself usually breaks that. - -That is normal for tools like tmux and Zellij. They are mature, stable, and their core job changes slowly. If the server/client protocol changes, the usual answer is to keep using the old server until you can restart it. That is annoying, but often acceptable. - -Herdr has a different problem. It is a terminal multiplexer, sure, but it is also an AI development workspace. It is new. I released it a few months ago, and we are at 2.6k stars on GitHub. It has more than 15k downloads according to GitHub release stats. - -It gets frequent fixes for agent detection (isn't it cool that we have all the labs building their own CLIs, plugins, and hooks in their own broken way?), terminal behavior, remote workflows, worktrees, mobile use, update flows, and integrations. Users should not have to choose between getting those fixes and killing every pane in a running session. - -Herdr needed to update like an app while preserving processes like a terminal multiplexer. - -## The update problem - -A persistent terminal multiplexer usually has at least two sides. - -The server owns the session. It owns pane state, PTYs, readers, writers, layout, buffers, sockets, and child process lifetime. The client is mostly a view and input path into that server. - -When a new binary changes the client/server protocol, the new client may not be able to talk to the old server. Restarting the server fixes that, but it also risks killing the pane processes. For a terminal multiplexer, that is the worst possible update path. - -This is not only a Herdr problem. I looked around because I assumed somebody had already solved it. - -Zellij has multiple long-running issues about upgrade pain: sessions appearing lost after upgrade, users being unable to attach to sessions from a previous version, requests for an `upgrade-session` flow, and requests to import sessions from previous versions. See [zellij#1255](https://github.com/zellij-org/zellij/issues/1255), [zellij#2943](https://github.com/zellij-org/zellij/issues/2943), [zellij#3371](https://github.com/zellij-org/zellij/issues/3371), [zellij#3420](https://github.com/zellij-org/zellij/issues/3420), [zellij#3449](https://github.com/zellij-org/zellij/issues/3449), and [zellij#4305](https://github.com/zellij-org/zellij/issues/4305). Zellij 0.44 added forward-compatible client/server support, which helps future clients connect to older running sessions, but that is still not replacing the running server while preserving live PTYs. - -Tmux has the same class of version-skew behavior. Users hit protocol mismatch errors, and the practical answer is to restart tmux, delay the update, or attach with the old binary through `/proc/<pid>/exe` where that exists. See [tmux#99](https://github.com/tmux/tmux/issues/99), [tmux#2189](https://github.com/tmux/tmux/issues/2189), [tmux#4356](https://github.com/tmux/tmux/issues/4356), and [tmux#4890](https://github.com/tmux/tmux/issues/4890). - -As far as I can tell, mainstream terminal multiplexers do not do in-place server binary replacement while keeping arbitrary live pane processes attached. - -That is a small breakthrough, but a real one. - -## The important detail is PTY ownership - -At first, the problem sounds like process migration. But no, that would be much harder. - -Herdr does not move the child processes. It moves ownership of the terminals those processes are already attached to. - -Each pane process talks to the slave side of a PTY. Herdr's server owns the master side. If the final master FD disappears, the child can lose I/O, receive HUP, or exit. But if the master FD stays alive and a new server takes over reading and writing it, the child process does not need to know the server changed. - -That changes the problem from “move a process” to “transfer open file descriptors and rebuild runtime ownership around them.” - -On Unix, that is possible with `SCM_RIGHTS`: one process can send open file descriptors to another process over a Unix socket. - -## The handoff - -The final shape is a two-server transaction. - -```text -old server: pause PTY readers -old server: snapshot pure app state -old server: duplicate PTY master FDs -old server: spawn new Herdr in import mode -handoff: send manifest + FDs over a private Unix socket -new server: restore state and rebuild pane runtimes -new server: bind public sockets and report ready -old server: commit and exit without killing pane process groups -new server: unpause readers and continue -``` - -The old server does not just dump state and hope. It enters handoff mode, rejects new mutations, disconnects clients, and pauses every transferred PTY reader. That pause acknowledgement matters because only one server may actively read from a PTY at a time. If both servers read from the same master, output can be stolen by the wrong runtime. - -After readers are paused, the old server captures Herdr's normal session snapshot. That works because Herdr already separates pure application state from runtime ownership. `AppState` can be serialized. Pane runtimes, PTY handles, readers, writers, and detector loops are separate runtime objects. - -Then the old server creates a handoff manifest and sends the duplicated PTY master FDs to the new server. - -```text -manifest = app snapshot + pane metadata + optional replay bytes -FDs = duplicated PTY master file descriptors -transport = private Unix socket + SCM_RIGHTS -``` - -The new server validates the manifest, checks the expected protocol and version, receives the FDs, restores `AppState`, constructs imported pane runtimes around the received PTY masters, binds the normal Herdr sockets, and reports ready. - -Only then does the old server commit. Before commit, any failure rolls back to the old server. After commit, the new server owns the panes and the old server exits without signaling the pane process groups. - -That commit boundary is the safety line. - -## Why Herdr could do it - -Herdr had the pieces already, mostly because of earlier architecture choices. - -The state model is separated from runtime ownership. Workspaces, tabs, panes, labels, focus, layout, and agent metadata live in serializable state. PTYs, child handles, reader threads, writer paths, resize state, and detectors live in runtime objects. - -The server/client boundary is explicit. Herdr already has a persistent server, reconnecting clients, local sockets, named sessions, remote attach flows, and documented [session state paths](/docs/session-state/). That made it possible to add a hidden import mode instead of rewriting the app around live upgrades. - -The terminal engine also helped with visual continuity. Herdr uses `libghostty-vt` for pane terminal state. For normal primary-screen panes, the old server can export recent ANSI history and the new server can seed that history before it starts reading future PTY bytes. That is not required for process survival, but it makes shells and dev servers look continuous after reconnect. - -Full-screen terminal apps are different. Herdr does not fake replay for alternate-screen panes like `nvim`, `btop`, or `htop`. Their processes survive the handoff, but the visual state is best effort until the app redraws. That is the honest trade-off. - -## The invariants - -Sending FDs was easier than making the transaction safe. - -These were the rules: - -- exactly one server may read each PTY at a time -- every transferred reader must acknowledge pause before snapshot and replay capture -- before commit, failure leaves the old server alive and owning the panes -- after commit, the old server must not destructively drop transferred runtimes -- no failure path may close the final PTY master FD -- agent restore must not run during [live handoff](/docs/session-state/#live-handoff) -- alternate-screen panes must stay live without pretending their screen was perfectly serialized - -The implementation was tested with shell panes, a Python HTTP server, `btop`, and `nvim` on Linux and macOS. The HTTP server kept serving requests across handoff. Shell panes accepted input after handoff. Full-screen TUIs stayed alive and redrew on a best-effort basis. - -## What this is not - -This is not checkpoint/restore. It does not survive a reboot. It does not migrate processes across machines. It does not preserve arbitrary process memory. It does not make every future internal state change automatically compatible. - -It is a same-machine Unix handoff. The running pane process stays attached to its PTY slave. The old Herdr server transfers the PTY master to the new Herdr server. The server binary changes; the pane process keeps running. - -## Why this is cool - -Herdr users often keep the exact kind of work open that should not be casually restarted: coding agents, dev servers, shells, REPLs, SSH sessions, debugging sessions, long builds, and local experiments. - -A terminal multiplexer should make that work safer, not make updates feel dangerous. - -The process inside the pane does not have to know an update happened. diff --git a/website/src/content/blog/ten-agents-three-clients-95-percent-less-cpu.md b/website/src/content/blog/ten-agents-three-clients-95-percent-less-cpu.md deleted file mode 100644 index 6aef6af1..00000000 --- a/website/src/content/blog/ten-agents-three-clients-95-percent-less-cpu.md +++ /dev/null @@ -1,157 +0,0 @@ ---- -title: Ten agents, three clients, 95% less CPU -description: Herdr stopped animating its sidebar, drawing hidden panes, and repainting unchanged frames during mouse movement. The busiest sessions gained the most. -date: 2026-08-03 -ogImage: /assets/og-blog-frames-v1.png -draft: false ---- - -In workloads dominated by unnecessary rendering, total CPU for the Herdr server and its attached clients fell by 89 to 95 percent. The renderer still runs at the same speed. Herdr now asks it to draw fewer frames. - -The savings grow with the session. With one agent and one client, total CPU fell 91 percent. With ten agents producing background output and three clients, it fell 95 percent. Every frame the server skips is also one less frame to serialize, transmit, and apply in each client. - -We found all three changes without a profiler. The first started as a complaint and gave us a rule: motion should mean something changed. - -## The spinner was a complaint before it was a cost - -Herdr's job is to tell you which agent needs you. You run five, ten, twenty of them, and the sidebar is how you know where to look. - -So every working agent got a spinner. It seemed obvious at the time. Motion means alive, and an agent that is thinking should look like it is thinking. - -Run twenty agents and you get twenty things moving at once, all saying the same thing, none saying it is the one that needs you. The workspace list already carried state as coloured dots. The sidebar was the noisy half of the same information. People told us. We agreed. - -The first fix isolated spinner redraws to the status rectangle. Instead of rebuilding pane contents for every tick, the server reused the client's retained frame, patched the smaller result, and sent it. We had made the spinner about as cheap as possible. Then we removed it anyway. It was a design decision first; the CPU saving was a side effect. - -<figure class="pf"> - <div class="pf-2"> - <div class="pf-panel"> - <div class="pf-label"><span>0.7.5</span><b>3 spinners, 8fps</b></div> - <div class="pf-row working"><span class="pf-mark pf-spin"><i>⠋</i><i>⠙</i><i>⠹</i><i>⠸</i></span><div><strong>herdr</strong><small>working</small></div></div> - <div class="pf-row working"><span class="pf-mark pf-spin"><i>⠋</i><i>⠙</i><i>⠹</i><i>⠸</i></span><div><strong>web-dashboard</strong><small>working</small></div></div> - <div class="pf-row blocked"><span class="pf-mark">◉</span><div><strong>llm-proxy</strong><small>blocked, needs you</small></div></div> - <div class="pf-row working"><span class="pf-mark pf-spin"><i>⠋</i><i>⠙</i><i>⠹</i><i>⠸</i></span><div><strong>data-pipeline</strong><small>working</small></div></div> - <div class="pf-row done"><span class="pf-mark">●</span><div><strong>docs</strong><small>done</small></div></div> - </div> - <div class="pf-panel"> - <div class="pf-label"><span>0.8.0</span><b>nothing moves</b></div> - <div class="pf-row working"><span class="pf-mark">●</span><div><strong>herdr</strong><small>working</small></div></div> - <div class="pf-row working"><span class="pf-mark">●</span><div><strong>web-dashboard</strong><small>working</small></div></div> - <div class="pf-row blocked"><span class="pf-mark">◉</span><div><strong>llm-proxy</strong><small>blocked, needs you</small></div></div> - <div class="pf-row working"><span class="pf-mark">●</span><div><strong>data-pipeline</strong><small>working</small></div></div> - <div class="pf-row done"><span class="pf-mark">●</span><div><strong>docs</strong><small>done</small></div></div> - </div> - </div> - <figcaption class="pf-cap">Both panels carry the same information. Only one of them lets you find the blocked agent without reading. The left panel is animating in your browser right now, which is the entire argument.</figcaption> -</figure> - -Working state is now a static coloured mark. Entering the state still renders immediately, so a change catches your eye when it happens. Remaining in the state schedules no animation. - -That design change saved more CPU than we expected. The spinner woke the server on a timer, rendered the status surface, compared and prepared the frame, serialized it, sent it to every attached client, and made each client apply it. This happened roughly eight times a second for as long as any agent was working, even when the rest of the screen stayed unchanged. - -Removing the spinner removed all of it: the timer, the animation deadlines, the scans asking whether any workspace contained a working pane, the animation-only render causes, the frame patching. Master produces zero scheduled animation frames. One agent working with nothing else on screen went from 1.467 percent CPU to 0.133 on Linux, and from 3.280 to 0.265 on macOS. - -## The same rule, applied to output you cannot see - -The same rule applies to output from panes nobody can see. - -When a pane produced output, the PTY reader set a shared flag to say something had happened. That coalesced wakeups efficiently but discarded the source pane. By the time the server picked up the flag, it could not tell whether the pane was on screen. A background agent writing to a tab nobody was viewing still produced frames for every attached client. - -<figure class="pf"> - <div class="pf-2"> - <div class="pf-panel"> - <div class="pf-label"><span>0.7.5</span><b>a pane writes</b></div> - <div class="pf-flow"> - <div class="pf-step"><b>dirty flag set</b><s>something changed, we do not know what</s></div> - <div class="pf-arrow">│</div> - <div class="pf-step"><b>render</b><s>no way to rule it out</s></div> - <div class="pf-arrow">│</div> - <div class="pf-step"><b>serialize, transmit</b><s>once per attached client</s></div> - <div class="pf-arrow">│</div> - <div class="pf-step pf-cost"><b>3 clients apply a frame</b><s>for a tab none of them are showing</s></div> - </div> - </div> - <div class="pf-panel"> - <div class="pf-label"><span>0.8.0</span><b>a pane writes</b></div> - <div class="pf-flow"> - <div class="pf-step"><b>request_pty(pane 7)</b><s>the source is kept</s></div> - <div class="pf-arrow">│</div> - <div class="pf-step"><b>can any client see pane 7?</b><s>tabs, zoom, popups, direct attaches</s></div> - <div class="pf-fork"> - <div class="pf-step pf-stop"><em>├─ no</em><b>stop here</b><s>bytes already parsed, nothing drawn</s></div> - <div class="pf-step"><em>└─ yes</em><b>render as before</b><s>visible output is still visible output</s></div> - </div> - </div> - </div> - </div> - <figcaption class="pf-cap">Render requests now carry their source. Writes from one pane coalesce into a single entry, while simultaneous writes from different panes each keep their identity.</figcaption> -</figure> - -Skipping the render does not drop output. The bytes have already been parsed, so terminal state, scrollback, cursor, OSC metadata and agent detection all stay current. Switching to that tab renders from the current state. Where the mapping is unknown, Herdr treats the pane as visible, because a wrong skip leaves a stale screen while a wrong render only costs some CPU. - -## Mouse movement should not redraw an unchanged frame - -Every mouse event reported that the view had changed. Pointer movement over a pane is forwarded to the application inside it, but Herdr's own frame is usually identical before and after. At 60 motion events per second, that meant roughly 60 server renders and 60 client frames per second, to draw the same picture sixty times. - -Herdr now tracks which of its own modes respond to hover: the global menu, context menus and the navigator. Outside those modes, movement alone repaints nothing. The events still reach the pane exactly as before. - -The benchmark counted both ends. In every measured round, on both builds, 1,680 motion packets were sent and all 1,680 arrived in the pane. Version 0.7.5 emitted around 56 to 60 frames per second doing it. Master emitted none. Linux CPU fell from 9.450 percent to 0.667. - -## Why three changes became ninety percent - -Skipping one server render also skips frame preparation, serialization, transmission, and frame handling in every attached client. - -More clients multiply the saving. Ten silent working panes with one client improved 78 percent. The same panes with three clients improved 91 percent. - -<figure class="pf"> - <div class="pf-panel"> - <div class="pf-label"><span>total CPU, server plus every attached client</span><b>linux</b></div> - <div class="pf-bars"> - <div class="pf-bar-row"> - <div class="pf-bar-name">one silent working pane</div> - <div class="pf-bar old"><span>0.7.5</span><i style="width:11.3%"></i><em>1.467%</em></div> - <div class="pf-bar new"><span>0.8.0</span><i style="width:1%"></i><em>0.133%</em></div> - </div> - <div class="pf-bar-row"> - <div class="pf-bar-name">ten silent working panes, three clients</div> - <div class="pf-bar old"><span>0.7.5</span><i style="width:33.2%"></i><em>4.316%</em></div> - <div class="pf-bar new"><span>0.8.0</span><i style="width:3.1%"></i><em>0.400%</em></div> - </div> - <div class="pf-bar-row"> - <div class="pf-bar-name">ten panes, nine hidden writers, three clients</div> - <div class="pf-bar old"><span>0.7.5</span><i style="width:100%"></i><em>13.014%</em></div> - <div class="pf-bar new"><span>0.8.0</span><i style="width:4.7%"></i><em>0.617%</em></div> - </div> - <div class="pf-bar-row"> - <div class="pf-bar-name">passive mouse motion at 60Hz</div> - <div class="pf-bar old"><span>0.7.5</span><i style="width:72.6%"></i><em>9.450%</em></div> - <div class="pf-bar new"><span>0.8.0</span><i style="width:5.1%"></i><em>0.667%</em></div> - </div> - <div class="pf-bar-row"> - <div class="pf-bar-name">forty-nine hidden writers at 60Hz</div> - <div class="pf-bar old"><span>0.7.5</span><i style="width:44.6%"></i><em>5.800%</em></div> - <div class="pf-bar new"><span>0.8.0</span><i style="width:34.1%"></i><em>4.433%</em></div> - </div> - </div> - </div> - <figcaption class="pf-cap">macOS followed the same shape: 22.958 percent to 1.673 in the hidden-output case, and 14.857 to 13.498 with forty-nine hidden writers at 60Hz.</figcaption> -</figure> - -On 0.7.5, adding hidden background output to a session cost 8.7 CPU points on Linux and 12.7 on macOS. On master the same activity costs 0.2 and 0.6, which is roughly the price of reading and parsing the bytes and nothing else. - -## Where it does not help - -Fifty panes, forty-nine of them writing at 60Hz, is about 2,940 terminal updates per second. Every one of them still has to be read from the PTY and parsed into terminal state, because that is what makes the pane correct when you switch to it. Suppressing frames cannot suppress terminal emulation. - -That case improved 23.6 percent on Linux and 9.1 on macOS. Visible output at 60Hz, in a pane you are watching, improved 33.9 percent on Linux and 2.7 on macOS. Idle sat at the measurement floor on both builds and deserves no percentage claim at all. - -The optimization targets unnecessary rendering. Output-heavy cases remain dominated by PTY reads and terminal parsing. - -## How this was measured - -We compared official 0.7.5 release binaries with hashed master builds on separate Linux and macOS machines, giving 54 isolated observations per operating system. Every run used a fresh named session, real attached clients in fixed 86 by 47 terminals, a five-second warmup and twenty-one-second samples. The figures show total CPU for the Herdr server plus every attached client. - -We paired every headline CPU result with a behavioural counter: scheduled frames per second for the sidebar, hidden-source skips for background output, and exact packet delivery counts for mouse motion. A CPU number on its own can be explained by a dozen things. A CPU number with a matching mechanism counter is harder to argue with. - -This compares a shipped release against master, so intervening changes and build environments are part of the comparison. The profiler evidence matches the mechanisms closely, but not every CPU point can be assigned to one commit. - -The work started with people saying the sidebar was too busy, along with bug reports and performance reports from people running far stranger sessions than we test. Keep them coming, and thank you to everyone helping make Herdr better. |