diff options
| author | Can Celik <ogulcancelik@gmail.com> | 2026-08-09 02:04:39 +0300 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2026-08-09 02:04:39 +0300 |
| commit | 1777e9bba32b953ed1ad203b4a16d01105539000 (patch) | |
| tree | 22096973812c9baba822453d8470bdcfe64fc2a6 /docs/next/website/src/content | |
| parent | 36a8d36a7258e08b40c01bf521b1f4006525f143 (diff) | |
feat(graphics): add direct pane frame streaming (#2523)
* feat(graphics): add direct pane frame streaming
* feat(graphics): expose pane visibility
* test(graphics): preserve legacy terminal image output
* fix(graphics): harden streaming lifecycle
* fix(input): preserve mouse releases outside panes
Diffstat (limited to 'docs/next/website/src/content')
| -rw-r--r-- | docs/next/website/src/content/docs/socket-api.mdx | 41 |
1 files changed, 34 insertions, 7 deletions
diff --git a/docs/next/website/src/content/docs/socket-api.mdx b/docs/next/website/src/content/docs/socket-api.mdx index bcf0f14c..86551c03 100644 --- a/docs/next/website/src/content/docs/socket-api.mdx +++ b/docs/next/website/src/content/docs/socket-api.mdx @@ -177,10 +177,20 @@ Clients can treat `offset_from_bottom == 0` as at-bottom state. Pane graphics let a plugin place image data over a pane. They are available only when `[experimental].kitty_graphics = true`; otherwise every pane graphics -method returns `feature_disabled`. Calling `pane.graphics.info` returns the -attached client's cell width and height in pixels without creating a graphics -layer. `pane.graphics.set` accepts `png`, `rgb`, or `rgba` data in `data_base64`, -and `pane.graphics.clear` removes the layer. +method returns `feature_disabled`. Calling `pane.graphics.info` explicitly +activates capability discovery and returns the attached client's cell size, +file-frame options, pixel-mouse support, the 16-layer limit, and `pane_visible`. +`pane_visible` is true only when the target is in the active workspace and tab +and is not hidden by zoom. Short-lived UI modes do not change it. + +`pane.graphics.set`, `pane.graphics.clear`, and `pane.graphics.stream` accept an +optional `layer_id` (default `primary`). Set and stream also accept `z_index`; +layers are placed in stable `(z_index, layer_id)` order. Each stream exclusively owns its layer, and +closing it removes that layer. Inline frames accept `png`, `rgb`, `rgba`, or +`bgra`; BGRA is normalized once to owned RGBA. Herdr advances the host cache +one image transaction per render pass, so arbitrary layer sets progress without +an aggregate frame. Headless transport keeps each transaction within its 32 MiB +wire limit; local monolithic rendering does not apply that transport limit. ```json {"id":"graphics_info","method":"pane.graphics.info","params":{"pane_id":"w1:p1"}} @@ -190,14 +200,31 @@ and `pane.graphics.clear` removes the layer. For repeated frames, open a dedicated socket with `pane.graphics.stream`. After Herdr replies with `ok`, send one JSON header and then exactly `data_length` raw -bytes per frame. A stream owns that pane's graphics layer until the socket -closes; concurrent set, clear, or stream requests return `stream_conflict`. +bytes per inline frame. Concurrent operations on that layer return +`stream_conflict`. ```json -{"id":"graphics_stream","method":"pane.graphics.stream","params":{"pane_id":"w1:p1"}} +{"id":"graphics_stream","method":"pane.graphics.stream","params":{"pane_id":"w1:p1","z_index":0}} {"format":"png","image_width":800,"image_height":600,"data_length":12345,"placement":{"viewport_col":0,"viewport_row":0,"grid_cols":80,"grid_rows":30}} ``` +When `pane.graphics.info` advertises `file_frame_transport: "direct-kitty"`, an +eligible local Ghostty, kitty, or WezTerm client may submit an immutable private +`rgba` or `bgra` file with `file.path`, `sequence`, and `revision`. Direct Kitty +file transport is reserved for the default `primary` page layer; named secondary +layers use owned inline RGBA. BGRA is always copied, swizzled, and rendered +inline. Herdr replies with a `pane_graphics_frame_ack` only after the terminal +accepts the file, or after a safe owned inline fallback is installed. Confirmed +file-transport failure disables direct files for that client connection without +disabling exact pixel mouse. A timeout or client loss closes the stream without +acknowledging source reuse. Monolithic `--no-session` mode advertises neither +fast file transport nor exact pixel mouse and remains on owned inline fallback. + +Direct files are always complete canonical `width * height * 4` RGBA frames. +`file_frame_damage: true` means Herdr accepts optional damage metadata for +producer-side canonical-ring efficiency; it still copies or presents the full file. +Resize and full redraw replay placements without retransmitting pixels. + `pane.layout` returns the tab layout snapshot with `workspace_id`, `tab_id`, `zoomed`, outer `area`, `focused_pane_id`, pane rects, and split rects/ratios. `pane.neighbor` and `pane.edges` include that same layout snapshot so clients |
