The frame server could only capture by running a program once per frame
(grim). That works, but ~95% of a grim frame is waiting -- fork, connect,
handshake, round trip -- not encoding, and grim makes the compositor import
a *new* shared-memory buffer for every single frame.
This adds `--capture wayland[:OUTPUT]`: one long-lived zwlr-screencopy-v1
client per server, with one shm buffer reused for the whole run, capturing
inside the process. Measured here at 1280x720 (spike capture-speed, 200
frames; live re-check on Virtual-1, 60 frames):
grim (subprocess) 117.3 ms/frame 8.1 fps compositor 116.4 ms
grim-rs (Rust crate) 124.6 ms/frame 7.5 fps compositor 123.1 ms
this backend 78.0 ms/frame 13.9 fps compositor 67.5 ms
live, PNG both sides: this 128.5 ms/frame against grim's 389.3 ms (3.0x)
Three findings, each of which shaped the code:
* The win is buffer reuse, not language. grim and grim-rs both hand the
compositor a fresh buffer per frame, which is why their compositor CPU is
nearly double ours. The per-frame allocate-copy-swizzle that grim-rs pays
was measured in isolation at 2.4 ms of *client* time; the rest of its
deficit lands inside the compositor.
* ext-image-copy-capture-v1 is no faster (78.0 vs 77.7 ms) and its
create_frame path appears in a Hyprland segfault stack, so this keeps the
deprecated protocol grim uses. The switch is one constant when that crash
is fixed.
* PNG only, deliberately: PNG is the one encoding this package produces
without a dependency, so --format jpeg is refused rather than publishing a
Hello whose MIME type the frames do not have. A PNG frame is ~2.1 MB
against ~120 KB for JPEG q80, which is why the command backend stays the
default and why a libjpeg-turbo encoder (1.7 ms/frame, measured) is the
obvious follow-up.
The timeout is the load-bearing part of the client: pywayland only reads
events from read()/dispatch(block=True), neither of which takes a timeout,
and the server captures in a worker thread where SIGALRM is unavailable. It
therefore polls the display fd itself, so a compositor that stops answering
produces a CaptureError instead of a capture thread that never returns.
The protocol bindings are vendored under src/hhstream/_protocol/ exactly the
way hhstream_pointer already vendors its two, with regenerate.py recording
where each XML came from. No new dependency: pywayland was already required
by the input driver.
Contract: the new server flag and the two new module options are documented
in docs/CONTRACT.md, and docs/ARCHITECTURE.md's prediction that a native
screencopy client would be "a swap rather than a rewrite" is now the record
of what the swap cost.
Unit suite: 977 passed, 2 skipped, 0 failed (nix build -L .#checks.unit).
The live numbers were taken on this VM against the real compositor.
10 KiB
Architecture
┌──────────────────────────────────────────┐
│ hyprland-headless.service │
│ Hyprland 0.56.2, no physical monitor │
│ private XDG_RUNTIME_DIR, as `user` │
│ outputs come from `outputs[]` │
│ + one headless output per session │
└───┬───────────┬───────────────┬──────────┘
│ │ │
IPC: request / │ │ Wayland │ Wayland
event sockets │ │ session │ (virtual pointer)
│ ▼ │
│ ┌────────────────────────┴─────────┐
│ │ hyprland-headless-stream.service │
│ │ hhstream-server │
│ │ capture ──▶ one frame slot │
│ │ accept ──▶ thread per client│
│ └───────────────┬──────────────────┘
│ │ AF_UNIX, mode 0660
│ ▼
│ many clients ── hhstream-client / any reader
│
┌────────────┴──────────────┐ ┌──────────────────────────┐
│ hhstream-sessions.service │─────▶│ hhstream-pointer.service │
│ hhstreamd │ │ one Wayland connection │
│ registry, containers, │ JSON │ one virtual pointer │
│ the pointer lease │ over │ per output │
└────────────┬──────────────┘ a └──────────────────────────┘
│ private socket
│ AF_UNIX control.sock, mode 0660
▼
hhstream session ... (the CLI an agent drives)
Five pieces, deliberately separate:
- The compositor (
Hyprland, unchanged upstream) renders into virtual outputs. It knows nothing about frames on a socket. - The frame server (
hhstream) captures, holds exactly one frame, and publishes it. It owns the socket, the framing and the drop policy. - The client is any process that can open a Unix socket and read the
contract's byte format. The shipped
hhstream-clientis a convenience, not a requirement. - The session broker (
hhstreamd) starts allowlisted applications as tracked sessions, gives each one its own headless output and workspace (a container), reads the compositor back over its own IPC, and owns the pointer lease. It is the only part that keeps state across a restart. - The pointer driver (
hhstream-pointer) is a Wayland client of the same session that creates onezwlr_virtual_pointer_v1per output. It holds no lease and no session state: the broker decides, the driver executes. It is a separate process because a Wayland client needs its own connection and a driver blocked on the compositor must not stall session lifecycle.
Why containers are outputs
A session's container is a headless output, not just a workspace, and the two consequences are the reason:
grim -o hh-<id>captures exactly one session, so one agent's screenshot cannot contain another's window;zwlr_virtual_pointer_manager_v1version 2 creates a pointer bound to an output, so a warp to (0.5, 0.5) is the middle of that session's screen and not the middle of the desktop. Version 1 is refused rather than used, because its coordinates are global and every click would land somewhere plausible and wrong.
The cursor itself is singular — there is one seat — so pointer access is serialised by a broker-wide lease: one holder, a TTL the broker enforces with its own sweeper, and buttons lifted on release or expiry. The lease disciplines these sessions; it does not enforce against a hostile client, which can create its own virtual pointer (that is what sandboxing, not the lease, would fix).
Why one frame, not a queue
The server keeps a single slot holding the most recent frame. Each client thread
sends whatever is in that slot when it is ready. A client that cannot keep up
therefore misses frames instead of forcing the server to buffer them: memory use
does not grow with the number of stalled clients, and a stalled client cannot
delay the others. See docs/CONTRACT.md for exactly what a client observes.
Capture
Capturing is a separate program (by default grim, a wlr-screencopy client) run
inside the compositor's session and pointed at stdout. That choice is what keeps
hhstream free of Wayland and GPU code — the awkward part lives in a tool that
already exists and is already tested — and it is why the capture command is a
module option: tests inject a fake, and a different backend is a swap rather than
a rewrite.
That swap has now happened once. captureBackend = "wayland" captures in-process
(src/hhstream/wayland_capture.py): one long-lived zwlr-screencopy-v1 client
with one shared-memory buffer reused for every frame, instead of a grim
process per frame. Measured at 1280×720 over 200 frames, that is ~1.5× the frame
rate (78 ms against 117 ms per frame) and — the number that matters on this
machine — roughly half the compositor's CPU per frame (67 ms against 116 ms),
because grim makes the compositor import a new buffer every time. It is
deliberately the deprecated protocol: ext-image-copy-capture-v1 measured
identically and appears in a Hyprland segfault stack (create_frame →
CScreenshareFrame::transform). The price of the in-process route is that the
package now carries Wayland protocol bindings of its own (src/hhstream/_protocol/,
vendored the same way hhstream_pointer already vendors its two) and that it can
only publish PNG, since PNG is the one format it can encode without a
dependency. Both costs are why command is still the default.
Comments in the spike that produced these numbers live in the branch's
capture-speed worktree; the honest caveat that came with them is that this box
is saturated by rendering whatever is on screen, so the win is in how much room
the compositor keeps, not in headroom that did not exist.
The hardware requirement, stated plainly
Hyprland 0.56.2 asks aquamarine for its headless backend as mandatory
(src/Compositor.cpp) and does not accept the DRM backend if it is
unavailable. Aquamarine's headless backend still needs an EGL/GBM render
node; there is no software-only path that works without one. Consequently a
host with no graphics device at all (/dev/dri missing) cannot run this, and
Pyprland fails with CBackend::create() failed! before any of its managers come
up.
So the requirements are:
- a render node, i.e.
/dev/dri/renderD*— from a real GPU, a virtual GPU (virtio-gpu,virtio-gpu-gl, QXL) or a NixOS test VM's own QEMU display; - the
video/rendergroups for whichever user runs it (the module sets them).
Testing, and what each layer proves
| Layer | Proves |
|---|---|
tests/unit |
the framing, the capture wrapper and the server's client handling, with no compositor and no GPU |
checks.vm-basic |
the module actually starts both units, the socket appears with the right owner/mode, the Hello parses and two frames arrive — with a fake capture program |
checks.vm-hyprland |
the real thing: headless Hyprland comes up in the VM and grim captures a frame of the configured size |
checks.vm-sessions |
the session broker side: a real application runs as a tracked session on the compositor, the compositor's IPC sees its window, the idle reaper fires for real (and does not fire when the session is touched), and a broker restart adopts its sessions back |
checks.vm-ipc |
both IPC sockets exist at the names the contract freezes and fit in sun_path; hyprctl answers; and a subscriber on the event socket receives an openwindow for a window the test creates — the one assertion that can tell a working event socket from a silently truncated one |
checks.vm-containers |
one hh-* output per session at the container mode, the application's window on it, a second window of the same application beside the first, an unmatched window quarantined instead of landing on whoever had focus, grim -o isolating one container, and teardown removing the output and releasing its workspace slot |
checks.vm-pointer |
the driver binds a version-2 virtual pointer; acquire warps into the container and restores from the bookmark; a click arrives at the application, which reports the coordinates it was delivered at; a second acquire is BUSY naming the holder; an expired lease is swept and its held button lifted; a broker restart leaves the lock free, the bookmark intact and the driver reachable |
The three session checks share one guest (checks/lib/session-machine.nix) and
one set of test-script helpers (checks/lib/session-prelude.nix); the helpers
contain no assertions, so each check's script is still a complete account of
what it proves.
vm-hyprland is the only check that can fail for reasons outside this repo, so
it is kept separate from vm-basic on purpose: when it fails, the compositor's
log is the answer and the other two layers are already known good.
Run the VM checks one at a time. Each boots a guest with a software-rendered compositor; two at once on a two-core machine make the timing-sensitive assertions flake, and a flaky green is worth less than a slow one.