Files
hyprland-headless-stream-pl…/spikes/capture-speed
Hermenix bf0667303e spike: --resume for the runner, and run the matrix from a transient unit
Two attempts were reaped mid-run by agent lifecycle cleanup (exit 143 with no
error line); a nix-shell process tree of tens of minutes belongs in a systemd
transient unit, not an agent background process. --resume skips any cell whose
result file already exists, so a kill costs the cell in flight and nothing else.
2026-10-11 16:25:38 +00:00
..

capture-speed — can we drop grim, and what would it buy?

Question (the operator's): which approach renders images faster for live streaming the workspaces to the web and for agent eyes — the shipped grim subprocess, the Rust grim-rs crate, or a raw pywayland client? And, since speed alone does not decide it: what does each cost in RAM and CPU?

Answer, short version: the capture is not the expensive part — the round trip is. In-process acquisition roughly doubles the frame rate and divides the producer's own CPU by four, but on this machine both routes push the box to saturation, because the compositor is software-rendering an animating screen. Numbers, method and the bugs found are below.

Method

Everything is measured through the frozen contract in HARNESS.md, by bench.py, on one target so the cells are comparable:

  • Target: a Chromium session (--app=http://127.0.0.1/spin.html, a CSS-animated page — motion with no JavaScript and no timers, so the screen really changes and a damage-driven capture cannot go quiet and flatter a route). Pixels verified to differ between frames before any run.
  • Cells: one session container (1280×720) and the whole desktop stitched (2560×720) × raw and JPEG q80 × two runs each.
  • Baseline: grim as it ships — one process per frame, fork and encode inside the timed span.
  • Resources: producer CPU/RSS (getrusage), machine CPU (/proc/stat), the compositor's own CPU (/proc/<Hyprland>/stat), plus an idle control sample taken with nothing capturing at all — without it a per-frame CPU number would silently include the animation's cost and be attributed to the capture.
  • No GPU on this machine. Hyprland renders through software (kms_swrast/llvmpipe), so no route was tuned to a dma-buf path and none is claimed. Each producer says in its own result which protocol actually served its frames.

Where the time actually goes (the finding that reframes it)

grim at 1280×720 JPEG q80: p50 119 ms per frame, 7.9–8.3 fps, but only 6–8 ms of CPU. About 95% of each frame is spent waiting — on the fork, the connect, the compositor's round trip — not encoding. Two consequences:

  1. Attacking the encoder is pointless: JPEG q80 of a 1280×720 frame costs 1.7 ms in-process.
  2. The win available is in not paying the round trip per frame — which is exactly what a persistent in-process client does, and why the two new routes land so close to each other.

And the resource measurement adds the part speed hides: at 1280×720 with motion on screen, the machine is already saturated (≈199% of one core, i.e. both cores) whichever route is used, because the compositor is repainting a software-rendered animated page. The capture route changes the compositor's per-frame CPU materially (see the table), but it cannot make the box idle.

Head-to-head

Bugs and traps found on the way (each cost real time)

  1. grim -o NAME must come before the trailing -. The other order exits 0 and writes 637 bytes of nothing to the pipe. Silent, and it cost the first harness build.
  2. grim's only uncompressed format is ppm (pbm/pnm/rgb/bmp are rejected), which is what makes the encode/capture split measurable.
  3. display.dispatch(block=False) never reads the socket — it processes only events already read. A wait loop built on it plus select times out against a perfectly healthy compositor. The read is dispatch(block=True); the "never hang" guarantee has to come from an alarm (SIGALRM), not from polling.
  4. pywayland dispatchers receive the proxy as their first argument (four parameters for a three-field event), and display.disconnect() is mandatory or the process segfaults after printing — a green run that reports exit 139 to any runner reading exit codes.
  5. turbojpeg reuses *jpegBuf when it is non-NULL. Freeing it with tjFree and leaving the pointer in place makes the next call write into freed memory: a heap-corrupting segfault several frames later with no stack trace near the encoder. Reset the pointer to NULL after freeing.
  6. libjpeg_turbo's bare attribute resolves to its -bin output in that Nix shell — no shared library in it — and wayland.xml lives in the wayland-scanner package, not in wayland. Both failures look like missing dependencies rather than shell gaps.
  7. ext-image-capture-source-v1 defines one manager per source kind (ext_output_… and ext_foreign_toplevel_…), not a generic ext_image_capture_source_manager_v1. Looking for the generic name finds nothing and reads as "the modern protocol is unavailable here" — a wrong conclusion this spike drew once before checking the real global list. The toplevel variant means single-window capture is available on this compositor, which matters for agent eyes.

Files

File What it is
HARNESS.md the frozen measurement contract (argv, JSON, timing boundary, resource fields)
bench.py the runner: every route through every cell, raw JSON kept, table derived from the files
producers/grim_producer.py route 000, the baseline
producers/wayland_producer.py route 001b, raw pywayland (screencopy and ext-image-copy-capture)
producers/resources.py RAM/CPU sampling shared by the Python producers
001a-grim-rs/ route 001a, the Rust grim-rs producer
results/ every run's stdout/stderr, results.json, and control.json
shell.nix the spike's own environment (pywayland, turbojpeg, rust) — deliberately not in the repo flake

Verdict