Files
hyprland-headless-stream-pl…/spikes/capture-speed/README.md
T
Hermenix 36efa316fd spike: the verdict, the table, the chart -- in-process pywayland wins on zwlr-screencopy
Four routes measured on one frozen contract at one session's container size, with
RAM and CPU as first-class results: in-process pywayland is ~1.5x faster (78 vs
117 ms), uses ~2.8x less of our own CPU, and nearly halves the compositor's
per-frame CPU (67 vs 116 ms). grim-rs buys nothing and costs 746 MB RSS.
ext-image-copy-capture measures identical to screencopy and appears in a
compositor crash stack, so the deprecated-but-working protocol is the pick.

Also fixes the client's unclean exit (pywayland teardown segfaulted after the
result was printed, 16 times, one batch coinciding with a compositor crash).
2026-10-11 17:08:13 +00:00

9.8 KiB
Raw Blame History

capture-speed — can we drop grim, and what would it buy?

Question (the operator's): which capture route renders images faster for live streaming the workspaces to the web and for agent eyes — the shipped grim subprocess, the Rust grim-rs crate, or a raw pywayland client? And what does each cost in RAM and CPU?

Answer: in-process wins, decisively and cheaply — ~1.5× the frame rate, ~2.8× less of our own CPU, and ~42% less CPU inside the compositor — and neither Rust nor the newer capture protocol is needed to get it. grim-rs is no faster than the subprocess it would replace and costs 746 MB of RSS to grim's 18 MB.

Method

One frozen contract (HARNESS.md), one runner (bench.py), one target per pass.

  • Target: a Chromium session on a CSS-animated page (no JavaScript, no timers) so the screen genuinely changes; frames verified to differ between captures before any run. The still half of the contract lives in results/still/ and results/final-still/ (partial — see "What went wrong").
  • Cells: one session's container at 1280×720, raw and jpeg q80, 200 measured frames per run, 2 runs each (the contract asks 300; 200 is stated here rather than hidden).
  • Baseline: grim exactly as it ships — one process per frame, fork and encode inside the timed span.
  • Resources: producer CPU/RSS (getrusage), machine CPU (/proc/stat), the compositor's own CPU (/proc/<Hyprland>/stat), and an idle control sample with nothing capturing, so an animating page's cost is not attributed to a route.
  • No GPU: Hyprland renders through software (kms_swrast/llvmpipe). No route was tuned to a dma-buf path, and none is claimed. Each producer reports which protocol actually served its frames.

Head-to-head — 1280×720, moving screen, one session, sessions run sequentially

route mode p50 ms fps own CPU ms machine CPU ms compositor CPU ms machine % RSS MB
grim (subprocess, ships today) jpeg 117.3 8.1 5.03 247.9 116.4 199.1 18.5
grim raw 121.4 7.9 3.86 255.2 118.4 199.4 20.5
grim-rs (Rust crate) jpeg 124.6 7.5 6.60 266.7 123.1 199.2 746.3
grim-rs raw 120.1 7.8 1.59 254.1 119.2 198.9 746.1
pywayland screencopy jpeg 78.0 13.9 1.78 143.0 67.5 199.1 29.3
pywayland screencopy raw 76.7 14.2 0.33 140.4 67.1 198.7 24.4
pywayland ext-image-copy-capture jpeg 77.7 14.0 1.79 142.2 67.2 199.3 29.1
pywayland ext-image-copy-capture raw 73.5 14.4 0.31 138.5 66.2 198.9 24.5

Idle control — nothing capturing at all: 199.1% of one core, the compositor alone at 94.1%. Frames are 105–130 KB as JPEG, 2.76 MB as raw PPM.

What the table says

  1. In-process is ~1.5× faster and much cheaper: 78 ms vs 117 ms, 13.9 fps vs 8.1, our own CPU 1.8 ms vs 5.0 ms per frame.
  2. The win that matters is the compositor's, not ours: 67 ms vs 116 ms of compositor CPU per frame. On a 2-core box that is the difference between the desktop being starved and merely busy.
  3. grim-rs is not a candidate: slightly slower than the subprocess (124.6 vs 117.3 ms), higher own CPU (6.6 vs 5.0 ms), and 746 MB RSS — 40× grim. Its own notes explain the speed: it allocates a fresh shm buffer and copies into a Vec per capture. The memory figure is reproducible across runs.
  4. screencopy vs ext-image-copy-capture: no measurable difference (78.0 vs 77.7 ms; compositor 67.5 vs 67.2 ms). The newer protocol buys nothing here, and its frame-creation path is where one compositor crash landed — so the recommendation is the older, deprecated-but-working protocol, which is also what grim uses (apples to apples).
  5. The box is saturated in every cell (≈199% of one core) — and the idle control is also 199%. Rendering the animating screen, not capturing it, is what fills this machine. No capture change fixes that; what changes is how much room the compositor has left.

Where the time actually goes (the finding that reframes it)

grim at 1280×720 JPEG q80 spends ~95% of each frame waiting — on the fork, the connect, the compositor's round trip — not encoding: JPEG q80 of a 1280×720 frame costs 1.7 ms in-process. So the available win is in not paying the round trip per frame, which is exactly what a persistent in-process client does — and why screencopy and ext-image-copy-capture land within noise of each other.

What went wrong (each item cost real time; none is hidden)

  • The compositor crashed three times during measurement (.Hyprland-wrapped SEGV in CScreenshareFrame::transform() reached via ext-image-copy-capture's create_frame, plus twice in libaquamarine), taking the stream/sessions/pointer units and both session outputs with it. When that happened, cells failed and hhstream status hung for minutes. The ext stack could not be reproduced in isolation (5 clean frames afterwards), so this is recorded as unstable under sustained capture load, not as "the modern protocol is broken".
  • Our own client segfaulted on exit 16 times (python3.14 … segfault … libwayland-client.so) — pywayland/CFFI teardown, after the result printed, so green runs reported exit 139. Fixed with os._exit(0) after flushing. A batch of these coincided in time with a compositor crash, so the fix may matter beyond tidiness.
  • Two SIGTERM kills of the benchmark (exit 143, no error line): a 40-minute nix-shell tree does not belong in an agent background process. Moved to a transient systemd unit, and the runner made resumable.
  • grim -o NAME must precede the trailing -, or grim exits 0 having written 637 bytes of nothing.
  • grim's only uncompressed format is ppm; pbm/pnm/rgb/bmp are rejected. Raw is not cheaper than JPEG for it — writing 2.76 MB down a pipe costs what encoding costs.
  • turbojpeg reuses a non-NULL *jpegBuf: freeing it and leaving the pointer makes the next call write into freed memory (heap-corrupting segfault with no stack trace near the encoder).
  • display.dispatch(block=False) never reads the socket — it processes only what is already read, so a wait built on it times out against a healthy compositor. The read is dispatch(block=True).
  • ext-image-capture-source-v1 has one manager per source kind (ext_output_…, ext_foreign_toplevel_…). Looking for a generic ext_image_capture_source_manager_v1 finds nothing and reads as "the modern protocol is unavailable" — a wrong conclusion this spike drew once. The toplevel variant means single-window capture is available, which matters for agent eyes.
  • --all cells silently changed geometry (2560→3840) once a second session existed, so they are excluded from this table rather than averaged.
  • A --resume keyed on file existence skips failed cells too, because a failure also writes a file; it now requires a parseable result line.

Verdict: VALIDATED — with a caveat that changes the shape of the answer

  • Replace grim with an in-process pywayland client on zwlr-screencopy for the one-session case (live streaming a workspace, agent eyes). ~1.5× faster, ~2.8× less own CPU, and — the number that decides it — the compositor's per-frame CPU nearly halves.
  • Do not adopt grim-rs: no speed win, 40× the memory.
  • Do not adopt ext-image-copy-capture yet on this Hyprland build: no measured gain, and its create-frame path appears in a compositor crash stack.
  • Keep raw/ppm out of the wire path: it is not cheaper than JPEG anywhere measured.
  • The caveat: on this software-rendered 2-core VM the machine is saturated by rendering whatever is on screen, in every configuration including idle. The frame-rate win is real, but no capture change buys headroom — it changes how much of a saturated box the compositor keeps.

What a production version would take

  • Replace the subprocess behind src/hhstream/capture.py's CommandCapture with a pywayland screencopy client: ~250 lines, vendored protocol XML (the pattern already used by src/hhstream_pointer/_protocol/), and no new dependency — pywayland is already a dependency of the package, so the stdlib-only policy for the frame socket, CLI and broker is untouched.
  • JPEG encoding then needs libjpeg-turbo via ctypes (stdlib zlib PNG is the fallback, at 2–3× the bytes) — that is a genuine new dependency for that module, and the one thing worth arguing about in review.
  • Single-window capture comes free via ext_foreign_toplevel_image_capture_source_manager_v1; worth a follow-up spike once the compositor's stability under sustained capture is understood.

Files

File What it is
HARNESS.md the frozen measurement contract (argv, JSON, timing boundary, resource fields)
bench.py the runner: every route through every cell, raw JSON kept, table derived from the files, --resume
producers/grim_producer.py route 000, the baseline
producers/wayland_producer.py route 001b, raw pywayland (screencopy and ext-image-copy-capture)
producers/resources.py RAM/CPU sampling shared by the Python producers
001a-grim-rs/ route 001a, the Rust grim-rs producer (Rust toolchain confined to its own shell.nix)
run-final.sh, run-remaining.sh the pass drivers, meant to run in a transient systemd unit
chart.py the two charts (python3 chart.py results/final-motion)
results/final-motion/ the table above, its chart, and every run's raw stdout/stderr
shell.nix the spike's environment — deliberately not in the repo flake