Files
hyprland-headless-stream-pl…/SPIKE-BRIEF.md
T
Hermenix da14048010 spike: measure grim, raw pywayland and grim-rs on one contract, with RAM and CPU
The question is which capture route to ship for live streaming and agent eyes.
Speed alone does not decide it, so every route now also reports producer CPU and
RSS, machine CPU from /proc/stat, the compositor's own CPU, and a per-frame
buffer footprint -- all sampled around the measured loop, with an idle control
sample so an animating page's cost is not attributed to the capture.

The baseline forks a process per frame and spends 95% of each frame waiting, so
the interesting finding is not which route is faster but which one leaves the
compositor able to do its job: in-process acquisition roughly halves the
compositor's per-frame CPU even though the machine saturates either way.
2026-10-11 15:45:09 +00:00

7.8 KiB
Raw Blame History

SPIKE BRIEF — two capture routes, measured head to head

An experiment, in a fresh worktree: /var/lib/hermes/worktrees/hhs-capture-speed, branch spike/capture-speed, forked from main at f3ac9ad. Read README.dev.md and docs/CONTRACT.md first; the repo's conventions apply (docstrings that say why, nix develop, no CLI shims where a socket will do).

This is a spike, not production: spikes/ is throwaway code, but the numbers must be real. See the spike convention: decompose → research → build → verdict.

The question the operator asked

Which approach lets us render images faster — the existing grim subprocess, a Rust library (grim-rs), or a raw Python Wayland client? This is for live streaming the workspaces to the web, and for agent eyes.

"Render images faster" means, concretely: time from "wake up and ask" to "pixels/JFIF bytes in a buffer I can publish", sustained, and the frame rate that follows. Encode cost is part of it, because what gets published is JPEG.

The three routes

Measure all three with one harness, in this order (baseline first, so the comparison is anchored):

  • 000 — baseline: grim as a subprocess. The shipped path: grim -c -t jpeg -q 80 -o <output> - per frame. One process per frame. A prior measurement in this repo says ~46 ms/frame at 1280x720 q80 — do not trust it, re-measure it here.
  • 001a — grim-rs. The Rust crate (crates.io grim-rs, ~0.2): native capture over ext-image-copy-capture-v1 with a zwlr_screencopy_manager_v1 fallback, PNG/JPEG encoders included. Measure two variants separately and label which number came from which:
    • (a1) a long-lived Rust producer process — one connection, N frames captured and encoded in a loop (the "another grim, but persistent" model);
    • (a2) in-process from Python via PyO3/maturin, if it fits in the spike. If it does not, say so plainly and leave a1 as the number. Do not fake a2.
  • 001b — raw Python. pywayland + vendored protocol XML for ext-image-copy-capture-v1 (Hyprland has supported it since 0.52.1; this machine runs 0.56.2 and ships src/protocols/ImageCopyCapture.cpp), with a zwlr_screencopy fallback if the newer protocol turns out to be unavailable in practice. Follow the precedent already in this repo: the input driver vendors its protocol bindings under src/hhstream_pointer/_protocol/ and documents how to regenerate them (regenerate.py) — do the same in the spike, and say where you got the XML.

Research before building (that is the point of the research step): read the grim-rs API and the ext-image-copy-capture-v1 spec, and check what pywayland needs to bind a new protocol. Pick approaches and say why.

Environment facts — get these right or the numbers are meaningless

  • The compositor is headless Hyprland 0.56.2 running as a system service, not in your shell. The published Wayland socket is WAYLAND_DISPLAY=/run/hypr-stream/wayland-1 (an absolute path). Do NOT set XDG_RUNTIME_DIR=/run/hypr-stream — that directory is owned by the hhstream user and libwayland refuses a runtime dir the caller does not own; the failure reads as "failed to create display".
  • The frame socket is /run/hypr-stream/stream.sock (hhstream status reports all of this). hhstream shot --output <name> is the quickest way to prove a capture path works before you time it.
  • There is no GPU. The compositor renders through software (kms_swrast / llvmpipe). So: do not use the zero-copy dma-buf / GPU paths to make the Rust route look good — they cannot help on this box, and a benchmark that silently takes a GPU path and falls back would not be honest. Measure the CPU path, and say in the verdict what a GPU box would change.
  • Geometries to measure (both, always): 1280×720 (one session container) and 2560×720 (the whole desktop as hhstream publishes it today — this is the geometry on the wire now).
  • Measure a moving screen and a still one. A still screen with damage-driven capture can block or go quiet, which is exactly the kind of thing that decides a live-stream design — the current hhstream path re-encodes the same frame on a timer instead. If a route behaves differently on a still screen, that is a finding, not a nuisance. To make motion: start a session (hhstream session start chromium -- --ozone-platform=wayland --new-window https://example.com — the allowlist is in the NixOS module) or use examples/agent-eyes-demo.py, which already drives a page. A foot session with something printing in a loop is cheaper and moves plenty.
  • A live session for testing may already exist (hhstream session list); you may start your own. Do not kill another agent's session without saying so.

The harness (build this once, use it for all routes)

  • Warm-up frames excluded; then ≥300 measured frames per cell; report p50 / p95 / max per-frame ms, sustained fps, and CPU time per frame (time.process_time / /proc for the subprocess cases) — a route that is fast by pinning a core is not fast.
  • Separate the two stages so the verdict can say where the time goes: (i) frame ready in a buffer, (ii) + JPEG encode. Report both. If a route cannot separate them, say so.
  • Same output, same geometry, same content for every route; run each cell at least twice and report the spread, not one lucky run.
  • Print a machine-readable line per cell (JSON) and a human table, and write the raw runs to a file so the numbers in the README can be re-derived.

Deliverables

  1. spikes/capture-speed/README.md — the question, the method, the head-to-head table, and a ## Verdict (VALIDATED / PARTIAL / INVALIDATED) with what worked, what didn't, surprises, and a recommendation for the real build. The existing grim route and the two new ones all get a row.
  2. spikes/capture-speed/001a-grim-rs/README.md and 001b-pywayland/README.md with their own verdicts and the raw evidence.
  3. A chart PNG (frame time and sustained fps per route, both geometries). ad-hoc-charts conventions if you want a recipe; matplotlib via nix shell nixpkgs#python3Packages.matplotlib is fine.
  4. Something the operator can watch — this matters more than the table. The cheapest honest version: run each winning-capable backend behind an MJPEG endpoint the operator can open in a browser, both at once or one at a time, with the achieved fps printed on the page or in its sidecar status.json. Reuse src/hhstream_live/ (its publisher takes a capture command/loop) rather than writing a second web app; if reuse turns out to be the harder path, say so and keep the demo small. Bind it sensibly: the guest is in NAT mode, so a browser on the host may or may not reach a port — say what you bound and how you verified it with curl, and do not touch the system firewall.
  5. A short "what a production version would take" section: which file in src/hhstream/ the winner would replace, what dependencies it adds, and what the repo's stdlib-only policy would cost.

Rules

  • Do not touch the running system: no nixos-rebuild, no edits under /etc/nixos, no changes to the running hhstream units. You may run the CLI and start/stop your own sessions.
  • Keep Rust out of the repo's main flake: a spike-local shell.nix/flake under spikes/capture-speed/ is the place for cargo, rustc, maturin. cargo/rustc 1.98 are in nixpkgs and crates.io is reachable from here.
  • Commit on spike/capture-speed as you go. Do not merge, do not touch main.
  • Every number in a README must be reproducible by a command you paste next to it. If something did not work, it goes in the README as a finding, not a silent omission.