The question is which capture route to ship for live streaming and agent eyes. Speed alone does not decide it, so every route now also reports producer CPU and RSS, machine CPU from /proc/stat, the compositor's own CPU, and a per-frame buffer footprint -- all sampled around the measured loop, with an idle control sample so an animating page's cost is not attributed to the capture. The baseline forks a process per frame and spends 95% of each frame waiting, so the interesting finding is not which route is faster but which one leaves the compositor able to do its job: in-process acquisition roughly halves the compositor's per-frame CPU even though the machine saturates either way.
7.8 KiB
SPIKE BRIEF — two capture routes, measured head to head
An experiment, in a fresh worktree: /var/lib/hermes/worktrees/hhs-capture-speed,
branch spike/capture-speed, forked from main at f3ac9ad. Read
README.dev.md and docs/CONTRACT.md first; the repo's conventions apply
(docstrings that say why, nix develop, no CLI shims where a socket will do).
This is a spike, not production: spikes/ is throwaway code, but the numbers
must be real. See the spike convention: decompose → research → build → verdict.
The question the operator asked
Which approach lets us render images faster — the existing
grimsubprocess, a Rust library (grim-rs), or a raw Python Wayland client? This is for live streaming the workspaces to the web, and for agent eyes.
"Render images faster" means, concretely: time from "wake up and ask" to "pixels/JFIF bytes in a buffer I can publish", sustained, and the frame rate that follows. Encode cost is part of it, because what gets published is JPEG.
The three routes
Measure all three with one harness, in this order (baseline first, so the comparison is anchored):
- 000 — baseline:
grimas a subprocess. The shipped path:grim -c -t jpeg -q 80 -o <output> -per frame. One process per frame. A prior measurement in this repo says ~46 ms/frame at 1280x720 q80 — do not trust it, re-measure it here. - 001a —
grim-rs. The Rust crate (crates.iogrim-rs, ~0.2): native capture overext-image-copy-capture-v1with azwlr_screencopy_manager_v1fallback, PNG/JPEG encoders included. Measure two variants separately and label which number came from which:- (a1) a long-lived Rust producer process — one connection, N frames captured and encoded in a loop (the "another grim, but persistent" model);
- (a2) in-process from Python via PyO3/maturin, if it fits in the spike. If it does not, say so plainly and leave a1 as the number. Do not fake a2.
- 001b — raw Python.
pywayland+ vendored protocol XML forext-image-copy-capture-v1(Hyprland has supported it since 0.52.1; this machine runs 0.56.2 and shipssrc/protocols/ImageCopyCapture.cpp), with azwlr_screencopyfallback if the newer protocol turns out to be unavailable in practice. Follow the precedent already in this repo: the input driver vendors its protocol bindings undersrc/hhstream_pointer/_protocol/and documents how to regenerate them (regenerate.py) — do the same in the spike, and say where you got the XML.
Research before building (that is the point of the research step): read the
grim-rs API and the ext-image-copy-capture-v1 spec, and check what
pywayland needs to bind a new protocol. Pick approaches and say why.
Environment facts — get these right or the numbers are meaningless
- The compositor is headless Hyprland 0.56.2 running as a system service, not
in your shell. The published Wayland socket is
WAYLAND_DISPLAY=/run/hypr-stream/wayland-1(an absolute path). Do NOT setXDG_RUNTIME_DIR=/run/hypr-stream— that directory is owned by thehhstreamuser and libwayland refuses a runtime dir the caller does not own; the failure reads as "failed to create display". - The frame socket is
/run/hypr-stream/stream.sock(hhstream statusreports all of this).hhstream shot --output <name>is the quickest way to prove a capture path works before you time it. - There is no GPU. The compositor renders through software (kms_swrast / llvmpipe). So: do not use the zero-copy dma-buf / GPU paths to make the Rust route look good — they cannot help on this box, and a benchmark that silently takes a GPU path and falls back would not be honest. Measure the CPU path, and say in the verdict what a GPU box would change.
- Geometries to measure (both, always): 1280×720 (one session container) and
2560×720 (the whole desktop as
hhstreampublishes it today — this is the geometry on the wire now). - Measure a moving screen and a still one. A still screen with damage-driven
capture can block or go quiet, which is exactly the kind of thing that decides
a live-stream design — the current
hhstreampath re-encodes the same frame on a timer instead. If a route behaves differently on a still screen, that is a finding, not a nuisance. To make motion: start a session (hhstream session start chromium -- --ozone-platform=wayland --new-window https://example.com— the allowlist is in the NixOS module) or useexamples/agent-eyes-demo.py, which already drives a page. Afootsession with something printing in a loop is cheaper and moves plenty. - A live session for testing may already exist (
hhstream session list); you may start your own. Do not kill another agent's session without saying so.
The harness (build this once, use it for all routes)
- Warm-up frames excluded; then ≥300 measured frames per cell; report
p50 / p95 / max per-frame ms, sustained fps, and CPU time per frame
(
time.process_time//procfor the subprocess cases) — a route that is fast by pinning a core is not fast. - Separate the two stages so the verdict can say where the time goes: (i) frame ready in a buffer, (ii) + JPEG encode. Report both. If a route cannot separate them, say so.
- Same output, same geometry, same content for every route; run each cell at least twice and report the spread, not one lucky run.
- Print a machine-readable line per cell (JSON) and a human table, and write the raw runs to a file so the numbers in the README can be re-derived.
Deliverables
spikes/capture-speed/README.md— the question, the method, the head-to-head table, and a## Verdict(VALIDATED / PARTIAL / INVALIDATED) with what worked, what didn't, surprises, and a recommendation for the real build. The existinggrimroute and the two new ones all get a row.spikes/capture-speed/001a-grim-rs/README.mdand001b-pywayland/README.mdwith their own verdicts and the raw evidence.- A chart PNG (frame time and sustained fps per route, both geometries).
ad-hoc-chartsconventions if you want a recipe; matplotlib vianix shell nixpkgs#python3Packages.matplotlibis fine. - Something the operator can watch — this matters more than the table. The
cheapest honest version: run each winning-capable backend behind an MJPEG
endpoint the operator can open in a browser, both at once or one at a time,
with the achieved fps printed on the page or in its sidecar
status.json. Reusesrc/hhstream_live/(its publisher takes a capture command/loop) rather than writing a second web app; if reuse turns out to be the harder path, say so and keep the demo small. Bind it sensibly: the guest is in NAT mode, so a browser on the host may or may not reach a port — say what you bound and how you verified it withcurl, and do not touch the system firewall. - A short "what a production version would take" section: which file in
src/hhstream/the winner would replace, what dependencies it adds, and what the repo's stdlib-only policy would cost.
Rules
- Do not touch the running system: no
nixos-rebuild, no edits under/etc/nixos, no changes to the runninghhstreamunits. You may run the CLI and start/stop your own sessions. - Keep Rust out of the repo's main flake: a spike-local
shell.nix/flake underspikes/capture-speed/is the place forcargo,rustc,maturin.cargo/rustc1.98 are in nixpkgs and crates.io is reachable from here. - Commit on
spike/capture-speedas you go. Do not merge, do not touchmain. - Every number in a README must be reproducible by a command you paste next to it. If something did not work, it goes in the README as a finding, not a silent omission.