Files
hyprland-headless-stream-pl…/SPIKE-BRIEF.md
T
Hermenix da14048010 spike: measure grim, raw pywayland and grim-rs on one contract, with RAM and CPU
The question is which capture route to ship for live streaming and agent eyes.
Speed alone does not decide it, so every route now also reports producer CPU and
RSS, machine CPU from /proc/stat, the compositor's own CPU, and a per-frame
buffer footprint -- all sampled around the measured loop, with an idle control
sample so an animating page's cost is not attributed to the capture.

The baseline forks a process per frame and spends 95% of each frame waiting, so
the interesting finding is not which route is faster but which one leaves the
compositor able to do its job: in-process acquisition roughly halves the
compositor's per-frame CPU even though the machine saturates either way.
2026-10-11 15:45:09 +00:00

133 lines
7.8 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# SPIKE BRIEF — two capture routes, measured head to head
An experiment, in a fresh worktree: `/var/lib/hermes/worktrees/hhs-capture-speed`,
branch `spike/capture-speed`, forked from `main` at `f3ac9ad`. Read
`README.dev.md` and `docs/CONTRACT.md` first; the repo's conventions apply
(docstrings that say *why*, `nix develop`, no CLI shims where a socket will do).
This is a **spike**, not production: `spikes/` is throwaway code, but the numbers
must be real. See the `spike` convention: decompose → research → build → verdict.
## The question the operator asked
> Which approach lets us render images faster — the existing `grim` subprocess, a
> Rust library (`grim-rs`), or a raw Python Wayland client? This is for **live
> streaming the workspaces to the web, and for agent eyes.**
"Render images faster" means, concretely: **time from "wake up and ask" to
"pixels/JFIF bytes in a buffer I can publish"**, sustained, and the frame rate
that follows. Encode cost is part of it, because what gets published is JPEG.
## The three routes
Measure all three with **one harness**, in this order (baseline first, so the
comparison is anchored):
- **000 — baseline: `grim` as a subprocess.** The shipped path:
`grim -c -t jpeg -q 80 -o <output> -` per frame. One process per frame.
A prior measurement in this repo says ~46 ms/frame at 1280x720 q80 **— do not
trust it, re-measure it here.**
- **001a — `grim-rs`.** The Rust crate (crates.io `grim-rs`, ~0.2): native
capture over `ext-image-copy-capture-v1` with a `zwlr_screencopy_manager_v1`
fallback, PNG/JPEG encoders included. Measure **two variants separately** and
label which number came from which:
- **(a1)** a long-lived Rust producer process — one connection, N frames
captured and encoded in a loop (the "another grim, but persistent" model);
- **(a2)** in-process from Python via PyO3/maturin, **if it fits in the
spike**. If it does not, say so plainly and leave a1 as the number. Do not
fake a2.
- **001b — raw Python.** `pywayland` + vendored protocol XML for
`ext-image-copy-capture-v1` (Hyprland has supported it since 0.52.1; this
machine runs 0.56.2 and ships `src/protocols/ImageCopyCapture.cpp`), with a
`zwlr_screencopy` fallback if the newer protocol turns out to be unavailable
in practice. **Follow the precedent already in this repo**: the input driver
vendors its protocol bindings under `src/hhstream_pointer/_protocol/` and
documents how to regenerate them (`regenerate.py`) — do the same in the spike,
and say where you got the XML.
Research before building (that is the point of the research step): read the
`grim-rs` API and the `ext-image-copy-capture-v1` spec, and check what
`pywayland` needs to bind a new protocol. Pick approaches and say why.
## Environment facts — get these right or the numbers are meaningless
- The compositor is **headless Hyprland 0.56.2 running as a system service**, not
in your shell. The published Wayland socket is
**`WAYLAND_DISPLAY=/run/hypr-stream/wayland-1`** (an absolute path).
**Do NOT set `XDG_RUNTIME_DIR=/run/hypr-stream`** — that directory is owned by
the `hhstream` user and libwayland refuses a runtime dir the caller does not
own; the failure reads as "failed to create display".
- The frame socket is `/run/hypr-stream/stream.sock` (`hhstream status` reports
all of this). `hhstream shot --output <name>` is the quickest way to prove a
capture path works before you time it.
- **There is no GPU.** The compositor renders through software (kms_swrast /
llvmpipe). So: **do not use the zero-copy dma-buf / GPU paths to make the Rust
route look good** — they cannot help on this box, and a benchmark that silently
takes a GPU path and falls back would not be honest. Measure the CPU path, and
say in the verdict what a GPU box would change.
- Geometries to measure (both, always): **1280×720** (one session container) and
**2560×720** (the whole desktop as `hhstream` publishes it today — this is the
geometry on the wire now).
- Measure **a moving screen and a still one.** A still screen with damage-driven
capture can block or go quiet, which is exactly the kind of thing that decides
a live-stream design — the current `hhstream` path re-encodes the same frame on
a timer instead. If a route behaves differently on a still screen, that is a
finding, not a nuisance. To make motion: start a session
(`hhstream session start chromium -- --ozone-platform=wayland --new-window
https://example.com` — the allowlist is in the NixOS module) or use
`examples/agent-eyes-demo.py`, which already drives a page. A `foot` session
with something printing in a loop is cheaper and moves plenty.
- A live session for testing may already exist (`hhstream session list`); you may
start your own. **Do not kill another agent's session without saying so.**
## The harness (build this once, use it for all routes)
- Warm-up frames excluded; then **≥300 measured frames per cell**; report
**p50 / p95 / max per-frame ms**, sustained fps, and **CPU time per frame**
(`time.process_time` / `/proc` for the subprocess cases) — a route that is fast
by pinning a core is not fast.
- Separate the two stages so the verdict can say where the time goes:
**(i) frame ready in a buffer**, **(ii) + JPEG encode**. Report both. If a
route cannot separate them, say so.
- Same output, same geometry, same content for every route; run each cell at
least twice and report the spread, not one lucky run.
- Print a machine-readable line per cell (JSON) **and** a human table, and write
the raw runs to a file so the numbers in the README can be re-derived.
## Deliverables
1. `spikes/capture-speed/README.md` — the question, the method, the head-to-head
table, and a `## Verdict` (VALIDATED / PARTIAL / INVALIDATED) with what worked,
what didn't, surprises, and a recommendation for the real build. The existing
`grim` route and the two new ones all get a row.
2. `spikes/capture-speed/001a-grim-rs/README.md` and `001b-pywayland/README.md`
with their own verdicts and the raw evidence.
3. **A chart PNG** (frame time and sustained fps per route, both geometries).
`ad-hoc-charts` conventions if you want a recipe; matplotlib via
`nix shell nixpkgs#python3Packages.matplotlib` is fine.
4. **Something the operator can watch** — this matters more than the table. The
cheapest honest version: run each *winning-capable* backend behind an MJPEG
endpoint the operator can open in a browser, both at once or one at a time,
with the achieved fps printed on the page or in its sidecar `status.json`.
Reuse `src/hhstream_live/` (its publisher takes a capture command/loop) rather
than writing a second web app; if reuse turns out to be the harder path, say
so and keep the demo small. Bind it sensibly: the guest is in NAT mode, so a
browser on the host may or may not reach a port — say what you bound and how
you verified it with `curl`, and do not touch the system firewall.
5. A short "what a production version would take" section: which file in
`src/hhstream/` the winner would replace, what dependencies it adds, and what
the repo's stdlib-only policy would cost.
## Rules
- **Do not touch the running system**: no `nixos-rebuild`, no edits under
`/etc/nixos`, no changes to the running `hhstream` units. You may run the CLI
and start/stop your own sessions.
- Keep Rust out of the repo's main flake: a spike-local `shell.nix`/flake under
`spikes/capture-speed/` is the place for `cargo`, `rustc`, `maturin`.
`cargo`/`rustc` 1.98 are in nixpkgs and crates.io is reachable from here.
- Commit on `spike/capture-speed` as you go. Do not merge, do not touch `main`.
- Every number in a README must be reproducible by a command you paste next to
it. If something did not work, it goes in the README as a finding, not a
silent omission.