The question is which capture route to ship for live streaming and agent eyes. Speed alone does not decide it, so every route now also reports producer CPU and RSS, machine CPU from /proc/stat, the compositor's own CPU, and a per-frame buffer footprint -- all sampled around the measured loop, with an idle control sample so an animating page's cost is not attributed to the capture. The baseline forks a process per frame and spends 95% of each frame waiting, so the interesting finding is not which route is faster but which one leaves the compositor able to do its job: in-process acquisition roughly halves the compositor's per-frame CPU even though the machine saturates either way.
133 lines
7.8 KiB
Markdown
133 lines
7.8 KiB
Markdown
# SPIKE BRIEF — two capture routes, measured head to head
|
||
|
||
An experiment, in a fresh worktree: `/var/lib/hermes/worktrees/hhs-capture-speed`,
|
||
branch `spike/capture-speed`, forked from `main` at `f3ac9ad`. Read
|
||
`README.dev.md` and `docs/CONTRACT.md` first; the repo's conventions apply
|
||
(docstrings that say *why*, `nix develop`, no CLI shims where a socket will do).
|
||
|
||
This is a **spike**, not production: `spikes/` is throwaway code, but the numbers
|
||
must be real. See the `spike` convention: decompose → research → build → verdict.
|
||
|
||
## The question the operator asked
|
||
|
||
> Which approach lets us render images faster — the existing `grim` subprocess, a
|
||
> Rust library (`grim-rs`), or a raw Python Wayland client? This is for **live
|
||
> streaming the workspaces to the web, and for agent eyes.**
|
||
|
||
"Render images faster" means, concretely: **time from "wake up and ask" to
|
||
"pixels/JFIF bytes in a buffer I can publish"**, sustained, and the frame rate
|
||
that follows. Encode cost is part of it, because what gets published is JPEG.
|
||
|
||
## The three routes
|
||
|
||
Measure all three with **one harness**, in this order (baseline first, so the
|
||
comparison is anchored):
|
||
|
||
- **000 — baseline: `grim` as a subprocess.** The shipped path:
|
||
`grim -c -t jpeg -q 80 -o <output> -` per frame. One process per frame.
|
||
A prior measurement in this repo says ~46 ms/frame at 1280x720 q80 **— do not
|
||
trust it, re-measure it here.**
|
||
- **001a — `grim-rs`.** The Rust crate (crates.io `grim-rs`, ~0.2): native
|
||
capture over `ext-image-copy-capture-v1` with a `zwlr_screencopy_manager_v1`
|
||
fallback, PNG/JPEG encoders included. Measure **two variants separately** and
|
||
label which number came from which:
|
||
- **(a1)** a long-lived Rust producer process — one connection, N frames
|
||
captured and encoded in a loop (the "another grim, but persistent" model);
|
||
- **(a2)** in-process from Python via PyO3/maturin, **if it fits in the
|
||
spike**. If it does not, say so plainly and leave a1 as the number. Do not
|
||
fake a2.
|
||
- **001b — raw Python.** `pywayland` + vendored protocol XML for
|
||
`ext-image-copy-capture-v1` (Hyprland has supported it since 0.52.1; this
|
||
machine runs 0.56.2 and ships `src/protocols/ImageCopyCapture.cpp`), with a
|
||
`zwlr_screencopy` fallback if the newer protocol turns out to be unavailable
|
||
in practice. **Follow the precedent already in this repo**: the input driver
|
||
vendors its protocol bindings under `src/hhstream_pointer/_protocol/` and
|
||
documents how to regenerate them (`regenerate.py`) — do the same in the spike,
|
||
and say where you got the XML.
|
||
|
||
Research before building (that is the point of the research step): read the
|
||
`grim-rs` API and the `ext-image-copy-capture-v1` spec, and check what
|
||
`pywayland` needs to bind a new protocol. Pick approaches and say why.
|
||
|
||
## Environment facts — get these right or the numbers are meaningless
|
||
|
||
- The compositor is **headless Hyprland 0.56.2 running as a system service**, not
|
||
in your shell. The published Wayland socket is
|
||
**`WAYLAND_DISPLAY=/run/hypr-stream/wayland-1`** (an absolute path).
|
||
**Do NOT set `XDG_RUNTIME_DIR=/run/hypr-stream`** — that directory is owned by
|
||
the `hhstream` user and libwayland refuses a runtime dir the caller does not
|
||
own; the failure reads as "failed to create display".
|
||
- The frame socket is `/run/hypr-stream/stream.sock` (`hhstream status` reports
|
||
all of this). `hhstream shot --output <name>` is the quickest way to prove a
|
||
capture path works before you time it.
|
||
- **There is no GPU.** The compositor renders through software (kms_swrast /
|
||
llvmpipe). So: **do not use the zero-copy dma-buf / GPU paths to make the Rust
|
||
route look good** — they cannot help on this box, and a benchmark that silently
|
||
takes a GPU path and falls back would not be honest. Measure the CPU path, and
|
||
say in the verdict what a GPU box would change.
|
||
- Geometries to measure (both, always): **1280×720** (one session container) and
|
||
**2560×720** (the whole desktop as `hhstream` publishes it today — this is the
|
||
geometry on the wire now).
|
||
- Measure **a moving screen and a still one.** A still screen with damage-driven
|
||
capture can block or go quiet, which is exactly the kind of thing that decides
|
||
a live-stream design — the current `hhstream` path re-encodes the same frame on
|
||
a timer instead. If a route behaves differently on a still screen, that is a
|
||
finding, not a nuisance. To make motion: start a session
|
||
(`hhstream session start chromium -- --ozone-platform=wayland --new-window
|
||
https://example.com` — the allowlist is in the NixOS module) or use
|
||
`examples/agent-eyes-demo.py`, which already drives a page. A `foot` session
|
||
with something printing in a loop is cheaper and moves plenty.
|
||
- A live session for testing may already exist (`hhstream session list`); you may
|
||
start your own. **Do not kill another agent's session without saying so.**
|
||
|
||
## The harness (build this once, use it for all routes)
|
||
|
||
- Warm-up frames excluded; then **≥300 measured frames per cell**; report
|
||
**p50 / p95 / max per-frame ms**, sustained fps, and **CPU time per frame**
|
||
(`time.process_time` / `/proc` for the subprocess cases) — a route that is fast
|
||
by pinning a core is not fast.
|
||
- Separate the two stages so the verdict can say where the time goes:
|
||
**(i) frame ready in a buffer**, **(ii) + JPEG encode**. Report both. If a
|
||
route cannot separate them, say so.
|
||
- Same output, same geometry, same content for every route; run each cell at
|
||
least twice and report the spread, not one lucky run.
|
||
- Print a machine-readable line per cell (JSON) **and** a human table, and write
|
||
the raw runs to a file so the numbers in the README can be re-derived.
|
||
|
||
## Deliverables
|
||
|
||
1. `spikes/capture-speed/README.md` — the question, the method, the head-to-head
|
||
table, and a `## Verdict` (VALIDATED / PARTIAL / INVALIDATED) with what worked,
|
||
what didn't, surprises, and a recommendation for the real build. The existing
|
||
`grim` route and the two new ones all get a row.
|
||
2. `spikes/capture-speed/001a-grim-rs/README.md` and `001b-pywayland/README.md`
|
||
with their own verdicts and the raw evidence.
|
||
3. **A chart PNG** (frame time and sustained fps per route, both geometries).
|
||
`ad-hoc-charts` conventions if you want a recipe; matplotlib via
|
||
`nix shell nixpkgs#python3Packages.matplotlib` is fine.
|
||
4. **Something the operator can watch** — this matters more than the table. The
|
||
cheapest honest version: run each *winning-capable* backend behind an MJPEG
|
||
endpoint the operator can open in a browser, both at once or one at a time,
|
||
with the achieved fps printed on the page or in its sidecar `status.json`.
|
||
Reuse `src/hhstream_live/` (its publisher takes a capture command/loop) rather
|
||
than writing a second web app; if reuse turns out to be the harder path, say
|
||
so and keep the demo small. Bind it sensibly: the guest is in NAT mode, so a
|
||
browser on the host may or may not reach a port — say what you bound and how
|
||
you verified it with `curl`, and do not touch the system firewall.
|
||
5. A short "what a production version would take" section: which file in
|
||
`src/hhstream/` the winner would replace, what dependencies it adds, and what
|
||
the repo's stdlib-only policy would cost.
|
||
|
||
## Rules
|
||
|
||
- **Do not touch the running system**: no `nixos-rebuild`, no edits under
|
||
`/etc/nixos`, no changes to the running `hhstream` units. You may run the CLI
|
||
and start/stop your own sessions.
|
||
- Keep Rust out of the repo's main flake: a spike-local `shell.nix`/flake under
|
||
`spikes/capture-speed/` is the place for `cargo`, `rustc`, `maturin`.
|
||
`cargo`/`rustc` 1.98 are in nixpkgs and crates.io is reachable from here.
|
||
- Commit on `spike/capture-speed` as you go. Do not merge, do not touch `main`.
|
||
- Every number in a README must be reproducible by a command you paste next to
|
||
it. If something did not work, it goes in the README as a finding, not a
|
||
silent omission.
|