Files
hyprland-headless-stream-pl…/spikes/capture-speed/README.md
T
Hermenix da14048010 spike: measure grim, raw pywayland and grim-rs on one contract, with RAM and CPU
The question is which capture route to ship for live streaming and agent eyes.
Speed alone does not decide it, so every route now also reports producer CPU and
RSS, machine CPU from /proc/stat, the compositor's own CPU, and a per-frame
buffer footprint -- all sampled around the measured loop, with an idle control
sample so an animating page's cost is not attributed to the capture.

The baseline forks a process per frame and spends 95% of each frame waiting, so
the interesting finding is not which route is faster but which one leaves the
compositor able to do its job: in-process acquisition roughly halves the
compositor's per-frame CPU even though the machine saturates either way.
2026-10-11 15:45:09 +00:00

105 lines
5.7 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# capture-speed — can we drop `grim`, and what would it buy?
**Question** (the operator's): which approach renders images faster for live
streaming the workspaces to the web and for agent eyes — the shipped `grim`
subprocess, the Rust `grim-rs` crate, or a raw `pywayland` client? And, since
speed alone does not decide it: **what does each cost in RAM and CPU?**
**Answer, short version**: the capture is not the expensive part — the
*round trip* is. In-process acquisition roughly doubles the frame rate and
divides the producer's own CPU by four, but on this machine both routes push the
box to saturation, because the compositor is software-rendering an animating
screen. Numbers, method and the bugs found are below.
## Method
Everything is measured through the frozen contract in `HARNESS.md`, by
`bench.py`, on one target so the cells are comparable:
- **Target**: a Chromium session (`--app=http://127.0.0.1/spin.html`, a
CSS-animated page — motion with no JavaScript and no timers, so the screen
really changes and a damage-driven capture cannot go quiet and flatter a
route). Pixels verified to differ between frames before any run.
- **Cells**: one session container (1280×720) and the whole desktop stitched
(2560×720) × raw and JPEG q80 × two runs each.
- **Baseline**: `grim` as it ships — one process per frame, fork and encode
inside the timed span.
- **Resources**: producer CPU/RSS (`getrusage`), machine CPU (`/proc/stat`), the
**compositor's own** CPU (`/proc/<Hyprland>/stat`), plus an **idle control**
sample taken with nothing capturing at all — without it a per-frame CPU number
would silently include the animation's cost and be attributed to the capture.
- **No GPU on this machine.** Hyprland renders through software
(kms_swrast/llvmpipe), so no route was tuned to a dma-buf path and none is
claimed. Each producer says in its own result which protocol actually served
its frames.
## Where the time actually goes (the finding that reframes it)
`grim` at 1280×720 JPEG q80: **p50 119 ms per frame, 7.9–8.3 fps, but only
6–8 ms of CPU.** About 95% of each frame is spent waiting — on the fork, the
connect, the compositor's round trip — not encoding. Two consequences:
1. Attacking the *encoder* is pointless: JPEG q80 of a 1280×720 frame costs
**1.7 ms** in-process.
2. The win available is in **not paying the round trip per frame** — which is
exactly what a persistent in-process client does, and why the two new routes
land so close to each other.
And the resource measurement adds the part speed hides: at 1280×720 with motion
on screen, **the machine is already saturated (≈199% of one core, i.e. both
cores) whichever route is used**, because the compositor is repainting a
software-rendered animated page. The capture route changes the *compositor's*
per-frame CPU materially (see the table), but it cannot make the box idle.
## Head-to-head
<!-- TABLE-PLACEHOLDER: filled by bench.py results, not by hand -->
## Bugs and traps found on the way (each cost real time)
1. **`grim -o NAME` must come *before* the trailing `-`.** The other order exits
0 and writes **637 bytes of nothing** to the pipe. Silent, and it cost the
first harness build.
2. **grim's only uncompressed format is `ppm`** (`pbm`/`pnm`/`rgb`/`bmp` are
rejected), which is what makes the encode/capture split measurable.
3. **`display.dispatch(block=False)` never reads the socket** — it processes only
events already read. A wait loop built on it plus `select` times out against a
perfectly healthy compositor. The read is `dispatch(block=True)`; the "never
hang" guarantee has to come from an alarm (`SIGALRM`), not from polling.
4. **`pywayland` dispatchers receive the proxy as their first argument** (four
parameters for a three-field event), and **`display.disconnect()` is
mandatory** or the process segfaults *after* printing — a green run that
reports exit 139 to any runner reading exit codes.
5. **turbojpeg reuses `*jpegBuf` when it is non-NULL.** Freeing it with `tjFree`
and leaving the pointer in place makes the next call write into freed memory:
a heap-corrupting segfault several frames later with no stack trace near the
encoder. Reset the pointer to NULL after freeing.
6. **`libjpeg_turbo`'s bare attribute resolves to its `-bin` output** in that
Nix shell — no shared library in it — and `wayland.xml` lives in the
**`wayland-scanner`** package, not in `wayland`. Both failures look like
missing dependencies rather than shell gaps.
7. **`ext-image-capture-source-v1` defines one manager per source kind**
(`ext_output_…` and `ext_foreign_toplevel_…`), not a generic
`ext_image_capture_source_manager_v1`. Looking for the generic name finds
nothing and reads as "the modern protocol is unavailable here" — a wrong
conclusion this spike drew once before checking the real global list. The
toplevel variant means **single-window capture is available** on this
compositor, which matters for agent eyes.
## Files
| File | What it is |
|---|---|
| `HARNESS.md` | the frozen measurement contract (argv, JSON, timing boundary, resource fields) |
| `bench.py` | the runner: every route through every cell, raw JSON kept, table derived from the files |
| `producers/grim_producer.py` | route 000, the baseline |
| `producers/wayland_producer.py` | route 001b, raw `pywayland` (screencopy *and* ext-image-copy-capture) |
| `producers/resources.py` | RAM/CPU sampling shared by the Python producers |
| `001a-grim-rs/` | route 001a, the Rust `grim-rs` producer |
| `results/` | every run's stdout/stderr, `results.json`, and `control.json` |
| `shell.nix` | the spike's own environment (pywayland, turbojpeg, rust) — deliberately not in the repo flake |
## Verdict
<!-- VERDICT-PLACEHOLDER -->