The question is which capture route to ship for live streaming and agent eyes. Speed alone does not decide it, so every route now also reports producer CPU and RSS, machine CPU from /proc/stat, the compositor's own CPU, and a per-frame buffer footprint -- all sampled around the measured loop, with an idle control sample so an animating page's cost is not attributed to the capture. The baseline forks a process per frame and spends 95% of each frame waiting, so the interesting finding is not which route is faster but which one leaves the compositor able to do its job: in-process acquisition roughly halves the compositor's per-frame CPU even though the machine saturates either way.
105 lines
5.7 KiB
Markdown
105 lines
5.7 KiB
Markdown
# capture-speed — can we drop `grim`, and what would it buy?
|
||
|
||
**Question** (the operator's): which approach renders images faster for live
|
||
streaming the workspaces to the web and for agent eyes — the shipped `grim`
|
||
subprocess, the Rust `grim-rs` crate, or a raw `pywayland` client? And, since
|
||
speed alone does not decide it: **what does each cost in RAM and CPU?**
|
||
|
||
**Answer, short version**: the capture is not the expensive part — the
|
||
*round trip* is. In-process acquisition roughly doubles the frame rate and
|
||
divides the producer's own CPU by four, but on this machine both routes push the
|
||
box to saturation, because the compositor is software-rendering an animating
|
||
screen. Numbers, method and the bugs found are below.
|
||
|
||
## Method
|
||
|
||
Everything is measured through the frozen contract in `HARNESS.md`, by
|
||
`bench.py`, on one target so the cells are comparable:
|
||
|
||
- **Target**: a Chromium session (`--app=http://127.0.0.1/spin.html`, a
|
||
CSS-animated page — motion with no JavaScript and no timers, so the screen
|
||
really changes and a damage-driven capture cannot go quiet and flatter a
|
||
route). Pixels verified to differ between frames before any run.
|
||
- **Cells**: one session container (1280×720) and the whole desktop stitched
|
||
(2560×720) × raw and JPEG q80 × two runs each.
|
||
- **Baseline**: `grim` as it ships — one process per frame, fork and encode
|
||
inside the timed span.
|
||
- **Resources**: producer CPU/RSS (`getrusage`), machine CPU (`/proc/stat`), the
|
||
**compositor's own** CPU (`/proc/<Hyprland>/stat`), plus an **idle control**
|
||
sample taken with nothing capturing at all — without it a per-frame CPU number
|
||
would silently include the animation's cost and be attributed to the capture.
|
||
- **No GPU on this machine.** Hyprland renders through software
|
||
(kms_swrast/llvmpipe), so no route was tuned to a dma-buf path and none is
|
||
claimed. Each producer says in its own result which protocol actually served
|
||
its frames.
|
||
|
||
## Where the time actually goes (the finding that reframes it)
|
||
|
||
`grim` at 1280×720 JPEG q80: **p50 119 ms per frame, 7.9–8.3 fps, but only
|
||
6–8 ms of CPU.** About 95% of each frame is spent waiting — on the fork, the
|
||
connect, the compositor's round trip — not encoding. Two consequences:
|
||
|
||
1. Attacking the *encoder* is pointless: JPEG q80 of a 1280×720 frame costs
|
||
**1.7 ms** in-process.
|
||
2. The win available is in **not paying the round trip per frame** — which is
|
||
exactly what a persistent in-process client does, and why the two new routes
|
||
land so close to each other.
|
||
|
||
And the resource measurement adds the part speed hides: at 1280×720 with motion
|
||
on screen, **the machine is already saturated (≈199% of one core, i.e. both
|
||
cores) whichever route is used**, because the compositor is repainting a
|
||
software-rendered animated page. The capture route changes the *compositor's*
|
||
per-frame CPU materially (see the table), but it cannot make the box idle.
|
||
|
||
## Head-to-head
|
||
|
||
<!-- TABLE-PLACEHOLDER: filled by bench.py results, not by hand -->
|
||
|
||
## Bugs and traps found on the way (each cost real time)
|
||
|
||
1. **`grim -o NAME` must come *before* the trailing `-`.** The other order exits
|
||
0 and writes **637 bytes of nothing** to the pipe. Silent, and it cost the
|
||
first harness build.
|
||
2. **grim's only uncompressed format is `ppm`** (`pbm`/`pnm`/`rgb`/`bmp` are
|
||
rejected), which is what makes the encode/capture split measurable.
|
||
3. **`display.dispatch(block=False)` never reads the socket** — it processes only
|
||
events already read. A wait loop built on it plus `select` times out against a
|
||
perfectly healthy compositor. The read is `dispatch(block=True)`; the "never
|
||
hang" guarantee has to come from an alarm (`SIGALRM`), not from polling.
|
||
4. **`pywayland` dispatchers receive the proxy as their first argument** (four
|
||
parameters for a three-field event), and **`display.disconnect()` is
|
||
mandatory** or the process segfaults *after* printing — a green run that
|
||
reports exit 139 to any runner reading exit codes.
|
||
5. **turbojpeg reuses `*jpegBuf` when it is non-NULL.** Freeing it with `tjFree`
|
||
and leaving the pointer in place makes the next call write into freed memory:
|
||
a heap-corrupting segfault several frames later with no stack trace near the
|
||
encoder. Reset the pointer to NULL after freeing.
|
||
6. **`libjpeg_turbo`'s bare attribute resolves to its `-bin` output** in that
|
||
Nix shell — no shared library in it — and `wayland.xml` lives in the
|
||
**`wayland-scanner`** package, not in `wayland`. Both failures look like
|
||
missing dependencies rather than shell gaps.
|
||
7. **`ext-image-capture-source-v1` defines one manager per source kind**
|
||
(`ext_output_…` and `ext_foreign_toplevel_…`), not a generic
|
||
`ext_image_capture_source_manager_v1`. Looking for the generic name finds
|
||
nothing and reads as "the modern protocol is unavailable here" — a wrong
|
||
conclusion this spike drew once before checking the real global list. The
|
||
toplevel variant means **single-window capture is available** on this
|
||
compositor, which matters for agent eyes.
|
||
|
||
## Files
|
||
|
||
| File | What it is |
|
||
|---|---|
|
||
| `HARNESS.md` | the frozen measurement contract (argv, JSON, timing boundary, resource fields) |
|
||
| `bench.py` | the runner: every route through every cell, raw JSON kept, table derived from the files |
|
||
| `producers/grim_producer.py` | route 000, the baseline |
|
||
| `producers/wayland_producer.py` | route 001b, raw `pywayland` (screencopy *and* ext-image-copy-capture) |
|
||
| `producers/resources.py` | RAM/CPU sampling shared by the Python producers |
|
||
| `001a-grim-rs/` | route 001a, the Rust `grim-rs` producer |
|
||
| `results/` | every run's stdout/stderr, `results.json`, and `control.json` |
|
||
| `shell.nix` | the spike's own environment (pywayland, turbojpeg, rust) — deliberately not in the repo flake |
|
||
|
||
## Verdict
|
||
|
||
<!-- VERDICT-PLACEHOLDER -->
|