capture: an in-process Wayland backend, and the measurements that justify it
The frame server could only capture by running a program once per frame
(grim). That works, but ~95% of a grim frame is waiting -- fork, connect,
handshake, round trip -- not encoding, and grim makes the compositor import
a *new* shared-memory buffer for every single frame.
This adds `--capture wayland[:OUTPUT]`: one long-lived zwlr-screencopy-v1
client per server, with one shm buffer reused for the whole run, capturing
inside the process. Measured here at 1280x720 (spike capture-speed, 200
frames; live re-check on Virtual-1, 60 frames):
grim (subprocess) 117.3 ms/frame 8.1 fps compositor 116.4 ms
grim-rs (Rust crate) 124.6 ms/frame 7.5 fps compositor 123.1 ms
this backend 78.0 ms/frame 13.9 fps compositor 67.5 ms
live, PNG both sides: this 128.5 ms/frame against grim's 389.3 ms (3.0x)
Three findings, each of which shaped the code:
* The win is buffer reuse, not language. grim and grim-rs both hand the
compositor a fresh buffer per frame, which is why their compositor CPU is
nearly double ours. The per-frame allocate-copy-swizzle that grim-rs pays
was measured in isolation at 2.4 ms of *client* time; the rest of its
deficit lands inside the compositor.
* ext-image-copy-capture-v1 is no faster (78.0 vs 77.7 ms) and its
create_frame path appears in a Hyprland segfault stack, so this keeps the
deprecated protocol grim uses. The switch is one constant when that crash
is fixed.
* PNG only, deliberately: PNG is the one encoding this package produces
without a dependency, so --format jpeg is refused rather than publishing a
Hello whose MIME type the frames do not have. A PNG frame is ~2.1 MB
against ~120 KB for JPEG q80, which is why the command backend stays the
default and why a libjpeg-turbo encoder (1.7 ms/frame, measured) is the
obvious follow-up.
The timeout is the load-bearing part of the client: pywayland only reads
events from read()/dispatch(block=True), neither of which takes a timeout,
and the server captures in a worker thread where SIGALRM is unavailable. It
therefore polls the display fd itself, so a compositor that stops answering
produces a CaptureError instead of a capture thread that never returns.
The protocol bindings are vendored under src/hhstream/_protocol/ exactly the
way hhstream_pointer already vendors its two, with regenerate.py recording
where each XML came from. No new dependency: pywayland was already required
by the input driver.
Contract: the new server flag and the two new module options are documented
in docs/CONTRACT.md, and docs/ARCHITECTURE.md's prediction that a native
screencopy client would be "a swap rather than a rewrite" is now the record
of what the swap cost.
Unit suite: 977 passed, 2 skipped, 0 failed (nix build -L .#checks.unit).
The live numbers were taken on this VM against the real compositor.
This commit is contained in:
1 parent
f3ac9ada27
commit
a7d9357b43
37 files changed
+7556
-10
No files matched your search
+22
-2
@@ -87,8 +87,28 @@ Capturing is a separate program (by default `grim`, a wlr-screencopy client) run
|
||||
inside the compositor's session and pointed at stdout. That choice is what keeps
|
||||
`hhstream` free of Wayland and GPU code — the awkward part lives in a tool that
|
||||
already exists and is already tested — and it is why the capture command is a
|
||||
module option: tests inject a fake, and a future backend (a native screencopy
|
||||
client, a hardware encoder) is a command-line swap rather than a rewrite.
|
||||
module option: tests inject a fake, and a different backend is a swap rather than
|
||||
a rewrite.
|
||||
|
||||
That swap has now happened once. `captureBackend = "wayland"` captures in-process
|
||||
(`src/hhstream/wayland_capture.py`): one long-lived `zwlr-screencopy-v1` client
|
||||
with **one shared-memory buffer reused for every frame**, instead of a `grim`
|
||||
process per frame. Measured at 1280×720 over 200 frames, that is ~1.5× the frame
|
||||
rate (78 ms against 117 ms per frame) and — the number that matters on this
|
||||
machine — roughly **half the compositor's CPU per frame** (67 ms against 116 ms),
|
||||
because `grim` makes the compositor import a new buffer every time. It is
|
||||
deliberately the *deprecated* protocol: `ext-image-copy-capture-v1` measured
|
||||
identically and appears in a Hyprland segfault stack (`create_frame` →
|
||||
`CScreenshareFrame::transform`). The price of the in-process route is that the
|
||||
package now carries Wayland protocol bindings of its own (`src/hhstream/_protocol/`,
|
||||
vendored the same way `hhstream_pointer` already vendors its two) and that it can
|
||||
only publish **PNG**, since PNG is the one format it can encode without a
|
||||
dependency. Both costs are why `command` is still the default.
|
||||
|
||||
Comments in the spike that produced these numbers live in the branch's
|
||||
`capture-speed` worktree; the honest caveat that came with them is that this box
|
||||
is saturated by *rendering* whatever is on screen, so the win is in how much room
|
||||
the compositor keeps, not in headroom that did not exist.
|
||||
|
||||
## The hardware requirement, stated plainly
|
||||
|
||||
|
||||
+11
-2
@@ -93,7 +93,9 @@ Namespace: `services.hyprland-headless-stream`
|
||||
| `extraConfig` | str | `""` | extra lines appended to `hyprland.conf` |
|
||||
| `hyprlandPackage` | package | `pkgs.hyprland` | which compositor |
|
||||
| `package` | package | the flake's own build | the frame server/client |
|
||||
| `captureCommand` | list of str | `[ "grim" … ]` | override the capture backend — used by tests |
|
||||
| `captureCommand` | list of str | `[ "grim" … ]` | override the capture *command* — used by tests |
|
||||
| `captureBackend` | `"command"` \| `"wayland"` | `"command"` | capture by command or in-process (`zwlr-screencopy-v1`); `wayland` needs `format = "png"` |
|
||||
| `captureOutput` | str or null | `null` | output to capture by name when `captureBackend = "wayland"`; null = the first one |
|
||||
|
||||
Services (names are part of the contract, tests refer to them):
|
||||
|
||||
@@ -109,7 +111,7 @@ status as the wire format. Exactly this shape, and no others:
|
||||
|
||||
```
|
||||
hhstream-server --socket <socketPath> --fps <frameRate> --format <png|jpeg> \
|
||||
[--capture-command <token> ...]
|
||||
[--capture-command <token> ...] [--capture wayland[:NAME]]
|
||||
```
|
||||
|
||||
- `--format` takes the **short name** (`png`, `jpeg`), not a MIME type; the
|
||||
@@ -119,6 +121,13 @@ hhstream-server --socket <socketPath> --fps <frameRate> --format <png|jpeg> \
|
||||
--capture-command -`. A single shell string cannot carry a leading `-`
|
||||
unambiguously. The `--capture-command=<token>` form is accepted as well, and
|
||||
the two may be mixed. When absent, the server derives `grim -t <format> -`.
|
||||
- `--capture` selects a **named backend** instead of a program: `wayland`
|
||||
captures the compositor's first output in-process, `wayland:NAME` the output
|
||||
Hyprland calls `NAME`. It is exclusive with `--capture-command` (the backend
|
||||
replaces the command; passing both is a caller error). It requires
|
||||
`--format png`, and the server refuses `jpeg` rather than publish a Hello whose
|
||||
MIME type the frames do not have — the in-process client can encode PNG without
|
||||
a dependency and nothing else.
|
||||
- There is deliberately **no** `--width` / `--height`: the Hello reports the
|
||||
captured frame's real size (see above).
|
||||
- The socket's **mode** (0660) is the server's business; its **ownership** is
|
||||
|
||||
Reference in new issue
Block a user