capture: encode JPEG in-process too, so the live stream keeps working

PNG-only was a real defect, not a nice-to-have: `hhstream_live.server` serves
`multipart/x-mixed-replace` with `image/jpeg` parts and its own capture default is
`grim -c -t jpeg -q 80 -`, so switching this backend on would have broken video
streaming while looking faster in a benchmark.

JPEG now goes through libjpeg-turbo by ctypes, using its `TJPF_BGRX` input format
-- which is exactly the compositor's B, G, R, X memory order, so the frame goes
into the encoder with no copy and no colour conversion. Measured on this VM
against the real compositor, the deployment's own settings (Virtual-1, 1280x720,
jpeg q80, 200 frames each, two runs):

                        p50 ms   fps    our CPU   compositor CPU
    in-process           76.2   13.1     2.1 ms      51.5 ms
    grim -c -t jpeg     137.0    7.3     0.5 ms      80.2 ms
                                                          (second run: 84/144,
                                                           55.1/79.8 -> 1.71x)

So ~1.7-1.8x the frame rate and ~1.5x less of the compositor's CPU, with frames
of the same order (188 KB against grim's 222 KB). PNG stays available through the
stdlib for a deployment that would rather not carry the library.

Both segfault-class traps from the spike are handled and documented: every
argtype is declared (an undeclared pitch is a crash, not a wrong number), and
`*jpegBuf` is reset to NULL after `tjFree` (turbojpeg reads a non-NULL pointer as
a caller buffer, so the next frame would write into freed memory). The decoder is
here because a lossy codec can only be tested by round-tripping pixels -- the new
suite encodes solid blocks whose channels differ enough that a swap is
unmistakable, and includes a padded stride.

Packaging: libjpeg-turbo is reached through ctypes, so nothing links it and the
path must reach the loader -- the console scripts get a wrapper with it on
LD_LIBRARY_PATH, and the unit check exports it too. A missing library is reported
at startup rather than once per frame: a server that is up but can never produce
a frame is worse than one that refuses to start.

Verified end to end: server with `--capture wayland:Virtual-1 --format jpeg`,
client Hello `image/jpeg`, three frames read, each magic `ffd8`.
This commit is contained in:
Hermenix committed 2026-10-11 22:58:28 +00:00
1 parent a813e0b4f8
commit 7190960120
10 files changed
+553 -53

No files matched your search

+6 -2
View File
@@ -102,8 +102,12 @@ identically and appears in a Hyprland segfault stack (`create_frame` →
`CScreenshareFrame::transform`). The price of the in-process route is that the
package now carries Wayland protocol bindings of its own (`src/hhstream/_protocol/`,
vendored the same way `hhstream_pointer` already vendors its two) and that it can
only publish **PNG**, since PNG is the one format it can encode without a
dependency. Both costs are why `command` is still the default.
publish **JPEG** (through libjpeg-turbo) as well as PNG — and JPEG is the one
that matters, because the live viewer serves `image/jpeg` parts and a PNG-only
backend would have broken video streaming while looking faster on a benchmark.
The library is reached through `ctypes` and has to be on `LD_LIBRARY_PATH`, which
the package's wrapper arranges. Both costs are why `command` is still the default:
switching is the operator's call, not something a merge should decide.
Comments in the spike that produced these numbers live in the branch's
`capture-speed` worktree; the honest caveat that came with them is that this box
+8 -5
View File
@@ -94,7 +94,7 @@ Namespace: `services.hyprland-headless-stream`
| `hyprlandPackage` | package | `pkgs.hyprland` | which compositor |
| `package` | package | the flake's own build | the frame server/client |
| `captureCommand` | list of str | `[ "grim" … ]` | override the capture *command* — used by tests |
| `captureBackend` | `"command"` \| `"wayland"` | `"command"` | capture by command or in-process (`zwlr-screencopy-v1`); `wayland` needs `format = "png"` |
| `captureBackend` | `"command"` \| `"wayland"` | `"command"` | capture by command or in-process (`zwlr-screencopy-v1`); both `format` values work |
| `captureOutput` | str or null | `null` | output to capture by name when `captureBackend = "wayland"`; null = the first one |
Services (names are part of the contract, tests refer to them):
@@ -111,6 +111,7 @@ status as the wire format. Exactly this shape, and no others:
```
hhstream-server --socket <socketPath> --fps <frameRate> --format <png|jpeg> \
[--capture-quality <1..100>] \
[--capture-command <token> ...] [--capture wayland[:NAME]]
```
@@ -124,10 +125,12 @@ hhstream-server --socket <socketPath> --fps <frameRate> --format <png|jpeg> \
- `--capture` selects a **named backend** instead of a program: `wayland`
captures the compositor's first output in-process, `wayland:NAME` the output
Hyprland calls `NAME`. It is exclusive with `--capture-command` (the backend
replaces the command; passing both is a caller error). It requires
`--format png`, and the server refuses `jpeg` rather than publish a Hello whose
MIME type the frames do not have — the in-process client can encode PNG without
a dependency and nothing else.
replaces the command; passing both is a caller error). It honours both
`--format` values: `png` through the stdlib, `jpeg` through libjpeg-turbo. If
libjpeg-turbo is missing and `--format jpeg` was asked for, the server **refuses
to start** rather than serving a socket it can never produce a frame for.
- `--capture-quality` is the JPEG quality for the in-process backend (1..100,
default 80, matching the default command's `-q 80`); `--format png` ignores it.
- There is deliberately **no** `--width` / `--height`: the Hello reports the
captured frame's real size (see above).
- The socket's **mode** (0660) is the server's business; its **ownership** is