PNG-only was a real defect, not a nice-to-have: `hhstream_live.server` serves
`multipart/x-mixed-replace` with `image/jpeg` parts and its own capture default is
`grim -c -t jpeg -q 80 -`, so switching this backend on would have broken video
streaming while looking faster in a benchmark.
JPEG now goes through libjpeg-turbo by ctypes, using its `TJPF_BGRX` input format
-- which is exactly the compositor's B, G, R, X memory order, so the frame goes
into the encoder with no copy and no colour conversion. Measured on this VM
against the real compositor, the deployment's own settings (Virtual-1, 1280x720,
jpeg q80, 200 frames each, two runs):
p50 ms fps our CPU compositor CPU
in-process 76.2 13.1 2.1 ms 51.5 ms
grim -c -t jpeg 137.0 7.3 0.5 ms 80.2 ms
(second run: 84/144,
55.1/79.8 -> 1.71x)
So ~1.7-1.8x the frame rate and ~1.5x less of the compositor's CPU, with frames
of the same order (188 KB against grim's 222 KB). PNG stays available through the
stdlib for a deployment that would rather not carry the library.
Both segfault-class traps from the spike are handled and documented: every
argtype is declared (an undeclared pitch is a crash, not a wrong number), and
`*jpegBuf` is reset to NULL after `tjFree` (turbojpeg reads a non-NULL pointer as
a caller buffer, so the next frame would write into freed memory). The decoder is
here because a lossy codec can only be tested by round-tripping pixels -- the new
suite encodes solid blocks whose channels differ enough that a swap is
unmistakable, and includes a padded stride.
Packaging: libjpeg-turbo is reached through ctypes, so nothing links it and the
path must reach the loader -- the console scripts get a wrapper with it on
LD_LIBRARY_PATH, and the unit check exports it too. A missing library is reported
at startup rather than once per frame: a server that is up but can never produce
a frame is worse than one that refuses to start.
Verified end to end: server with `--capture wayland:Virtual-1 --format jpeg`,
client Hello `image/jpeg`, three frames read, each magic `ffd8`.
The frame server could only capture by running a program once per frame
(grim). That works, but ~95% of a grim frame is waiting -- fork, connect,
handshake, round trip -- not encoding, and grim makes the compositor import
a *new* shared-memory buffer for every single frame.
This adds `--capture wayland[:OUTPUT]`: one long-lived zwlr-screencopy-v1
client per server, with one shm buffer reused for the whole run, capturing
inside the process. Measured here at 1280x720 (spike capture-speed, 200
frames; live re-check on Virtual-1, 60 frames):
grim (subprocess) 117.3 ms/frame 8.1 fps compositor 116.4 ms
grim-rs (Rust crate) 124.6 ms/frame 7.5 fps compositor 123.1 ms
this backend 78.0 ms/frame 13.9 fps compositor 67.5 ms
live, PNG both sides: this 128.5 ms/frame against grim's 389.3 ms (3.0x)
Three findings, each of which shaped the code:
* The win is buffer reuse, not language. grim and grim-rs both hand the
compositor a fresh buffer per frame, which is why their compositor CPU is
nearly double ours. The per-frame allocate-copy-swizzle that grim-rs pays
was measured in isolation at 2.4 ms of *client* time; the rest of its
deficit lands inside the compositor.
* ext-image-copy-capture-v1 is no faster (78.0 vs 77.7 ms) and its
create_frame path appears in a Hyprland segfault stack, so this keeps the
deprecated protocol grim uses. The switch is one constant when that crash
is fixed.
* PNG only, deliberately: PNG is the one encoding this package produces
without a dependency, so --format jpeg is refused rather than publishing a
Hello whose MIME type the frames do not have. A PNG frame is ~2.1 MB
against ~120 KB for JPEG q80, which is why the command backend stays the
default and why a libjpeg-turbo encoder (1.7 ms/frame, measured) is the
obvious follow-up.
The timeout is the load-bearing part of the client: pywayland only reads
events from read()/dispatch(block=True), neither of which takes a timeout,
and the server captures in a worker thread where SIGALRM is unavailable. It
therefore polls the display fd itself, so a compositor that stops answering
produces a CaptureError instead of a capture thread that never returns.
The protocol bindings are vendored under src/hhstream/_protocol/ exactly the
way hhstream_pointer already vendors its two, with regenerate.py recording
where each XML came from. No new dependency: pywayland was already required
by the input driver.
Contract: the new server flag and the two new module options are documented
in docs/CONTRACT.md, and docs/ARCHITECTURE.md's prediction that a native
screencopy client would be "a swap rather than a rewrite" is now the record
of what the swap cost.
Unit suite: 977 passed, 2 skipped, 0 failed (nix build -L .#checks.unit).
The live numbers were taken on this VM against the real compositor.
grim's `-c` ("include cursors in the screenshot") exists because the screencopy
protocol takes the decision from the client: Hyprland's capture_output carries
an overlayCursor argument and the frame is only composited with the cursor when
it is set. Without it a frame shows that *something* changed and never where it
was clicked -- and a click on the wrong element looks exactly like a click on
the right one, which is the whole reason the pointer work is verifiable at all.
Found by reading Hyprland's Screencopy.cpp rather than guessing at a compositor
setting: `no_hardware_cursors` was already on, and the cursor was invisible
because nobody had asked for it.
Slice A renamed the runtime directory in the module, but the old path was still
live in the code and checks it does not own, so the VM gates pointed at a
directory the module no longer creates. This moves the 21 remaining live
occurrences (client/server defaults, the unit tests that pin them, all three VM
checks, the diagnostics script, both READMEs) and deliberately leaves the three
mentions that are history rather than configuration: the plan's problem
statement, module.nix's note about the previous name, and cli.py's explanation
for spelling its default out.
Also here: docs/CONTRACT-sessions.md is now v2 with every gap the four slices
reported resolved as a decision (Monitor.id, stable_id, pid -1, Event.data,
mark(), the quarantine argv type, the corrected wlr XML path, and the state
dir/frame socket rows), and docs/CONTRACT.md is v2 because its documented socket
default moved -- wire format unchanged.
Two comments that stated the old, wrong diagnosis (a 'misnamed' IPC socket) now
state the measured one: the event socket did not fit in sun_path.
Fill in the bodies the skeleton left as NotImplementedError, to docs/CONTRACT.md:
- protocol: the `\n`-terminated JSON Hello and the `<ascii length>\n<payload>`
frame, both directions. Malformed input raises (ValueError for a bad length
prefix, EOFError for a truncated payload), never hangs on endless garbage and
never silently truncates; a huge length prefix is not pre-allocated.
- capture: CommandCapture runs the configured command and reports a missing
binary, a non-zero exit, a timeout and empty output as CaptureError, with the
helper's stderr folded into the message so nothing leaks to the server's own
stdio. The Hello's pixel size is read from the frame's PNG/JPEG header, so no
compositor query is needed and an injected fake command behaves the same.
- server: one capture thread holding only the newest frame plus one thread per
client, which is what drops frames for a slow client instead of queueing
them. Survives a client dying mid-write, keeps the socket up across capture
failures, withholds the Hello until a size is known, replaces a stale socket
file (but never a non-socket one) and publishes it mode 0660.
- client: fetch()/read_stream() and the CLI, with --out suffixing per frame.
--capture-command folds its separated form into `--flag=value` before argparse
sees it, so the contract's default `grim -t png -` can actually be passed: its
tokens start with a dash.
Unit tests under tests/unit cover framing round-trips (including a payload
containing `\n` and a zero-length frame), truncated and garbage prefixes, a
stalled client not blocking a second one, a client dying mid-frame, frames being
dropped rather than queued, and every capture failure mode. A fake capture
command stands in for grim, so nothing needs a compositor or a GPU. They are
deliberately not wired into flake.nix: `checks` belongs to another workstream.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FYJ8ZQ5AZaDErYTwXZe7N9
Wires a `checks` output into the flake, one file per check under checks/,
all three asserting docs/CONTRACT.md rather than the implementation:
- unit: pytest over tests/unit/ in the build sandbox, against the
nix/package.nix build. Refuses an empty suite -- pytest exits 0 when
every test was skipped or deselected, so checks/lib/check_report.py
parses the JUnit report and fails on zero collected or zero executed.
- vm-basic: nixosTest with captureCommand pointed at tests/fake_capture.py,
which emits one fixed deterministic PNG. Because the image never changes,
"two frames arrived and they are the same bytes" is an assertion about
the framing rather than a coincidence.
- vm-hyprland: the same machine with no captureCommand override, so grim
captures the real compositor. Deliberately impatient -- bounded timeouts,
no retry loops, and a full diagnostics dump (compositor log, hyprctl
monitors, both journals) before any failure propagates.
tests/assert_stream.py is a second, independent reader of the contract: it
shares no code with hhstream.client, so a framing bug present in both the
server's writer and the shipped client's reader still fails. The unit
suite's negative tests use a `rejects` fixture instead of
pytest.raises(Exception), which an unimplemented stub's
NotImplementedError would otherwise satisfy.
Three defects found in code this branch does not own, recorded in
README.dev.md rather than worked around:
- nix/module.nix has no config side, so neither unit exists and both VM
checks fail at their first wait;
- services.hyprland-headless-stream.package is declared with no default,
against docs/CONTRACT.md;
- src/hhstream/*.py is all stubs, so 66 of 70 unit tests fail.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017YXMoCNxzScTaKNr43GQNz