Wires a `checks` output into the flake, one file per check under checks/, all three asserting docs/CONTRACT.md rather than the implementation: - unit: pytest over tests/unit/ in the build sandbox, against the nix/package.nix build. Refuses an empty suite -- pytest exits 0 when every test was skipped or deselected, so checks/lib/check_report.py parses the JUnit report and fails on zero collected or zero executed. - vm-basic: nixosTest with captureCommand pointed at tests/fake_capture.py, which emits one fixed deterministic PNG. Because the image never changes, "two frames arrived and they are the same bytes" is an assertion about the framing rather than a coincidence. - vm-hyprland: the same machine with no captureCommand override, so grim captures the real compositor. Deliberately impatient -- bounded timeouts, no retry loops, and a full diagnostics dump (compositor log, hyprctl monitors, both journals) before any failure propagates. tests/assert_stream.py is a second, independent reader of the contract: it shares no code with hhstream.client, so a framing bug present in both the server's writer and the shipped client's reader still fails. The unit suite's negative tests use a `rejects` fixture instead of pytest.raises(Exception), which an unimplemented stub's NotImplementedError would otherwise satisfy. Three defects found in code this branch does not own, recorded in README.dev.md rather than worked around: - nix/module.nix has no config side, so neither unit exists and both VM checks fail at their first wait; - services.hyprland-headless-stream.package is declared with no default, against docs/CONTRACT.md; - src/hhstream/*.py is all stubs, so 66 of 70 unit tests fail. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017YXMoCNxzScTaKNr43GQNz
11 KiB
Developing and testing hyprland-headless-stream-plugin
The interface everything here asserts is docs/CONTRACT.md.
That document is authoritative: if a test and the code disagree, the document
decides, and if the document is wrong it gets a version bump, not an edit.
Prerequisites
- Nix with flakes enabled.
/dev/kvmreadable, for the two VM checks. They arenixosTests; without KVM they still run, just slowly enough that the timeouts below get tight.- No GPU. Nothing in the test suite needs one — the compositor runs on Hyprland's headless backend over software rendering.
Nothing in the checks touches the host: no /etc/nixos, no network, and no
socket outside the build sandbox or the test VM.
The dev shell
nix develop
Puts python3, pytest, hyprland, grim, ffmpeg, jq and tmux on PATH
and sets PYTHONPATH=$PWD/src, so import hhstream picks up the working tree
rather than an installed copy.
Build
nix build .#hhstream # the package (server + client)
ls result/bin # hhstream-server, hhstream-client
Both entry points currently exit with NotImplementedError — see "Known gaps"
at the bottom. The package builds; it does not yet run.
The checks
| Check | What it proves | Needs a VM |
|---|---|---|
unit |
the framing, the capture backend, the server's client handling, the client — in-process, against the nix/package.nix build |
no |
vm-basic |
the NixOS module brings up both units and publishes a contract-conformant socket, with a fake capture backend | yes |
vm-hyprland |
the same, capturing the real headless compositor through grim |
yes |
All three at once:
nix flake check -L
nix flake check stops at the first failing check, which makes it a poor way to
see the whole picture while the tree is still red. Prefer running them
individually — they are independent:
nix build -L .#checks.x86_64-linux.unit
nix build -L .#checks.x86_64-linux.vm-basic
nix build -L .#checks.x86_64-linux.vm-hyprland
-L is not optional in practice: without it, Nix hides the build log, and the
build log is the test report.
unit
pytest over tests/unit/, run in the build sandbox with PYTHONPATH pointing at
what nix/package.nix installed (nix/package.nix sets doCheck = false on
purpose, so this is the only place the suite runs in a build).
The check fails if the suite is empty, not just if it is red. pytest exits
5 on "collected nothing", but it exits 0 when every test was skipped or
deselected, so checks/lib/check_report.py parses the JUnit report and refuses
a run with zero collected or zero executed tests. A green check that tested
nothing is worse than no check. On success the report is copied to
result/junit.xml.
To run the same tests directly, without Nix:
nix develop
pytest tests/unit -v # PYTHONPATH=$PWD/src comes from the shell
pytest tests/unit/test_protocol.py -v # one file
pytest tests/unit -k "hello or frame" -v # one theme
pytest tests/unit -x --ff # stop at the first failure, failures first
Note that the unit suite's negative tests use the rejects fixture from
tests/unit/conftest.py rather than pytest.raises(Exception). rejects
treats NotImplementedError as a failure, because a plain raises(Exception)
is satisfied by an unimplemented stub — the test would then be green precisely
because nothing had been written.
vm-basic
Boots one machine with services.hyprland-headless-stream.enable = true and
captureCommand pointed at tests/fake_capture.py, which writes one fixed,
deterministic PNG of the configured size to stdout. Because the image never
changes, "two frames arrived and they are the same bytes" is a real assertion
about the framing rather than a coincidence.
It checks, in order: both units start; the socket exists and is
hhstream:hhstream mode 0660; the Hello parses with magic = "HHS1" and
non-zero width/height matching the configured output; two frames arrive with
length > 0, identical bytes, and an IHDR matching the Hello; two clients are
served concurrently; and a client that dies mid-frame leaves the server serving.
The reader is tests/assert_stream.py, a second, independent implementation of
the contract that shares no code with hhstream.client — so a framing bug
present in both the server's writer and the shipped client's reader still fails
the test. It validates the length prefix strictly (0|[1-9][0-9]*: no leading
zeros, no sign, no spaces), never reads past a newline, and prints the Hello,
every frame's size and SHA-256, and the parsed IHDR before it decides anything.
vm-hyprland
The same machine with no captureCommand override, so the module's default
(grim) captures the real compositor.
This is the check expected to be flaky, so it is deliberately impatient: every wait has a bounded timeout, nothing is retried in a loop, and any failure dumps diagnostics before it propagates. It asserts a frame is a valid PNG whose IHDR dimensions match the configured output, and that it is more than a blank buffer:
- the IDAT must decompress;
- the decompressed scanlines must contain a non-zero byte and at least two distinct byte values — a zero-filled framebuffer fails both;
- the payload must be larger than a best-case blank PNG of the same geometry
(
blank_png_size()intests/assert_stream.py: an all-zero raster at zlib level 9, i.e. the smallest a blank frame of that size can be encoded).
It also runs grim directly against the session as a separate subtest, which
separates "grim cannot see the compositor" from "the server is wrong" — the two
failures that look identical from the socket.
When vm-hyprland fails: where the logs are
Every failing subtest in both VM checks runs hhs-hyprland-diagnostics
(tests/hyprland_diagnostics.sh) in the guest first and prints its output to the
build log. So the answer to "where are the logs" is usually in the build log
you already have, above the traceback, between
########## diagnostics: ... ########## markers. Re-read it with:
nix log .#checks.x86_64-linux.vm-hyprland
What the dump contains, and where it comes from in the guest:
| Section | Source |
|---|---|
units |
systemctl list-units --all 'hyprland*' |
per-unit status / journalctl |
hyprland-headless.service and hyprland-headless-stream.service, 200 lines each |
socket and runtime directories |
/run/hyprland-headless-stream, /run/user/* |
hyprland logs |
every hyprland*.log found under /run, /var/log, /tmp, /home |
hyprctl monitors |
run once per instance found, with the right XDG_RUNTIME_DIR and HYPRLAND_INSTANCE_SIGNATURE |
processes |
ps aux filtered to hypr, hhstream, grim |
Hyprland writes its log to
$XDG_RUNTIME_DIR/hypr/$HYPRLAND_INSTANCE_SIGNATURE/hyprland.log, and the
compositor runs on a private XDG_RUNTIME_DIR, so the path is not knowable in
advance — that is why the script searches for it rather than naming it. The
instance signature is the directory name under <runtime>/hypr, which is also
what hyprctl needs in the environment:
XDG_RUNTIME_DIR=/run/user/$(id -u hhstream) \
HYPRLAND_INSTANCE_SIGNATURE=$(ls /run/user/$(id -u hhstream)/hypr | head -1) \
hyprctl monitors
Poking at a live VM
The nicest way to debug a VM failure is to keep the machine:
nix build .#checks.x86_64-linux.vm-hyprland.driverInteractive
./result/bin/nixos-test-driver
That drops you in a Python REPL with machine bound. start_all(), then
machine.shell_interact() for a root shell in the guest, or run the test script
a line at a time. machine.succeed("hhs-hyprland-diagnostics") works there too.
Layout
checks/
default.nix wiring only: the three checks the flake exposes
unit.nix pytest in the sandbox, against the nix/package.nix build
vm-basic.nix nixosTest, fake capture backend
vm-hyprland.nix nixosTest, real compositor + grim
lib/test-tools.nix packages the tests/ helpers for the guest
lib/check_report.py JUnit report -> build verdict (and the empty-suite guard)
tests/
fake_capture.py writes one fixed PNG to stdout: hhs-fake-capture
assert_stream.py independent contract reader/asserter: hhs-assert-stream
hyprland_diagnostics.sh the failure dump: hhs-hyprland-diagnostics
unit/ the pytest suite
checks/lib/test-tools.nix self-tests the two Python helpers at build time
(the fake emits a PNG of the size it was asked for; the asserter starts), because
a broken helper otherwise surfaces as a baffling failure three layers away.
Adding a unit test: drop a test_*.py in tests/unit/. Nothing needs
registering. Adding a check: one file in checks/, one line in
checks/default.nix.
Known gaps in the interface (as of this branch)
These are defects in code this branch does not own; they are recorded here because the tests are written against the contract and therefore trip over them.
-
nix/module.nixhas noconfigside yet. Neitherhyprland-headless.servicenorhyprland-headless-stream.serviceexists, so both VM checks fail at their first wait withUnit hyprland-headless.service could not be found.Nothing inchecks/works around this. -
services.hyprland-headless-stream.packageis declared with no default.docs/CONTRACT.mdsays the default is "the flake's own build", but the option has nodefault, so reading it fails witherror: The option `services.hyprland-headless-stream.package' was accessed but has no value defined. Try setting the option.Today that only bites code that reads
cfg.package, which the unwrittenconfigside does not yet do; it will bite every consumer that sets onlyenable = truethe moment it does. Both VM checks setpackageexplicitly. -
src/hhstream/*.pyis stubs. Every function raisesNotImplementedError, so 66 of the 70 unit tests fail. The 4 that pass aretests/unit/test_package.py, which only reads constants — they are the canary for theunitcheck's own wiring. -
The frame server's command line is unspecified.
docs/CONTRACT.mdfixes the module's option surface and the wire format but never says how the module invokeshhstream-server.tests/unit/test_server.py::test_build_parser_exposes_the_options_the_module_needscurrently assumes--socket PATH --fps N --format png -- <captureCommand...>. That is a guess; if the module ends up passing something else, that one test is the thing to change, not the server.