Files
HermenixandClaude Opus 5 75a05cd72f tests: checks.{unit,vm-basic,vm-hyprland} + the harness behind them
Wires a `checks` output into the flake, one file per check under checks/,
all three asserting docs/CONTRACT.md rather than the implementation:

- unit: pytest over tests/unit/ in the build sandbox, against the
  nix/package.nix build. Refuses an empty suite -- pytest exits 0 when
  every test was skipped or deselected, so checks/lib/check_report.py
  parses the JUnit report and fails on zero collected or zero executed.
- vm-basic: nixosTest with captureCommand pointed at tests/fake_capture.py,
  which emits one fixed deterministic PNG. Because the image never changes,
  "two frames arrived and they are the same bytes" is an assertion about
  the framing rather than a coincidence.
- vm-hyprland: the same machine with no captureCommand override, so grim
  captures the real compositor. Deliberately impatient -- bounded timeouts,
  no retry loops, and a full diagnostics dump (compositor log, hyprctl
  monitors, both journals) before any failure propagates.

tests/assert_stream.py is a second, independent reader of the contract: it
shares no code with hhstream.client, so a framing bug present in both the
server's writer and the shipped client's reader still fails. The unit
suite's negative tests use a `rejects` fixture instead of
pytest.raises(Exception), which an unimplemented stub's
NotImplementedError would otherwise satisfy.

Three defects found in code this branch does not own, recorded in
README.dev.md rather than worked around:
- nix/module.nix has no config side, so neither unit exists and both VM
  checks fail at their first wait;
- services.hyprland-headless-stream.package is declared with no default,
  against docs/CONTRACT.md;
- src/hhstream/*.py is all stubs, so 66 of 70 unit tests fail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017YXMoCNxzScTaKNr43GQNz
2026-09-28 05:25:37 +00:00

242 lines
11 KiB
Markdown

# Developing and testing `hyprland-headless-stream-plugin`
The interface everything here asserts is [`docs/CONTRACT.md`](docs/CONTRACT.md).
That document is authoritative: if a test and the code disagree, the document
decides, and if the document is wrong it gets a version bump, not an edit.
## Prerequisites
- Nix with flakes enabled.
- `/dev/kvm` readable, for the two VM checks. They are `nixosTest`s; without
KVM they still run, just slowly enough that the timeouts below get tight.
- No GPU. Nothing in the test suite needs one — the compositor runs on
Hyprland's headless backend over software rendering.
Nothing in the checks touches the host: no `/etc/nixos`, no network, and no
socket outside the build sandbox or the test VM.
## The dev shell
```bash
nix develop
```
Puts `python3`, `pytest`, `hyprland`, `grim`, `ffmpeg`, `jq` and `tmux` on PATH
and sets `PYTHONPATH=$PWD/src`, so `import hhstream` picks up the working tree
rather than an installed copy.
## Build
```bash
nix build .#hhstream # the package (server + client)
ls result/bin # hhstream-server, hhstream-client
```
Both entry points currently exit with `NotImplementedError` — see "Known gaps"
at the bottom. The package builds; it does not yet run.
## The checks
| Check | What it proves | Needs a VM |
|---|---|---|
| `unit` | the framing, the capture backend, the server's client handling, the client — in-process, against the `nix/package.nix` build | no |
| `vm-basic` | the NixOS module brings up both units and publishes a contract-conformant socket, with a **fake** capture backend | yes |
| `vm-hyprland` | the same, capturing the **real** headless compositor through `grim` | yes |
All three at once:
```bash
nix flake check -L
```
`nix flake check` stops at the first failing check, which makes it a poor way to
see the whole picture while the tree is still red. Prefer running them
individually — they are independent:
```bash
nix build -L .#checks.x86_64-linux.unit
nix build -L .#checks.x86_64-linux.vm-basic
nix build -L .#checks.x86_64-linux.vm-hyprland
```
`-L` is not optional in practice: without it, Nix hides the build log, and the
build log *is* the test report.
### `unit`
pytest over `tests/unit/`, run in the build sandbox with `PYTHONPATH` pointing at
what `nix/package.nix` installed (`nix/package.nix` sets `doCheck = false` on
purpose, so this is the only place the suite runs in a build).
The check fails if the suite is **empty**, not just if it is red. `pytest` exits
5 on "collected nothing", but it exits 0 when every test was skipped or
deselected, so `checks/lib/check_report.py` parses the JUnit report and refuses
a run with zero collected or zero executed tests. A green check that tested
nothing is worse than no check. On success the report is copied to
`result/junit.xml`.
To run the same tests directly, without Nix:
```bash
nix develop
pytest tests/unit -v # PYTHONPATH=$PWD/src comes from the shell
pytest tests/unit/test_protocol.py -v # one file
pytest tests/unit -k "hello or frame" -v # one theme
pytest tests/unit -x --ff # stop at the first failure, failures first
```
Note that the unit suite's *negative* tests use the `rejects` fixture from
`tests/unit/conftest.py` rather than `pytest.raises(Exception)`. `rejects`
treats `NotImplementedError` as a failure, because a plain `raises(Exception)`
is satisfied by an unimplemented stub — the test would then be green precisely
because nothing had been written.
### `vm-basic`
Boots one machine with `services.hyprland-headless-stream.enable = true` and
`captureCommand` pointed at `tests/fake_capture.py`, which writes one fixed,
deterministic PNG of the configured size to stdout. Because the image never
changes, "two frames arrived and they are the *same bytes*" is a real assertion
about the framing rather than a coincidence.
It checks, in order: both units start; the socket exists and is
`hhstream:hhstream` mode `0660`; the Hello parses with `magic = "HHS1"` and
non-zero width/height matching the configured output; two frames arrive with
length > 0, identical bytes, and an IHDR matching the Hello; two clients are
served concurrently; and a client that dies mid-frame leaves the server serving.
The reader is `tests/assert_stream.py`, a second, independent implementation of
the contract that shares no code with `hhstream.client` — so a framing bug
present in both the server's writer and the shipped client's reader still fails
the test. It validates the length prefix strictly (`0|[1-9][0-9]*`: no leading
zeros, no sign, no spaces), never reads past a newline, and prints the Hello,
every frame's size and SHA-256, and the parsed IHDR before it decides anything.
### `vm-hyprland`
The same machine with **no** `captureCommand` override, so the module's default
(`grim`) captures the real compositor.
This is the check expected to be flaky, so it is deliberately impatient: every
wait has a bounded timeout, nothing is retried in a loop, and any failure dumps
diagnostics before it propagates. It asserts a frame is a valid PNG whose IHDR
dimensions match the configured output, and that it is more than a blank buffer:
- the IDAT must decompress;
- the decompressed scanlines must contain a non-zero byte and at least two
distinct byte values — a zero-filled framebuffer fails both;
- the payload must be larger than a best-case blank PNG of the same geometry
(`blank_png_size()` in `tests/assert_stream.py`: an all-zero raster at zlib
level 9, i.e. the smallest a blank frame of that size can be encoded).
It also runs `grim` directly against the session as a separate subtest, which
separates "grim cannot see the compositor" from "the server is wrong" — the two
failures that look identical from the socket.
## When `vm-hyprland` fails: where the logs are
Every failing subtest in both VM checks runs `hhs-hyprland-diagnostics`
(`tests/hyprland_diagnostics.sh`) in the guest first and prints its output to the
build log. So the answer to "where are the logs" is usually **in the build log
you already have**, above the traceback, between
`########## diagnostics: ... ##########` markers. Re-read it with:
```bash
nix log .#checks.x86_64-linux.vm-hyprland
```
What the dump contains, and where it comes from in the guest:
| Section | Source |
|---|---|
| `units` | `systemctl list-units --all 'hyprland*'` |
| per-unit `status` / `journalctl` | `hyprland-headless.service` and `hyprland-headless-stream.service`, 200 lines each |
| `socket and runtime directories` | `/run/hyprland-headless-stream`, `/run/user/*` |
| `hyprland logs` | every `hyprland*.log` found under `/run`, `/var/log`, `/tmp`, `/home` |
| `hyprctl monitors` | run once per instance found, with the right `XDG_RUNTIME_DIR` and `HYPRLAND_INSTANCE_SIGNATURE` |
| `processes` | `ps aux` filtered to `hypr`, `hhstream`, `grim` |
Hyprland writes its log to
`$XDG_RUNTIME_DIR/hypr/$HYPRLAND_INSTANCE_SIGNATURE/hyprland.log`, and the
compositor runs on a *private* `XDG_RUNTIME_DIR`, so the path is not knowable in
advance — that is why the script searches for it rather than naming it. The
instance signature is the directory name under `<runtime>/hypr`, which is also
what `hyprctl` needs in the environment:
```bash
XDG_RUNTIME_DIR=/run/user/$(id -u hhstream) \
HYPRLAND_INSTANCE_SIGNATURE=$(ls /run/user/$(id -u hhstream)/hypr | head -1) \
hyprctl monitors
```
### Poking at a live VM
The nicest way to debug a VM failure is to keep the machine:
```bash
nix build .#checks.x86_64-linux.vm-hyprland.driverInteractive
./result/bin/nixos-test-driver
```
That drops you in a Python REPL with `machine` bound. `start_all()`, then
`machine.shell_interact()` for a root shell in the guest, or run the test script
a line at a time. `machine.succeed("hhs-hyprland-diagnostics")` works there too.
## Layout
```
checks/
default.nix wiring only: the three checks the flake exposes
unit.nix pytest in the sandbox, against the nix/package.nix build
vm-basic.nix nixosTest, fake capture backend
vm-hyprland.nix nixosTest, real compositor + grim
lib/test-tools.nix packages the tests/ helpers for the guest
lib/check_report.py JUnit report -> build verdict (and the empty-suite guard)
tests/
fake_capture.py writes one fixed PNG to stdout: hhs-fake-capture
assert_stream.py independent contract reader/asserter: hhs-assert-stream
hyprland_diagnostics.sh the failure dump: hhs-hyprland-diagnostics
unit/ the pytest suite
```
`checks/lib/test-tools.nix` self-tests the two Python helpers at build time
(the fake emits a PNG of the size it was asked for; the asserter starts), because
a broken helper otherwise surfaces as a baffling failure three layers away.
Adding a unit test: drop a `test_*.py` in `tests/unit/`. Nothing needs
registering. Adding a check: one file in `checks/`, one line in
`checks/default.nix`.
## Known gaps in the interface (as of this branch)
These are defects in code this branch does not own; they are recorded here
because the tests are written against the contract and therefore trip over them.
1. **`nix/module.nix` has no `config` side yet.** Neither
`hyprland-headless.service` nor `hyprland-headless-stream.service` exists, so
both VM checks fail at their first wait with
`Unit hyprland-headless.service could not be found.` Nothing in `checks/`
works around this.
2. **`services.hyprland-headless-stream.package` is declared with no default.**
`docs/CONTRACT.md` says the default is "the flake's own build", but the option
has no `default`, so reading it fails with
```
error: The option `services.hyprland-headless-stream.package' was accessed but
has no value defined. Try setting the option.
```
Today that only bites code that reads `cfg.package`, which the unwritten
`config` side does not yet do; it will bite every consumer that sets only
`enable = true` the moment it does. Both VM checks set `package` explicitly.
3. **`src/hhstream/*.py` is stubs.** Every function raises
`NotImplementedError`, so 66 of the 70 unit tests fail. The 4 that pass are
`tests/unit/test_package.py`, which only reads constants — they are the
canary for the `unit` check's own wiring.
4. **The frame server's command line is unspecified.** `docs/CONTRACT.md` fixes
the module's option surface and the wire format but never says how the module
invokes `hhstream-server`. `tests/unit/test_server.py::test_build_parser_exposes_the_options_the_module_needs`
currently assumes `--socket PATH --fps N --format png -- <captureCommand...>`.
That is a guess; if the module ends up passing something else, that one test
is the thing to change, not the server.