Files
HermenixandClaude Opus 5 75a05cd72f tests: checks.{unit,vm-basic,vm-hyprland} + the harness behind them
Wires a `checks` output into the flake, one file per check under checks/,
all three asserting docs/CONTRACT.md rather than the implementation:

- unit: pytest over tests/unit/ in the build sandbox, against the
  nix/package.nix build. Refuses an empty suite -- pytest exits 0 when
  every test was skipped or deselected, so checks/lib/check_report.py
  parses the JUnit report and fails on zero collected or zero executed.
- vm-basic: nixosTest with captureCommand pointed at tests/fake_capture.py,
  which emits one fixed deterministic PNG. Because the image never changes,
  "two frames arrived and they are the same bytes" is an assertion about
  the framing rather than a coincidence.
- vm-hyprland: the same machine with no captureCommand override, so grim
  captures the real compositor. Deliberately impatient -- bounded timeouts,
  no retry loops, and a full diagnostics dump (compositor log, hyprctl
  monitors, both journals) before any failure propagates.

tests/assert_stream.py is a second, independent reader of the contract: it
shares no code with hhstream.client, so a framing bug present in both the
server's writer and the shipped client's reader still fails. The unit
suite's negative tests use a `rejects` fixture instead of
pytest.raises(Exception), which an unimplemented stub's
NotImplementedError would otherwise satisfy.

Three defects found in code this branch does not own, recorded in
README.dev.md rather than worked around:
- nix/module.nix has no config side, so neither unit exists and both VM
  checks fail at their first wait;
- services.hyprland-headless-stream.package is declared with no default,
  against docs/CONTRACT.md;
- src/hhstream/*.py is all stubs, so 66 of 70 unit tests fail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017YXMoCNxzScTaKNr43GQNz
2026-09-28 05:25:37 +00:00

11 KiB

Developing and testing hyprland-headless-stream-plugin

The interface everything here asserts is docs/CONTRACT.md. That document is authoritative: if a test and the code disagree, the document decides, and if the document is wrong it gets a version bump, not an edit.

Prerequisites

  • Nix with flakes enabled.
  • /dev/kvm readable, for the two VM checks. They are nixosTests; without KVM they still run, just slowly enough that the timeouts below get tight.
  • No GPU. Nothing in the test suite needs one — the compositor runs on Hyprland's headless backend over software rendering.

Nothing in the checks touches the host: no /etc/nixos, no network, and no socket outside the build sandbox or the test VM.

The dev shell

nix develop

Puts python3, pytest, hyprland, grim, ffmpeg, jq and tmux on PATH and sets PYTHONPATH=$PWD/src, so import hhstream picks up the working tree rather than an installed copy.

Build

nix build .#hhstream          # the package (server + client)
ls result/bin                 # hhstream-server, hhstream-client

Both entry points currently exit with NotImplementedError — see "Known gaps" at the bottom. The package builds; it does not yet run.

The checks

Check What it proves Needs a VM
unit the framing, the capture backend, the server's client handling, the client — in-process, against the nix/package.nix build no
vm-basic the NixOS module brings up both units and publishes a contract-conformant socket, with a fake capture backend yes
vm-hyprland the same, capturing the real headless compositor through grim yes

All three at once:

nix flake check -L

nix flake check stops at the first failing check, which makes it a poor way to see the whole picture while the tree is still red. Prefer running them individually — they are independent:

nix build -L .#checks.x86_64-linux.unit
nix build -L .#checks.x86_64-linux.vm-basic
nix build -L .#checks.x86_64-linux.vm-hyprland

-L is not optional in practice: without it, Nix hides the build log, and the build log is the test report.

unit

pytest over tests/unit/, run in the build sandbox with PYTHONPATH pointing at what nix/package.nix installed (nix/package.nix sets doCheck = false on purpose, so this is the only place the suite runs in a build).

The check fails if the suite is empty, not just if it is red. pytest exits 5 on "collected nothing", but it exits 0 when every test was skipped or deselected, so checks/lib/check_report.py parses the JUnit report and refuses a run with zero collected or zero executed tests. A green check that tested nothing is worse than no check. On success the report is copied to result/junit.xml.

To run the same tests directly, without Nix:

nix develop
pytest tests/unit -v                      # PYTHONPATH=$PWD/src comes from the shell
pytest tests/unit/test_protocol.py -v     # one file
pytest tests/unit -k "hello or frame" -v  # one theme
pytest tests/unit -x --ff                 # stop at the first failure, failures first

Note that the unit suite's negative tests use the rejects fixture from tests/unit/conftest.py rather than pytest.raises(Exception). rejects treats NotImplementedError as a failure, because a plain raises(Exception) is satisfied by an unimplemented stub — the test would then be green precisely because nothing had been written.

vm-basic

Boots one machine with services.hyprland-headless-stream.enable = true and captureCommand pointed at tests/fake_capture.py, which writes one fixed, deterministic PNG of the configured size to stdout. Because the image never changes, "two frames arrived and they are the same bytes" is a real assertion about the framing rather than a coincidence.

It checks, in order: both units start; the socket exists and is hhstream:hhstream mode 0660; the Hello parses with magic = "HHS1" and non-zero width/height matching the configured output; two frames arrive with length > 0, identical bytes, and an IHDR matching the Hello; two clients are served concurrently; and a client that dies mid-frame leaves the server serving.

The reader is tests/assert_stream.py, a second, independent implementation of the contract that shares no code with hhstream.client — so a framing bug present in both the server's writer and the shipped client's reader still fails the test. It validates the length prefix strictly (0|[1-9][0-9]*: no leading zeros, no sign, no spaces), never reads past a newline, and prints the Hello, every frame's size and SHA-256, and the parsed IHDR before it decides anything.

vm-hyprland

The same machine with no captureCommand override, so the module's default (grim) captures the real compositor.

This is the check expected to be flaky, so it is deliberately impatient: every wait has a bounded timeout, nothing is retried in a loop, and any failure dumps diagnostics before it propagates. It asserts a frame is a valid PNG whose IHDR dimensions match the configured output, and that it is more than a blank buffer:

  • the IDAT must decompress;
  • the decompressed scanlines must contain a non-zero byte and at least two distinct byte values — a zero-filled framebuffer fails both;
  • the payload must be larger than a best-case blank PNG of the same geometry (blank_png_size() in tests/assert_stream.py: an all-zero raster at zlib level 9, i.e. the smallest a blank frame of that size can be encoded).

It also runs grim directly against the session as a separate subtest, which separates "grim cannot see the compositor" from "the server is wrong" — the two failures that look identical from the socket.

When vm-hyprland fails: where the logs are

Every failing subtest in both VM checks runs hhs-hyprland-diagnostics (tests/hyprland_diagnostics.sh) in the guest first and prints its output to the build log. So the answer to "where are the logs" is usually in the build log you already have, above the traceback, between ########## diagnostics: ... ########## markers. Re-read it with:

nix log .#checks.x86_64-linux.vm-hyprland

What the dump contains, and where it comes from in the guest:

Section Source
units systemctl list-units --all 'hyprland*'
per-unit status / journalctl hyprland-headless.service and hyprland-headless-stream.service, 200 lines each
socket and runtime directories /run/hyprland-headless-stream, /run/user/*
hyprland logs every hyprland*.log found under /run, /var/log, /tmp, /home
hyprctl monitors run once per instance found, with the right XDG_RUNTIME_DIR and HYPRLAND_INSTANCE_SIGNATURE
processes ps aux filtered to hypr, hhstream, grim

Hyprland writes its log to $XDG_RUNTIME_DIR/hypr/$HYPRLAND_INSTANCE_SIGNATURE/hyprland.log, and the compositor runs on a private XDG_RUNTIME_DIR, so the path is not knowable in advance — that is why the script searches for it rather than naming it. The instance signature is the directory name under <runtime>/hypr, which is also what hyprctl needs in the environment:

XDG_RUNTIME_DIR=/run/user/$(id -u hhstream) \
HYPRLAND_INSTANCE_SIGNATURE=$(ls /run/user/$(id -u hhstream)/hypr | head -1) \
  hyprctl monitors

Poking at a live VM

The nicest way to debug a VM failure is to keep the machine:

nix build .#checks.x86_64-linux.vm-hyprland.driverInteractive
./result/bin/nixos-test-driver

That drops you in a Python REPL with machine bound. start_all(), then machine.shell_interact() for a root shell in the guest, or run the test script a line at a time. machine.succeed("hhs-hyprland-diagnostics") works there too.

Layout

checks/
  default.nix            wiring only: the three checks the flake exposes
  unit.nix               pytest in the sandbox, against the nix/package.nix build
  vm-basic.nix           nixosTest, fake capture backend
  vm-hyprland.nix        nixosTest, real compositor + grim
  lib/test-tools.nix     packages the tests/ helpers for the guest
  lib/check_report.py    JUnit report -> build verdict (and the empty-suite guard)
tests/
  fake_capture.py        writes one fixed PNG to stdout: hhs-fake-capture
  assert_stream.py       independent contract reader/asserter: hhs-assert-stream
  hyprland_diagnostics.sh  the failure dump: hhs-hyprland-diagnostics
  unit/                  the pytest suite

checks/lib/test-tools.nix self-tests the two Python helpers at build time (the fake emits a PNG of the size it was asked for; the asserter starts), because a broken helper otherwise surfaces as a baffling failure three layers away.

Adding a unit test: drop a test_*.py in tests/unit/. Nothing needs registering. Adding a check: one file in checks/, one line in checks/default.nix.

Known gaps in the interface (as of this branch)

These are defects in code this branch does not own; they are recorded here because the tests are written against the contract and therefore trip over them.

  1. nix/module.nix has no config side yet. Neither hyprland-headless.service nor hyprland-headless-stream.service exists, so both VM checks fail at their first wait with Unit hyprland-headless.service could not be found. Nothing in checks/ works around this.

  2. services.hyprland-headless-stream.package is declared with no default. docs/CONTRACT.md says the default is "the flake's own build", but the option has no default, so reading it fails with

    error: The option `services.hyprland-headless-stream.package' was accessed but
    has no value defined. Try setting the option.
    

    Today that only bites code that reads cfg.package, which the unwritten config side does not yet do; it will bite every consumer that sets only enable = true the moment it does. Both VM checks set package explicitly.

  3. src/hhstream/*.py is stubs. Every function raises NotImplementedError, so 66 of the 70 unit tests fail. The 4 that pass are tests/unit/test_package.py, which only reads constants — they are the canary for the unit check's own wiring.

  4. The frame server's command line is unspecified. docs/CONTRACT.md fixes the module's option surface and the wire format but never says how the module invokes hhstream-server. tests/unit/test_server.py::test_build_parser_exposes_the_options_the_module_needs currently assumes --socket PATH --fps N --format png -- <captureCommand...>. That is a guess; if the module ends up passing something else, that one test is the thing to change, not the server.