Merge branch 'slice/tests'

# Conflicts:
#	tests/unit/conftest.py
#	tests/unit/test_capture.py
#	tests/unit/test_client.py
#	tests/unit/test_protocol.py
#	tests/unit/test_server.py
This commit is contained in:
Hermenix committed 2026-09-28 11:51:07 +00:00
commit 72f359932a
12 files changed
+1291 -2

No files matched your search

+241
View File
@@ -0,0 +1,241 @@
# Developing and testing `hyprland-headless-stream-plugin`
The interface everything here asserts is [`docs/CONTRACT.md`](docs/CONTRACT.md).
That document is authoritative: if a test and the code disagree, the document
decides, and if the document is wrong it gets a version bump, not an edit.
## Prerequisites
- Nix with flakes enabled.
- `/dev/kvm` readable, for the two VM checks. They are `nixosTest`s; without
KVM they still run, just slowly enough that the timeouts below get tight.
- No GPU. Nothing in the test suite needs one — the compositor runs on
Hyprland's headless backend over software rendering.
Nothing in the checks touches the host: no `/etc/nixos`, no network, and no
socket outside the build sandbox or the test VM.
## The dev shell
```bash
nix develop
```
Puts `python3`, `pytest`, `hyprland`, `grim`, `ffmpeg`, `jq` and `tmux` on PATH
and sets `PYTHONPATH=$PWD/src`, so `import hhstream` picks up the working tree
rather than an installed copy.
## Build
```bash
nix build .#hhstream # the package (server + client)
ls result/bin # hhstream-server, hhstream-client
```
Both entry points currently exit with `NotImplementedError` — see "Known gaps"
at the bottom. The package builds; it does not yet run.
## The checks
| Check | What it proves | Needs a VM |
|---|---|---|
| `unit` | the framing, the capture backend, the server's client handling, the client — in-process, against the `nix/package.nix` build | no |
| `vm-basic` | the NixOS module brings up both units and publishes a contract-conformant socket, with a **fake** capture backend | yes |
| `vm-hyprland` | the same, capturing the **real** headless compositor through `grim` | yes |
All three at once:
```bash
nix flake check -L
```
`nix flake check` stops at the first failing check, which makes it a poor way to
see the whole picture while the tree is still red. Prefer running them
individually — they are independent:
```bash
nix build -L .#checks.x86_64-linux.unit
nix build -L .#checks.x86_64-linux.vm-basic
nix build -L .#checks.x86_64-linux.vm-hyprland
```
`-L` is not optional in practice: without it, Nix hides the build log, and the
build log *is* the test report.
### `unit`
pytest over `tests/unit/`, run in the build sandbox with `PYTHONPATH` pointing at
what `nix/package.nix` installed (`nix/package.nix` sets `doCheck = false` on
purpose, so this is the only place the suite runs in a build).
The check fails if the suite is **empty**, not just if it is red. `pytest` exits
5 on "collected nothing", but it exits 0 when every test was skipped or
deselected, so `checks/lib/check_report.py` parses the JUnit report and refuses
a run with zero collected or zero executed tests. A green check that tested
nothing is worse than no check. On success the report is copied to
`result/junit.xml`.
To run the same tests directly, without Nix:
```bash
nix develop
pytest tests/unit -v # PYTHONPATH=$PWD/src comes from the shell
pytest tests/unit/test_protocol.py -v # one file
pytest tests/unit -k "hello or frame" -v # one theme
pytest tests/unit -x --ff # stop at the first failure, failures first
```
Note that the unit suite's *negative* tests use the `rejects` fixture from
`tests/unit/conftest.py` rather than `pytest.raises(Exception)`. `rejects`
treats `NotImplementedError` as a failure, because a plain `raises(Exception)`
is satisfied by an unimplemented stub — the test would then be green precisely
because nothing had been written.
### `vm-basic`
Boots one machine with `services.hyprland-headless-stream.enable = true` and
`captureCommand` pointed at `tests/fake_capture.py`, which writes one fixed,
deterministic PNG of the configured size to stdout. Because the image never
changes, "two frames arrived and they are the *same bytes*" is a real assertion
about the framing rather than a coincidence.
It checks, in order: both units start; the socket exists and is
`hhstream:hhstream` mode `0660`; the Hello parses with `magic = "HHS1"` and
non-zero width/height matching the configured output; two frames arrive with
length > 0, identical bytes, and an IHDR matching the Hello; two clients are
served concurrently; and a client that dies mid-frame leaves the server serving.
The reader is `tests/assert_stream.py`, a second, independent implementation of
the contract that shares no code with `hhstream.client` — so a framing bug
present in both the server's writer and the shipped client's reader still fails
the test. It validates the length prefix strictly (`0|[1-9][0-9]*`: no leading
zeros, no sign, no spaces), never reads past a newline, and prints the Hello,
every frame's size and SHA-256, and the parsed IHDR before it decides anything.
### `vm-hyprland`
The same machine with **no** `captureCommand` override, so the module's default
(`grim`) captures the real compositor.
This is the check expected to be flaky, so it is deliberately impatient: every
wait has a bounded timeout, nothing is retried in a loop, and any failure dumps
diagnostics before it propagates. It asserts a frame is a valid PNG whose IHDR
dimensions match the configured output, and that it is more than a blank buffer:
- the IDAT must decompress;
- the decompressed scanlines must contain a non-zero byte and at least two
distinct byte values — a zero-filled framebuffer fails both;
- the payload must be larger than a best-case blank PNG of the same geometry
(`blank_png_size()` in `tests/assert_stream.py`: an all-zero raster at zlib
level 9, i.e. the smallest a blank frame of that size can be encoded).
It also runs `grim` directly against the session as a separate subtest, which
separates "grim cannot see the compositor" from "the server is wrong" — the two
failures that look identical from the socket.
## When `vm-hyprland` fails: where the logs are
Every failing subtest in both VM checks runs `hhs-hyprland-diagnostics`
(`tests/hyprland_diagnostics.sh`) in the guest first and prints its output to the
build log. So the answer to "where are the logs" is usually **in the build log
you already have**, above the traceback, between
`########## diagnostics: ... ##########` markers. Re-read it with:
```bash
nix log .#checks.x86_64-linux.vm-hyprland
```
What the dump contains, and where it comes from in the guest:
| Section | Source |
|---|---|
| `units` | `systemctl list-units --all 'hyprland*'` |
| per-unit `status` / `journalctl` | `hyprland-headless.service` and `hyprland-headless-stream.service`, 200 lines each |
| `socket and runtime directories` | `/run/hyprland-headless-stream`, `/run/user/*` |
| `hyprland logs` | every `hyprland*.log` found under `/run`, `/var/log`, `/tmp`, `/home` |
| `hyprctl monitors` | run once per instance found, with the right `XDG_RUNTIME_DIR` and `HYPRLAND_INSTANCE_SIGNATURE` |
| `processes` | `ps aux` filtered to `hypr`, `hhstream`, `grim` |
Hyprland writes its log to
`$XDG_RUNTIME_DIR/hypr/$HYPRLAND_INSTANCE_SIGNATURE/hyprland.log`, and the
compositor runs on a *private* `XDG_RUNTIME_DIR`, so the path is not knowable in
advance — that is why the script searches for it rather than naming it. The
instance signature is the directory name under `<runtime>/hypr`, which is also
what `hyprctl` needs in the environment:
```bash
XDG_RUNTIME_DIR=/run/user/$(id -u hhstream) \
HYPRLAND_INSTANCE_SIGNATURE=$(ls /run/user/$(id -u hhstream)/hypr | head -1) \
hyprctl monitors
```
### Poking at a live VM
The nicest way to debug a VM failure is to keep the machine:
```bash
nix build .#checks.x86_64-linux.vm-hyprland.driverInteractive
./result/bin/nixos-test-driver
```
That drops you in a Python REPL with `machine` bound. `start_all()`, then
`machine.shell_interact()` for a root shell in the guest, or run the test script
a line at a time. `machine.succeed("hhs-hyprland-diagnostics")` works there too.
## Layout
```
checks/
default.nix wiring only: the three checks the flake exposes
unit.nix pytest in the sandbox, against the nix/package.nix build
vm-basic.nix nixosTest, fake capture backend
vm-hyprland.nix nixosTest, real compositor + grim
lib/test-tools.nix packages the tests/ helpers for the guest
lib/check_report.py JUnit report -> build verdict (and the empty-suite guard)
tests/
fake_capture.py writes one fixed PNG to stdout: hhs-fake-capture
assert_stream.py independent contract reader/asserter: hhs-assert-stream
hyprland_diagnostics.sh the failure dump: hhs-hyprland-diagnostics
unit/ the pytest suite
```
`checks/lib/test-tools.nix` self-tests the two Python helpers at build time
(the fake emits a PNG of the size it was asked for; the asserter starts), because
a broken helper otherwise surfaces as a baffling failure three layers away.
Adding a unit test: drop a `test_*.py` in `tests/unit/`. Nothing needs
registering. Adding a check: one file in `checks/`, one line in
`checks/default.nix`.
## Known gaps in the interface (as of this branch)
These are defects in code this branch does not own; they are recorded here
because the tests are written against the contract and therefore trip over them.
1. **`nix/module.nix` has no `config` side yet.** Neither
`hyprland-headless.service` nor `hyprland-headless-stream.service` exists, so
both VM checks fail at their first wait with
`Unit hyprland-headless.service could not be found.` Nothing in `checks/`
works around this.
2. **`services.hyprland-headless-stream.package` is declared with no default.**
`docs/CONTRACT.md` says the default is "the flake's own build", but the option
has no `default`, so reading it fails with
```
error: The option `services.hyprland-headless-stream.package' was accessed but
has no value defined. Try setting the option.
```
Today that only bites code that reads `cfg.package`, which the unwritten
`config` side does not yet do; it will bite every consumer that sets only
`enable = true` the moment it does. Both VM checks set `package` explicitly.
3. **`src/hhstream/*.py` is stubs.** Every function raises
`NotImplementedError`, so 66 of the 70 unit tests fail. The 4 that pass are
`tests/unit/test_package.py`, which only reads constants — they are the
canary for the `unit` check's own wiring.
4. **The frame server's command line is unspecified.** `docs/CONTRACT.md` fixes
the module's option surface and the wire format but never says how the module
invokes `hhstream-server`. `tests/unit/test_server.py::test_build_parser_exposes_the_options_the_module_needs`
currently assumes `--socket PATH --fps N --format png -- <captureCommand...>`.
That is a guess; if the module ends up passing something else, that one test
is the thing to change, not the server.
+28
View File
@@ -0,0 +1,28 @@
# The flake's `checks` output. One file per check; this is only the wiring.
#
# unit -- pytest over tests/unit/, against the nix/package.nix build
# vm-basic -- nixosTest with a fake capture backend: module + wire contract
# vm-hyprland -- nixosTest with the real compositor and grim
#
# Everything the guests need lives in checks/lib/test-tools.nix.
{
pkgs,
self,
system,
}:
let
hhstream = self.packages.${system}.hhstream;
hhsModule = self.nixosModules.default;
testTools = pkgs.callPackage ./lib/test-tools.nix { };
in
{
unit = pkgs.callPackage ./unit.nix { inherit hhstream; };
vm-basic = pkgs.callPackage ./vm-basic.nix {
inherit hhstream hhsModule testTools;
};
vm-hyprland = pkgs.callPackage ./vm-hyprland.nix {
inherit hhstream hhsModule testTools;
};
}
+90
View File
@@ -0,0 +1,90 @@
#!/usr/bin/env python3
"""Turn a pytest JUnit report into a build verdict.
Two jobs pytest's exit code alone does not do well inside a Nix build:
* say out loud how many tests ran, and which ones failed, in the build log;
* refuse a suite that collected nothing. An empty suite exits 5, but it also
exits 0 if every test was deselected or skipped away, and a check that passes
because it tested nothing is worse than no check at all.
Exit codes: 0 all good, 1 the suite is empty or the report is unusable,
2 the suite ran and some tests failed. The caller distinguishes them so the
build log does not accuse an empty suite of being a failing one.
"""
from __future__ import annotations
import sys
import xml.etree.ElementTree as ET
def main(argv: list[str]) -> int:
if len(argv) != 2:
print("usage: check_report.py <junit.xml>", file=sys.stderr)
return 2
path = argv[1]
try:
root = ET.parse(path).getroot()
except FileNotFoundError:
print(
f"ERROR: pytest wrote no report at {path}: it collected nothing, or died "
"before it could run a single test.",
file=sys.stderr,
)
return 1
except ET.ParseError as exc:
print(f"ERROR: {path} is not parseable JUnit XML: {exc}", file=sys.stderr)
return 1
suites = root.findall("testsuite") or ([root] if root.tag == "testsuite" else [])
total = errors = failures = skipped = 0
failing: list[str] = []
for suite in suites:
total += int(suite.get("tests", 0))
errors += int(suite.get("errors", 0))
failures += int(suite.get("failures", 0))
skipped += int(suite.get("skipped", 0))
for case in suite.iter("testcase"):
for kind in ("failure", "error"):
for node in case.findall(kind):
name = f"{case.get('classname', '')}::{case.get('name', '')}"
message = (node.get("message") or "").splitlines()
head = message[0][:160] if message else node.get("type", kind)
failing.append(f" {kind.upper():7} {name}\n {head}")
executed = total - skipped
print(
f"unit suite: {total} collected, {executed} executed, "
f"{failures} failed, {errors} errored, {skipped} skipped"
)
if total == 0:
print(
"ERROR: the unit suite collected no tests. An empty suite is not a "
"passing suite -- check that tests/unit/ is in the build and that the "
"files are named test_*.py.",
file=sys.stderr,
)
return 1
if executed == 0:
print(
f"ERROR: all {total} collected tests were skipped; nothing was actually "
"exercised.",
file=sys.stderr,
)
return 1
if failing:
print(f"\n{len(failing)} failing test(s):")
for line in failing:
print(line)
return 2
print("unit suite: all executed tests passed")
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv))
+32
View File
@@ -0,0 +1,32 @@
# The helper programs the checks need inside the guest (and, for the fake
# capture, inside the guest's `captureCommand`). Plain python3/sh scripts from
# `tests/`, put on PATH with their shebangs pointing at the store.
{
runCommandLocal,
python3,
coreutils,
gnugrep,
findutils,
systemd,
}:
runCommandLocal "hhstream-test-tools"
{
nativeBuildInputs = [ python3 ];
# hhs-hyprland-diagnostics runs on a failing guest, where PATH is whatever
# systemd left behind; give it the tools it needs explicitly.
diagPath = "${coreutils}/bin:${gnugrep}/bin:${findutils}/bin:${systemd}/bin";
meta.description = "Fake capture backend and contract asserter for the hhstream checks";
} ''
install -Dm755 ${../../tests/fake_capture.py} $out/bin/hhs-fake-capture
install -Dm755 ${../../tests/assert_stream.py} $out/bin/hhs-assert-stream
install -Dm755 ${../../tests/hyprland_diagnostics.sh} $out/bin/hhs-hyprland-diagnostics
patchShebangs $out/bin
substituteInPlace $out/bin/hhs-hyprland-diagnostics --replace-fail '@diagPath@' "$diagPath"
# A broken helper shows up as a baffling test failure three layers away, so
# prove here that both python programs start and produce what they promise.
$out/bin/hhs-fake-capture --width 8 --height 4 > frame.png
python3 -c 'import struct,sys; d=open("frame.png","rb").read(); assert d[:8]==b"\x89PNG\r\n\x1a\n", d[:8]; assert struct.unpack(">II", d[16:24])==(8,4), d[16:24]'
$out/bin/hhs-assert-stream --help > /dev/null
$out/bin/hhs-hyprland-diagnostics --self-test > /dev/null
''
+74
View File
@@ -0,0 +1,74 @@
# checks.<system>.unit -- the Python unit suite from tests/unit/, run against the
# package built by nix/package.nix, inside the build sandbox.
#
# nix/package.nix sets doCheck = false deliberately, so this is the only place
# the suite runs in a build.
{
lib,
runCommand,
python3,
hhstream,
}:
let
pytestEnv = python3.withPackages (ps: [ ps.pytest ]);
# Only the suite itself, so an editor's droppings or a local .pytest_cache
# cannot change the derivation's hash.
testSrc = lib.cleanSourceWith {
name = "hhstream-unit-tests";
src = ../tests/unit;
filter =
path: type:
let
base = baseNameOf (toString path);
in
!(type == "directory" && (base == "__pycache__" || base == ".pytest_cache"))
&& !lib.hasSuffix ".pyc" base;
};
in
runCommand "hhstream-unit"
{
nativeBuildInputs = [ pytestEnv ];
passthru.testSrc = testSrc;
meta.description = "hhstream unit tests (pytest)";
} ''
cp -r ${testSrc} suite
chmod -R u+w suite
# The package is an application, so its library is not on the default path:
# point PYTHONPATH at what nix/package.nix installed. Importing it here rather
# than letting pytest do it makes a packaging problem distinguishable from a
# test failure.
export PYTHONPATH=${hhstream}/${python3.sitePackages}
export PYTHONDONTWRITEBYTECODE=1
export HOME=$PWD
python3 -c 'import hhstream; print("testing hhstream", hhstream.__version__, "from", hhstream.__file__)'
set +e
pytest suite \
--color=no -rf -q \
--junit-xml=report.xml \
-p no:cacheprovider \
--strict-markers \
--strict-config
pytest_rc=$?
set -e
echo
set +e
python3 ${./lib/check_report.py} report.xml
report_rc=$?
set -e
if [ "$report_rc" -eq 1 ]; then
echo "unit check: the suite is empty or its report is unusable -- failing." >&2
exit 1
fi
if [ "$report_rc" -ne 0 ] || [ "$pytest_rc" -ne 0 ]; then
echo "unit check: the suite ran but did not pass (pytest exit $pytest_rc)." >&2
exit 1
fi
mkdir -p $out
cp report.xml $out/junit.xml
''
+145
View File
@@ -0,0 +1,145 @@
# checks.<system>.vm-basic -- boot a real machine with the module enabled, point
# `captureCommand` at a fake capture program, and read the socket.
#
# This check is about the module's wiring and the wire contract, not about
# Hyprland: the fake capture emits one fixed PNG, so "two frames arrived and
# they are the same image" is a real assertion rather than a coincidence.
{
testers,
hhstream,
testTools,
hhsModule,
width ? 1280,
height ? 720,
fps ? 30,
socketPath ? "/run/hyprland-headless-stream/stream.sock",
}:
testers.runNixOSTest {
name = "hhstream-vm-basic";
# KVM is available, so this is a minute or two; cap it so a hang is a failure
# rather than a wedged builder.
globalTimeout = 900;
nodes.machine = {
imports = [ hhsModule ];
services.hyprland-headless-stream = {
enable = true;
# The module declares `package` with no default (see README.dev.md), so the
# test has to supply the flake's build.
package = hhstream;
inherit socketPath;
frameRate = fps;
format = "png";
outputs = [
{
name = "HEADLESS-1";
inherit width height;
refresh = fps;
}
];
captureCommand = [
"${testTools}/bin/hhs-fake-capture"
"--width"
(toString width)
"--height"
(toString height)
];
};
environment.systemPackages = [
testTools
hhstream
];
virtualisation = {
memorySize = 1024;
cores = 2;
diskSize = 3072;
graphics = false;
};
};
testScript = ''
SOCKET = "${socketPath}"
ASSERT = "hhs-assert-stream --socket " + SOCKET
def dump(label):
print(f"########## diagnostics: {label} ##########")
print(machine.execute("hhs-hyprland-diagnostics")[1])
start_all()
with subtest("the module's two units come up"):
try:
machine.wait_for_unit("hyprland-headless.service", timeout=120)
machine.wait_for_unit("hyprland-headless-stream.service", timeout=120)
except Exception:
dump("units did not start")
raise
with subtest("the socket exists, owned by the service user, mode 0660"):
try:
machine.wait_for_file(SOCKET, timeout=60)
print(machine.succeed(f"stat -c 'owner=%U group=%G mode=%a type=%F' {SOCKET}"))
machine.succeed(f"[ \"$(stat -c %a {SOCKET})\" = 660 ]")
machine.succeed(f"[ \"$(stat -c %U {SOCKET})\" = hhstream ]")
machine.succeed(f"[ \"$(stat -c %G {SOCKET})\" = hhstream ]")
machine.succeed(f"[ \"$(stat -c %F {SOCKET})\" = socket ]")
except Exception:
dump("socket missing or wrong ownership")
raise
with subtest("the Hello parses, magic is HHS1, size is non-zero"):
try:
print(machine.succeed(f"{ASSERT} --frames 1 --timeout 30"))
except Exception:
dump("Hello did not parse")
raise
with subtest("two frames arrive, non-empty, byte-identical, matching the Hello"):
try:
print(
machine.succeed(
f"{ASSERT} --frames 2 --timeout 60"
" --expect-width ${toString width} --expect-height ${toString height}"
" --expect-format image/png --expect-fps ${toString fps}"
" --require-png --identical-frames --min-bytes 1"
)
)
except Exception:
dump("frames did not satisfy the contract")
raise
with subtest("two clients are served concurrently and independently"):
try:
print(
machine.succeed(
f"{ASSERT} --frames 3 --timeout 60 --require-png > /tmp/a.log 2>&1 & p1=$!; "
f"{ASSERT} --frames 3 --timeout 60 --require-png > /tmp/b.log 2>&1 & p2=$!; "
"wait $p1; r1=$?; wait $p2; r2=$?; "
"echo '--- client a ---'; cat /tmp/a.log; "
"echo '--- client b ---'; cat /tmp/b.log; "
"exit $((r1 + r2))"
)
)
except Exception:
dump("concurrent clients were not both served")
raise
with subtest("a client dying mid-stream leaves the server serving"):
try:
machine.succeed(
"timeout 5 python3 -c 'import socket;"
f"s=socket.socket(socket.AF_UNIX);s.connect(\"{SOCKET}\");s.recv(1);s.close()'"
)
print(machine.succeed(f"{ASSERT} --frames 2 --timeout 60 --require-png"))
machine.succeed("systemctl is-active hyprland-headless-stream.service")
except Exception:
dump("server did not survive a client dying mid-frame")
raise
'';
}
+157
View File
@@ -0,0 +1,157 @@
# checks.<system>.vm-hyprland -- the real one. Same machine as vm-basic, but no
# `captureCommand` override, so the server captures the actual headless
# compositor through grim.
#
# This is the check expected to be flaky, so it is deliberately impatient: every
# wait has a bounded timeout, nothing is retried in a loop, and any failure dumps
# the compositor's log and `hyprctl monitors` before it propagates. A flake you
# can read is worth more than a flake that eventually goes green.
{
testers,
hhstream,
testTools,
hhsModule,
hyprland,
grim,
width ? 1280,
height ? 720,
fps ? 30,
socketPath ? "/run/hyprland-headless-stream/stream.sock",
}:
testers.runNixOSTest {
name = "hhstream-vm-hyprland";
globalTimeout = 1800;
nodes.machine = {
imports = [ hhsModule ];
services.hyprland-headless-stream = {
enable = true;
package = hhstream;
hyprlandPackage = hyprland;
inherit socketPath;
frameRate = fps;
format = "png";
outputs = [
{
name = "HEADLESS-1";
inherit width height;
refresh = fps;
}
];
# No captureCommand: the module's default (grim) is the point of this check.
};
# hyprctl and grim are here for the diagnostics dump and for the manual
# cross-check below, not for the code path under test.
environment.systemPackages = [
testTools
hhstream
hyprland
grim
];
# The compositor needs more room than the fake capture did, and its software
# renderer wants a CPU. There is still no GPU: Hyprland runs on its headless
# backend over llvmpipe.
virtualisation = {
memorySize = 2048;
cores = 2;
diskSize = 4096;
graphics = false;
};
};
testScript = ''
SOCKET = "${socketPath}"
WIDTH, HEIGHT = ${toString width}, ${toString height}
def dump(label):
print(f"########## diagnostics: {label} ##########")
status, out = machine.execute("hhs-hyprland-diagnostics", timeout=120)
print(out)
print(f"########## end diagnostics (exit {status}) ##########")
start_all()
with subtest("the compositor unit comes up"):
try:
machine.wait_for_unit("hyprland-headless.service", timeout=180)
except Exception:
dump("hyprland-headless.service did not start")
raise
with subtest("the compositor reports the configured headless output"):
try:
# Proves the compositor is really running a 1280x720 HEADLESS-1
# before we blame the capture for a wrong-sized frame.
out = machine.succeed("hhs-hyprland-diagnostics", timeout=120)
print(out)
assert "HEADLESS-1" in out, (
"hyprctl never reported a HEADLESS-1 monitor; see the dump above"
)
assert f"{WIDTH}x{HEIGHT}" in out, (
f"hyprctl never reported a {WIDTH}x{HEIGHT} mode; see the dump above"
)
except Exception:
dump("compositor output not as configured")
raise
with subtest("the frame server unit comes up and publishes the socket"):
try:
machine.wait_for_unit("hyprland-headless-stream.service", timeout=180)
machine.wait_for_file(SOCKET, timeout=120)
except Exception:
dump("frame server did not publish the socket")
raise
with subtest("grim can capture the session at all"):
# Separates "grim cannot see the compositor" from "the server is wrong",
# which are the two failures that look identical from the socket.
try:
print(
machine.succeed(
"systemd-run --pipe --wait --uid=hhstream"
" --setenv=XDG_RUNTIME_DIR=/run/user/$(id -u hhstream)"
" grim -t png /tmp/direct.png 2>&1 || true",
timeout=120,
)
)
print(machine.execute("stat -c '%s bytes' /tmp/direct.png")[1])
except Exception:
dump("direct grim capture failed")
raise
with subtest("a frame off the socket is a real PNG of the configured size"):
try:
print(
machine.succeed(
f"hhs-assert-stream --socket {SOCKET} --frames 1 --timeout 60"
f" --expect-width {WIDTH} --expect-height {HEIGHT}"
" --expect-format image/png --require-png --reject-blank",
timeout=180,
)
)
except Exception:
dump("frame off the socket failed the PNG assertions")
raise
with subtest("the stream keeps producing frames"):
try:
print(
machine.succeed(
f"hhs-assert-stream --socket {SOCKET} --frames 3 --timeout 60"
f" --expect-width {WIDTH} --expect-height {HEIGHT}"
" --require-png --reject-blank",
timeout=240,
)
)
machine.succeed("systemctl is-active hyprland-headless-stream.service")
except Exception:
dump("the stream stopped after the first frame")
raise
'';
}
+8 -2
View File
@@ -8,6 +8,10 @@
outputs = { self, nixpkgs, ... }: let
systems = [ "x86_64-linux" "aarch64-linux" ];
forAllSystems = f: nixpkgs.lib.genAttrs systems (system: f (import nixpkgs { inherit system; }));
# `checks` needs the system name as well as the package set, to reach
# self.packages.<system>; forAllSystems only hands over pkgs.
forAllSystems' = f: nixpkgs.lib.genAttrs systems (system:
f { inherit system; pkgs = import nixpkgs { inherit system; }; });
in {
packages = forAllSystems (pkgs: rec {
hhstream = pkgs.callPackage ./nix/package.nix {};
@@ -19,8 +23,10 @@
default = hyprland-headless-stream;
};
# `checks` (unit tests + the VM integration test) is owned by the
# test-harness workstream -- see docs/CONTRACT.md for the interface.
# Unit tests + the two VM integration tests. See checks/default.nix and
# README.dev.md; the interface they assert is docs/CONTRACT.md.
checks = forAllSystems' ({ pkgs, system }:
import ./checks { inherit pkgs self system; });
devShells = forAllSystems (pkgs: {
default = pkgs.mkShell {
+307
View File
@@ -0,0 +1,307 @@
#!/usr/bin/env python3
"""Read a hhstream socket and assert it obeys docs/CONTRACT.md.
This is deliberately a *second, independent* implementation of the reader: it
shares no code with `hhstream.client`, so a bug that lives in both the server's
writer and the shipped client's reader still fails the test. Pure standard
library; it runs inside the test VM, where only python3 is guaranteed.
Every run prints what it saw before it decides, so a failure in CI is readable
without reproducing it.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import re
import socket
import struct
import sys
import zlib
MAGIC = "HHS1"
FORMATS = {"image/png", "image/jpeg"}
PNG_SIGNATURE = b"\x89PNG\r\n\x1a\n"
# Contract: plain ASCII digits, no leading zeros, no `+`, no spaces.
LENGTH_RE = re.compile(rb"\A(?:0|[1-9][0-9]*)\Z")
class ContractError(Exception):
"""The stream violated docs/CONTRACT.md."""
# --------------------------------------------------------------------------
# wire reading
# --------------------------------------------------------------------------
def read_line(sock: socket.socket, limit: int = 4096) -> bytes:
"""Read up to and including the next 0x0A. Never over-reads past it."""
out = bytearray()
while True:
b = sock.recv(1)
if not b:
raise ContractError(
f"socket closed after {len(out)} bytes while reading a line: {bytes(out)!r}"
)
if b == b"\n":
return bytes(out)
out += b
if len(out) > limit:
raise ContractError(f"no newline within {limit} bytes: {bytes(out[:120])!r}...")
def read_exact(sock: socket.socket, n: int) -> bytes:
out = bytearray()
while len(out) < n:
part = sock.recv(min(65536, n - len(out)))
if not part:
raise ContractError(
f"truncated payload: wanted {n} bytes, socket closed after {len(out)}"
)
out += part
return bytes(out)
def read_hello(sock: socket.socket) -> dict:
line = read_line(sock)
try:
obj = json.loads(line.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise ContractError(f"Hello is not UTF-8 JSON ({exc}): {line!r}") from exc
if not isinstance(obj, dict):
raise ContractError(f"Hello is not a JSON object: {line!r}")
if obj.get("magic") != MAGIC:
raise ContractError(f"Hello magic is {obj.get('magic')!r}, contract says {MAGIC!r}")
for key in ("width", "height", "fps"):
val = obj.get(key)
if not isinstance(val, int) or isinstance(val, bool):
raise ContractError(f"Hello {key} is {val!r}, expected an integer")
if val <= 0:
raise ContractError(
f"Hello {key} is {val}; the contract forbids a Hello with zero size"
)
if obj.get("format") not in FORMATS:
raise ContractError(
f"Hello format is {obj.get('format')!r}, expected one of {sorted(FORMATS)}"
)
return obj
def read_frame(sock: socket.socket) -> bytes:
prefix = read_line(sock)
if not LENGTH_RE.match(prefix):
raise ContractError(
f"malformed length prefix {prefix!r}: contract requires plain ASCII "
"digits with no leading zeros, sign or spaces"
)
length = int(prefix)
if length == 0:
raise ContractError("frame length is 0; a frame must carry a complete image")
return read_exact(sock, length)
# --------------------------------------------------------------------------
# PNG inspection
# --------------------------------------------------------------------------
def png_chunks(data: bytes):
off = len(PNG_SIGNATURE)
while off + 8 <= len(data):
(size,) = struct.unpack(">I", data[off : off + 4])
kind = data[off + 4 : off + 8]
body = data[off + 8 : off + 8 + size]
if len(body) != size:
raise ContractError(f"PNG chunk {kind!r} truncated: want {size}, got {len(body)}")
crc = data[off + 8 + size : off + 12 + size]
if len(crc) != 4:
raise ContractError(f"PNG chunk {kind!r} has no CRC")
want = struct.pack(">I", zlib.crc32(kind + body) & 0xFFFFFFFF)
if crc != want:
raise ContractError(f"PNG chunk {kind!r} CRC mismatch: {crc.hex()} != {want.hex()}")
yield kind, body
off += 12 + size
if kind == b"IEND":
return
raise ContractError("PNG ended without an IEND chunk")
def inspect_png(payload: bytes) -> dict:
"""Parse enough of a PNG to know its size and whether it carries a picture."""
if not payload.startswith(PNG_SIGNATURE):
raise ContractError(
f"payload is not a PNG: first 8 bytes are {payload[:8]!r}, "
f"expected {PNG_SIGNATURE!r}"
)
ihdr = None
idat = bytearray()
for kind, body in png_chunks(payload):
if kind == b"IHDR":
if ihdr is not None:
raise ContractError("PNG has more than one IHDR")
if len(body) != 13:
raise ContractError(f"IHDR is {len(body)} bytes, expected 13")
w, h, depth, colour, comp, filt, interlace = struct.unpack(">IIBBBBB", body)
ihdr = dict(
width=w, height=h, bit_depth=depth, colour_type=colour,
compression=comp, filter=filt, interlace=interlace,
)
elif kind == b"IDAT":
if ihdr is None:
raise ContractError("PNG has IDAT before IHDR")
idat += body
if ihdr is None:
raise ContractError("PNG has no IHDR chunk")
if not idat:
raise ContractError("PNG has no IDAT chunk: there is no image data")
try:
raw = zlib.decompress(bytes(idat))
except zlib.error as exc:
raise ContractError(f"PNG image data does not decompress: {exc}") from exc
info = dict(ihdr)
info["payload_bytes"] = len(payload)
info["raw_bytes"] = len(raw)
info["distinct_raw_bytes"] = len(set(raw))
info["nonzero_raw_bytes"] = len(raw) - raw.count(0)
return info
def blank_png_size(width: int, height: int, colour_type: int, bit_depth: int) -> int:
"""Smallest plausible encoding of an all-zero image of the same geometry.
Used as the floor for "bigger than a trivial blank image": a zero-filled
framebuffer, packed with the best compression available, cannot exceed this.
"""
channels = {0: 1, 2: 3, 3: 1, 4: 2, 6: 4}.get(colour_type, 3)
stride = 1 + (width * channels * bit_depth + 7) // 8
raw = b"\x00" * (stride * height)
body = zlib.compress(raw, 9)
# signature + IHDR (25) + IDAT header/crc (12) + IEND (12)
return len(PNG_SIGNATURE) + 25 + 12 + len(body) + 12
# --------------------------------------------------------------------------
# driver
# --------------------------------------------------------------------------
def build_parser() -> argparse.ArgumentParser:
p = argparse.ArgumentParser(description=__doc__)
p.add_argument("--socket", required=True, help="path of the hhstream socket")
p.add_argument("--frames", type=int, default=1, help="how many frames to read")
p.add_argument("--timeout", type=float, default=30.0,
help="per-recv socket timeout, seconds")
p.add_argument("--expect-width", type=int, help="assert Hello/IHDR width")
p.add_argument("--expect-height", type=int, help="assert Hello/IHDR height")
p.add_argument("--expect-fps", type=int, help="assert Hello fps")
p.add_argument("--expect-format", help="assert Hello format, e.g. image/png")
p.add_argument("--min-bytes", type=int, default=1,
help="every payload must be at least this many bytes")
p.add_argument("--require-png", action="store_true",
help="every payload must parse as a PNG whose IHDR matches the Hello")
p.add_argument("--identical-frames", action="store_true",
help="all payloads read must be byte-identical")
p.add_argument("--reject-blank", action="store_true",
help="reject a frame that decodes to an all-zero or single-valued "
"image, or that is no bigger than a best-case blank PNG")
return p
def run(args) -> None:
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
sock.settimeout(args.timeout)
sock.connect(args.socket)
try:
hello = read_hello(sock)
print(f"hello: {json.dumps(hello, sort_keys=True)}", flush=True)
for key, got in (("width", "expect_width"), ("height", "expect_height"),
("fps", "expect_fps"), ("format", "expect_format")):
want = getattr(args, got)
if want is not None and hello[key] != want:
raise ContractError(f"Hello {key} is {hello[key]!r}, expected {want!r}")
if args.require_png and hello["format"] != "image/png":
raise ContractError(
f"--require-png but Hello advertises format {hello['format']!r}"
)
payloads: list[bytes] = []
for i in range(args.frames):
payload = read_frame(sock)
digest = hashlib.sha256(payload).hexdigest()[:16]
print(f"frame {i}: {len(payload)} bytes sha256:{digest}", flush=True)
if len(payload) < args.min_bytes:
raise ContractError(
f"frame {i} is {len(payload)} bytes, below the {args.min_bytes}-byte floor"
)
payloads.append(payload)
if args.require_png:
info = inspect_png(payload)
print(f"frame {i}: png {json.dumps(info, sort_keys=True)}", flush=True)
if (info["width"], info["height"]) != (hello["width"], hello["height"]):
raise ContractError(
f"frame {i} IHDR is {info['width']}x{info['height']} but the Hello "
f"announced {hello['width']}x{hello['height']}"
)
if args.reject_blank:
floor = blank_png_size(
info["width"], info["height"], info["colour_type"], info["bit_depth"]
)
print(f"frame {i}: blank-png floor is {floor} bytes", flush=True)
if info["nonzero_raw_bytes"] == 0:
raise ContractError(
f"frame {i} decodes to {info['raw_bytes']} zero bytes: "
"this is a zero-filled buffer, not a captured frame"
)
if info["distinct_raw_bytes"] < 2:
raise ContractError(
f"frame {i} decodes to a single repeated byte value "
f"({info['raw_bytes']} bytes): a blank image, not a capture"
)
if len(payload) <= floor:
raise ContractError(
f"frame {i} is {len(payload)} bytes, no bigger than a "
f"best-case blank {info['width']}x{info['height']} PNG "
f"({floor} bytes)"
)
if args.identical_frames and len(payloads) > 1:
first = payloads[0]
for i, payload in enumerate(payloads[1:], start=1):
if payload != first:
raise ContractError(
f"frame {i} differs from frame 0 ({len(payload)} vs {len(first)} "
"bytes) but the fake capture emits one fixed image"
)
print(f"all {len(payloads)} frames byte-identical", flush=True)
finally:
sock.close()
def main(argv: list[str] | None = None) -> int:
args = build_parser().parse_args(argv)
if args.frames < 1:
print("assert-stream: --frames must be at least 1", file=sys.stderr)
return 2
try:
run(args)
except ContractError as exc:
print(f"CONTRACT VIOLATION: {exc}", file=sys.stderr)
return 1
except (OSError, socket.timeout) as exc:
print(f"SOCKET ERROR on {args.socket}: {exc!r}", file=sys.stderr)
return 1
print("OK", flush=True)
return 0
if __name__ == "__main__":
sys.exit(main())
+89
View File
@@ -0,0 +1,89 @@
#!/usr/bin/env python3
"""A fake capture backend: writes one deterministic PNG to stdout and exits.
Stands in for `grim` so `vm-basic` can exercise the socket, the Hello and the
framing without a compositor. Two properties matter to the test:
* the bytes are a *valid* PNG of exactly the requested size, so a server that
derives the Hello's width/height from the captured image agrees with a server
that derives them from the configured output;
* the bytes are *byte-identical* on every invocation, so the test can assert
that two frames off the wire are the same image rather than the same length.
Pure standard library (zlib + struct), because the test VM has nothing else.
"""
from __future__ import annotations
import argparse
import struct
import sys
import zlib
def chunk(kind: bytes, data: bytes) -> bytes:
return (
struct.pack(">I", len(data))
+ kind
+ data
+ struct.pack(">I", zlib.crc32(kind + data) & 0xFFFFFFFF)
)
def encode_png(width: int, height: int) -> bytes:
"""A solid frame with one deterministic marker rectangle in the corner.
The marker keeps the image from being uniform, which is what makes it a
believable stand-in for a real capture; being fixed keeps it reproducible.
"""
bg = (0x20, 0x30, 0x40)
fg = (0xE0, 0xC0, 0x10)
mark_w = min(64, width)
mark_h = min(32, height)
plain = b"\x00" + bytes(bg) * width
marked = b"\x00" + bytes(fg) * mark_w + bytes(bg) * (width - mark_w)
raw = marked * mark_h + plain * (height - mark_h)
ihdr = struct.pack(">IIBBBBB", width, height, 8, 2, 0, 0, 0) # 8-bit truecolour
return (
b"\x89PNG\r\n\x1a\n"
+ chunk(b"IHDR", ihdr)
+ chunk(b"IDAT", zlib.compress(raw, 9))
+ chunk(b"IEND", b"")
)
def build_parser() -> argparse.ArgumentParser:
p = argparse.ArgumentParser(description=__doc__)
p.add_argument("--width", type=int, default=1280)
p.add_argument("--height", type=int, default=720)
# Accepted and ignored so the fake can be dropped in where a grim-shaped
# argv is expected (`-t png -`).
p.add_argument("-t", "--type", default="png")
p.add_argument("output", nargs="?", default="-")
return p
def main(argv: list[str] | None = None) -> int:
args = build_parser().parse_args(argv)
if args.type != "png":
print(f"fake-capture: only png is supported, got {args.type!r}", file=sys.stderr)
return 2
if args.width <= 0 or args.height <= 0:
print("fake-capture: width and height must be positive", file=sys.stderr)
return 2
data = encode_png(args.width, args.height)
out = sys.stdout.buffer if args.output == "-" else open(args.output, "wb")
try:
out.write(data)
out.flush()
finally:
if out is not sys.stdout.buffer:
out.close()
return 0
if __name__ == "__main__":
sys.exit(main())
+82
View File
@@ -0,0 +1,82 @@
#!/bin/sh
# Dump everything worth knowing when `vm-hyprland` fails.
#
# The compositor's own log is the thing you actually want and the thing that is
# hardest to find: Hyprland writes it under
# $XDG_RUNTIME_DIR/hypr/<instance-signature>/hyprland.log, and the service runs
# on a private XDG_RUNTIME_DIR. So rather than guess the path, look for it.
#
# Never fails: a diagnostics dump that exits non-zero hides the real error.
PATH="@diagPath@:$PATH"
export PATH
if [ "$1" = "--self-test" ]; then
echo "hhs-hyprland-diagnostics: ok"
exit 0
fi
SERVICE_UNIT=hyprland-headless-stream.service
COMPOSITOR_UNIT=hyprland-headless.service
say() { printf '\n----- %s -----\n' "$*"; }
run() { say "$*"; "$@" 2>&1 || echo "[exit $?]"; }
say "units"
systemctl --no-pager --plain list-units --all 'hyprland*' 2>&1 || true
for unit in "$COMPOSITOR_UNIT" "$SERVICE_UNIT"; do
run systemctl --no-pager --full status "$unit"
run journalctl --no-pager --boot --lines=200 --unit "$unit"
done
run systemctl --no-pager --full list-dependencies "$SERVICE_UNIT"
say "socket and runtime directories"
ls -la /run/hyprland-headless-stream 2>&1 || echo "[no /run/hyprland-headless-stream]"
ls -la /run/user 2>&1 || echo "[no /run/user]"
for d in /run/user/*; do
[ -d "$d" ] && ls -la "$d" 2>&1
done
say "hyprland logs"
# hyprland.log lives beside the compositor's IPC sockets; -xdev keeps this out
# of /nix/store and /proc.
found=0
for root in /run /var/log /tmp /home; do
[ -d "$root" ] || continue
for log in $(find "$root" -xdev -maxdepth 6 -name 'hyprland*.log' 2>/dev/null); do
found=1
say "tail -n 200 $log"
tail -n 200 "$log" 2>&1
done
done
[ "$found" = 1 ] || echo "[no hyprland*.log anywhere under /run /var/log /tmp /home]"
say "hyprctl monitors"
# Needs both XDG_RUNTIME_DIR and the instance signature, which is the directory
# name under <runtime>/hypr. Try every instance we can find.
tried=0
for hyprdir in /run/user/*/hypr/* /run/*/hypr/*; do
[ -d "$hyprdir" ] || continue
sig=$(basename "$hyprdir")
runtime=$(dirname "$(dirname "$hyprdir")")
owner=$(stat -c %U "$hyprdir" 2>/dev/null || echo root)
tried=1
say "hyprctl monitors (instance $sig, XDG_RUNTIME_DIR=$runtime, as $owner)"
if command -v hyprctl > /dev/null 2>&1; then
XDG_RUNTIME_DIR="$runtime" HYPRLAND_INSTANCE_SIGNATURE="$sig" \
hyprctl monitors 2>&1 || echo "[exit $?]"
XDG_RUNTIME_DIR="$runtime" HYPRLAND_INSTANCE_SIGNATURE="$sig" \
hyprctl version 2>&1 || echo "[exit $?]"
else
echo "[hyprctl not on PATH; add the hyprlandPackage to environment.systemPackages]"
fi
done
[ "$tried" = 1 ] || echo "[no hypr instance directory found: the compositor never came up]"
say "processes"
ps aux 2>&1 | grep -E 'hypr|hhstream|grim' | grep -v grep || echo "[no hyprland/hhstream/grim process]"
say "end of diagnostics"
exit 0
+38
View File
@@ -0,0 +1,38 @@
"""The parts of the package that are plain constants, checked against the contract.
These are the only tests here that do not depend on an implementation, which
makes them the canary for the `unit` check's own wiring: if they fail, the
package is not importable from the build sandbox and nothing else means anything.
"""
from __future__ import annotations
import hhstream
from hhstream import capture, client, protocol, server
from conftest import CONTRACT_SOCKET
def test_magic_matches_the_contract():
# docs/CONTRACT.md: {"magic":"HHS1", ...}
assert hhstream.MAGIC == "HHS1"
assert protocol.MAGIC == "HHS1"
def test_default_socket_path_matches_the_contract():
assert server.DEFAULT_SOCKET == CONTRACT_SOCKET
assert client.DEFAULT_SOCKET == CONTRACT_SOCKET
def test_public_surface_is_present():
for name in ("hello", "parse_hello", "frame", "read_frame", "read_hello"):
assert callable(getattr(protocol, name)), f"protocol.{name}"
assert callable(client.fetch)
assert callable(client.main)
assert callable(server.main)
assert isinstance(server.FrameServer, type)
assert isinstance(capture.CommandCapture, type)
def test_version_is_the_packaged_version():
assert hhstream.__version__ == "0.1.0"