Item 1 said the broker reads none of the container or pointer configuration. That was true and is not: the settings are parsed beside Policy and a switched-off container host reports itself unavailable with the key named. The item now records what actually matters about it -- that containers are off on the deployment machine because no window can be put in one -- and the section links docs/TODO/, where each open problem has its impact written down. The TODO folder itself landed in the previous commit by accident (a `git add -A docs` swept it in with the scroll work); this commit only makes it findable.
331 lines
17 KiB
Markdown
331 lines
17 KiB
Markdown
# Developing and testing `hyprland-headless-stream-plugin`
|
|
|
|
The interface everything here asserts is [`docs/CONTRACT.md`](docs/CONTRACT.md).
|
|
That document is authoritative: if a test and the code disagree, the document
|
|
decides, and if the document is wrong it gets a version bump, not an edit.
|
|
|
|
## Prerequisites
|
|
|
|
- Nix with flakes enabled.
|
|
- `/dev/kvm` readable, for the six VM checks. They are `nixosTest`s; without
|
|
KVM they still run, just slowly enough that the timeouts below get tight.
|
|
- No GPU. Nothing in the test suite needs one — the compositor runs on
|
|
Hyprland's headless backend over software rendering.
|
|
|
|
Nothing in the checks touches the host: no `/etc/nixos`, no network, and no
|
|
socket outside the build sandbox or the test VM.
|
|
|
|
## The dev shell
|
|
|
|
```bash
|
|
nix develop
|
|
```
|
|
|
|
Puts `python3`, `pytest`, `hyprland`, `grim`, `ffmpeg`, `jq` and `tmux` on PATH
|
|
and sets `PYTHONPATH=$PWD/src`, so `import hhstream` picks up the working tree
|
|
rather than an installed copy.
|
|
|
|
## Build
|
|
|
|
```bash
|
|
nix build .#hhstream # the package
|
|
ls result/bin # hhstream-server, hhstream-client, hhstream,
|
|
# hhstreamd, hhstream-pointer
|
|
```
|
|
|
|
Five entry points: the frame server and a reference reader for it, the `hhstream`
|
|
front-door CLI (frames, `run`, and the `session` namespace), the session broker,
|
|
and the pointer driver.
|
|
|
|
## The checks
|
|
|
|
| Check | What it proves | Needs a VM |
|
|
|---|---|---|
|
|
| `unit` | the framing, the capture backend, the server's client handling, the broker, the CLI, the containers, the lease and the pointer driver — in-process, against the `nix/package.nix` build | no |
|
|
| `vm-basic` | the NixOS module brings up both units and publishes a contract-conformant socket, with a **fake** capture backend | yes |
|
|
| `vm-hyprland` | the same, capturing the **real** headless compositor through `grim` | yes |
|
|
| `vm-sessions` | the **session broker**: a real GUI application launched into the compositor as a tracked session, seen by the compositor's own IPC, killed as a tree, reaped on its idle deadline (and *not* reaped when touched), and adopted back after the broker restarts | yes |
|
|
| `vm-ipc` | the **compositor's IPC**: both sockets exist at the names `docs/CONTRACT-sessions.md` §5 freezes and fit inside `sun_path`, `hyprctl` answers, the broker's own `HyprlandIPC` reaches it, and a subscriber on the **event** socket receives an `openwindow` for a window the test creates | yes |
|
|
| `vm-containers` | one `hh-*` output per session at the container mode, the application's window on it, a second window of the same application beside it, an unmatched window **quarantined**, `grim -o` capturing that container alone, and teardown removing the output and freeing its workspace slot | yes |
|
|
| `vm-pointer` | the **lease and the driver**: a version-2 virtual pointer, a warp that lands where it was aimed *as the application reports it*, a click delivered as one press and one release, a contended acquire refused with BUSY, an expired lease swept with its held button lifted, and a broker restart that leaves the lock free, the bookmark intact and the driver reachable | yes |
|
|
|
|
All of them at once:
|
|
|
|
```bash
|
|
nix flake check -L
|
|
```
|
|
|
|
`nix flake check` stops at the first failing check, which makes it a poor way to
|
|
see the whole picture while the tree is still red. **Run the VM checks one at a
|
|
time** anyway — each boots a guest with a software-rendered compositor, and two
|
|
at once on a two-core machine make the timing-sensitive assertions flake. A
|
|
flaky green is worth less than a slow one.
|
|
|
|
```bash
|
|
nix build -L .#checks.x86_64-linux.unit
|
|
nix build -L .#checks.x86_64-linux.vm-basic
|
|
nix build -L .#checks.x86_64-linux.vm-hyprland
|
|
nix build -L .#checks.x86_64-linux.vm-sessions
|
|
nix build -L .#checks.x86_64-linux.vm-ipc
|
|
nix build -L .#checks.x86_64-linux.vm-containers
|
|
nix build -L .#checks.x86_64-linux.vm-pointer
|
|
```
|
|
|
|
`-L` is not optional in practice: without it, Nix hides the build log, and the
|
|
build log *is* the test report. Do **not** pipe a check into `tail` or `grep`:
|
|
the pipeline's exit status is the last command's, so a build that cannot fail
|
|
is not a test. Redirect to a file instead.
|
|
|
|
Each VM check's test script is type-checked before the guest boots, which is
|
|
much cheaper than finding a typo after a five-minute boot:
|
|
|
|
```bash
|
|
nix build -L .#checks.x86_64-linux.vm-pointer.driver --no-link
|
|
```
|
|
|
|
Watch for names the driver already defines — `log` is an `AbstractLogger` in
|
|
that namespace, and assigning a string to it is a type error, not a shadowing
|
|
warning.
|
|
|
|
### `unit`
|
|
|
|
pytest over `tests/unit/`, run in the build sandbox with `PYTHONPATH` pointing at
|
|
what `nix/package.nix` installed (`nix/package.nix` sets `doCheck = false` on
|
|
purpose, so this is the only place the suite runs in a build).
|
|
|
|
The check fails if the suite is **empty**, not just if it is red. `pytest` exits
|
|
5 on "collected nothing", but it exits 0 when every test was skipped or
|
|
deselected, so `checks/lib/check_report.py` parses the JUnit report and refuses
|
|
a run with zero collected or zero executed tests. A green check that tested
|
|
nothing is worse than no check. On success the report is copied to
|
|
`result/junit.xml`.
|
|
|
|
To run the same tests directly, without Nix:
|
|
|
|
```bash
|
|
nix develop
|
|
pytest tests/unit -v # PYTHONPATH=$PWD/src comes from the shell
|
|
pytest tests/unit/test_protocol.py -v # one file
|
|
pytest tests/unit -k "hello or frame" -v # one theme
|
|
pytest tests/unit -x --ff # stop at the first failure, failures first
|
|
```
|
|
|
|
Note that the unit suite's *negative* tests use the `rejects` fixture from
|
|
`tests/unit/conftest.py` rather than `pytest.raises(Exception)`. `rejects`
|
|
treats `NotImplementedError` as a failure, because a plain `raises(Exception)`
|
|
is satisfied by an unimplemented stub — the test would then be green precisely
|
|
because nothing had been written.
|
|
|
|
### `vm-basic`
|
|
|
|
Boots one machine with `services.hyprland-headless-stream.enable = true` and
|
|
`captureCommand` pointed at `tests/fake_capture.py`, which writes one fixed,
|
|
deterministic PNG of the configured size to stdout. Because the image never
|
|
changes, "two frames arrived and they are the *same bytes*" is a real assertion
|
|
about the framing rather than a coincidence.
|
|
|
|
It checks, in order: both units start; the socket exists and is
|
|
`hhstream:hhstream` mode `0660`; the Hello parses with `magic = "HHS1"` and
|
|
non-zero width/height matching the configured output; two frames arrive with
|
|
length > 0, identical bytes, and an IHDR matching the Hello; two clients are
|
|
served concurrently; and a client that dies mid-frame leaves the server serving.
|
|
|
|
The reader is `tests/assert_stream.py`, a second, independent implementation of
|
|
the contract that shares no code with `hhstream.client` — so a framing bug
|
|
present in both the server's writer and the shipped client's reader still fails
|
|
the test. It validates the length prefix strictly (`0|[1-9][0-9]*`: no leading
|
|
zeros, no sign, no spaces), never reads past a newline, and prints the Hello,
|
|
every frame's size and SHA-256, and the parsed IHDR before it decides anything.
|
|
|
|
### `vm-hyprland`
|
|
|
|
The same machine with **no** `captureCommand` override, so the module's default
|
|
(`grim`) captures the real compositor.
|
|
|
|
This is the check expected to be flaky, so it is deliberately impatient: every
|
|
wait has a bounded timeout, nothing is retried in a loop, and any failure dumps
|
|
diagnostics before it propagates. It asserts a frame is a valid PNG whose IHDR
|
|
dimensions match the configured output, and that it is more than a blank buffer:
|
|
|
|
- the IDAT must decompress;
|
|
- the decompressed scanlines must contain a non-zero byte and at least two
|
|
distinct byte values — a zero-filled framebuffer fails both;
|
|
- the payload must be larger than a best-case blank PNG of the same geometry
|
|
(`blank_png_size()` in `tests/assert_stream.py`: an all-zero raster at zlib
|
|
level 9, i.e. the smallest a blank frame of that size can be encoded).
|
|
|
|
It also runs `grim` directly against the session as a separate subtest, which
|
|
separates "grim cannot see the compositor" from "the server is wrong" — the two
|
|
failures that look identical from the socket.
|
|
|
|
## When `vm-hyprland` fails: where the logs are
|
|
|
|
Every failing subtest in both VM checks runs `hhs-hyprland-diagnostics`
|
|
(`tests/hyprland_diagnostics.sh`) in the guest first and prints its output to the
|
|
build log. So the answer to "where are the logs" is usually **in the build log
|
|
you already have**, above the traceback, between
|
|
`########## diagnostics: ... ##########` markers. Re-read it with:
|
|
|
|
```bash
|
|
nix log .#checks.x86_64-linux.vm-hyprland
|
|
```
|
|
|
|
What the dump contains, and where it comes from in the guest:
|
|
|
|
| Section | Source |
|
|
|---|---|
|
|
| `units` | `systemctl list-units --all 'hyprland*'` |
|
|
| per-unit `status` / `journalctl` | `hyprland-headless.service` and `hyprland-headless-stream.service`, 200 lines each |
|
|
| `socket and runtime directories` | `/run/hypr-stream`, `/run/user/*` |
|
|
| `hyprland logs` | every `hyprland*.log` found under `/run`, `/var/log`, `/tmp`, `/home` |
|
|
| `hyprctl monitors` | run once per instance found, with the right `XDG_RUNTIME_DIR` and `HYPRLAND_INSTANCE_SIGNATURE` |
|
|
| `processes` | `ps aux` filtered to `hypr`, `hhstream`, `grim` |
|
|
|
|
Hyprland writes its log to
|
|
`$XDG_RUNTIME_DIR/hypr/$HYPRLAND_INSTANCE_SIGNATURE/hyprland.log`, and the
|
|
compositor runs on a *private* `XDG_RUNTIME_DIR`, so the path is not knowable in
|
|
advance — that is why the script searches for it rather than naming it. The
|
|
instance signature is the directory name under `<runtime>/hypr`, which is also
|
|
what `hyprctl` needs in the environment:
|
|
|
|
```bash
|
|
XDG_RUNTIME_DIR=/run/user/$(id -u hhstream) \
|
|
HYPRLAND_INSTANCE_SIGNATURE=$(ls /run/user/$(id -u hhstream)/hypr | head -1) \
|
|
hyprctl monitors
|
|
```
|
|
|
|
### Poking at a live VM
|
|
|
|
The nicest way to debug a VM failure is to keep the machine:
|
|
|
|
```bash
|
|
nix build .#checks.x86_64-linux.vm-hyprland.driverInteractive
|
|
./result/bin/nixos-test-driver
|
|
```
|
|
|
|
That drops you in a Python REPL with `machine` bound. `start_all()`, then
|
|
`machine.shell_interact()` for a root shell in the guest, or run the test script
|
|
a line at a time. `machine.succeed("hhs-hyprland-diagnostics")` works there too.
|
|
|
|
## Layout
|
|
|
|
```
|
|
checks/
|
|
default.nix wiring only: the checks the flake exposes
|
|
unit.nix pytest in the sandbox, against the nix/package.nix build
|
|
vm-basic.nix nixosTest, fake capture backend
|
|
vm-hyprland.nix nixosTest, real compositor + grim
|
|
vm-sessions.nix nixosTest, the broker's lifecycle against the compositor
|
|
vm-ipc.nix nixosTest, the compositor's request and event sockets
|
|
vm-containers.nix nixosTest, one output and one workspace per session
|
|
vm-pointer.nix nixosTest, the lease, the driver and a click that lands
|
|
lib/test-tools.nix packages the tests/ helpers for the guest
|
|
lib/session-tools.nix packages the two guest programs below
|
|
lib/session-machine.nix the guest the three session checks share
|
|
lib/session-prelude.nix their test-script helpers (no assertions live here)
|
|
lib/gpu-session.nix the hardware a bare test machine does not have
|
|
lib/check_report.py JUnit report -> build verdict (and the empty-suite guard)
|
|
lib/ipc_watch.py subscribes to the event socket: hhs-ipc-watch
|
|
lib/pointer_target.py a Wayland window that reports pointer events it receives:
|
|
hhs-pointer-target
|
|
tests/
|
|
fake_capture.py writes one fixed PNG to stdout: hhs-fake-capture
|
|
assert_stream.py independent contract reader/asserter: hhs-assert-stream
|
|
hyprland_diagnostics.sh the failure dump: hhs-hyprland-diagnostics
|
|
unit/ ipc/ containers/ pointer/ sessions/ the pytest suite
|
|
```
|
|
|
|
`checks/lib/test-tools.nix` and `checks/lib/session-tools.nix` self-test their
|
|
programs at build time (the fake emits a PNG of the size it was asked for; the
|
|
asserter starts; the event subscriber can import the broker's IPC client; the
|
|
pointer target refuses to start without an app id), because a broken helper
|
|
otherwise surfaces as a baffling failure three layers away.
|
|
|
|
`hhs-ipc-watch` deliberately runs the **shipped** `hhstreamd.ipc.HyprlandIPC`
|
|
rather than a second implementation — the opposite choice from
|
|
`tests/assert_stream.py`. The difference is what each proves: `assert_stream`
|
|
asserts a wire format two programs must agree on, so a shared bug must still
|
|
fail it; `vm-ipc` asserts that *this* client receives events from *this*
|
|
compositor, which a reimplementation could not say anything about.
|
|
|
|
`hhs-pointer-target` exists because no application in nixpkgs does both of the
|
|
things the session checks need: set its `xdg_toplevel` app id from
|
|
`$HHSTREAM_APP_ID` (so its window matches its session's container rule) and
|
|
print the pointer events it receives with the coordinates it was given.
|
|
`foot` and `wev` do neither and ignore the environment entirely, which makes
|
|
them good *unmatched* windows and useless as matched ones.
|
|
|
|
Adding a unit test: drop a `test_*.py` under `tests/`. Nothing needs
|
|
registering, but the basename must be unique across the tree. Adding a check:
|
|
one file in `checks/`, one line in `checks/default.nix`.
|
|
|
|
## Known gaps in the interface (as of this branch)
|
|
|
|
Every item the previous revision of this section listed is closed: the module
|
|
has a `config` side, `package` has a default, `src/hhstream/*.py` is written,
|
|
and the frame server's command line is the one the module passes. What is open
|
|
is below. These are defects in code this branch does not own; they are recorded
|
|
here because the checks are written against the contract and therefore trip
|
|
over them.
|
|
|
|
The deep versions of these — the issue, more than one idea, and why each one
|
|
matters — live in [`docs/TODO/`](docs/TODO/README.md). Update the file there
|
|
when one closes, so this list stays a description of what is still true.
|
|
|
|
1. **The container host can be switched off, and the broker now honours it.**
|
|
*(This item previously read "the broker reads none of the container or
|
|
pointer configuration." That is fixed: `policy.py` parses those settings
|
|
beside `Policy`, read in `__main__`, and a host that is switched off reports
|
|
itself unavailable with the reason naming the key — so a disabled machine
|
|
cannot be mistaken for one whose compositor is down. The reason it matters is
|
|
what follows.)* Containers are consequently **off on the deployment machine**:
|
|
no window can be put in one yet, and enabling them quarantines every
|
|
allowlisted app. That join is the first item in `docs/TODO/`.
|
|
|
|
2. **`session start --no-container` is refused, not honoured.** PLAN §9 lists
|
|
the flag; `docs/CONTRACT-sessions.md` §4 leaves `start` unchanged, and
|
|
`SessionManager.start` takes no such argument. A request carrying a
|
|
`no_container` field would be accepted and ignored, and the session would
|
|
get a container anyway — so the CLI exits 2 and names the disagreement
|
|
instead of lying about what it did.
|
|
|
|
3. **`nix/hyprland-conf.nix` sets `input:follow_mouse = 0`, and that stops
|
|
pointer input reaching anything but the focused window.** The comment
|
|
justifying it — "No pointer will ever exist; do not let focus depend on
|
|
one" — was true before this build and is false now.
|
|
|
|
Measured by `vm-pointer`, with two sessions and the second holding focus
|
|
because its window opened last:
|
|
|
|
```
|
|
aimed at target-le22h7: it reports [] <- the session the warp named
|
|
focused is target-s76taj: it reports [] <- the session with focus
|
|
cursor now at {'x': 2048, 'y': 288} <- exactly where it was aimed
|
|
```
|
|
|
|
The compositor moved the cursor to the right pixel, inside the right
|
|
window, on the right output, and told no client at all. Every warp and
|
|
click in the subtests before that one is delivered perfectly, because there
|
|
the aimed-at session is also the focused one — which is why nothing noticed
|
|
until a check put two sessions on the screen at once.
|
|
|
|
It defeats the point of containers and the lease: an agent could only drive
|
|
whichever session the compositor last focused. `vm-pointer` overrides the
|
|
single line through the module's `extraConfig` and asserts the property
|
|
that fails without it, so the check is green and the defect is still
|
|
visible. The fix is one line in `nix/hyprland-conf.nix`; whether it should
|
|
be `1` (Hyprland's own default, keyboard focus follows the cursor too) or a
|
|
looser setting that leaves keyboard focus alone is a decision for the
|
|
keyboard work in PLAN §6.7.
|
|
|
|
4. **A container's window rule only binds for an application that reads the
|
|
environment.** The broker's only lever on an application's class is
|
|
`$HHSTREAM_APP_ID` and the three toolkit variables beside it
|
|
(`hhstreamd/launcher.py`), so a GTK or Qt application — and `foot`, and
|
|
`wev` — keeps its own class, matches no per-session rule, and is swept into
|
|
the quarantine workspace. That is reported by the read-back rather than
|
|
papered over, and it is why `vm-containers` and `vm-pointer` use a
|
|
cooperating client (`hhs-pointer-target`) for the matched case and `foot`
|
|
for the unmatched one. Closing it needs a per-app `--app-id` in the
|
|
allowlist, which is a policy change.
|