Three layers, because the stream has three and each one can be wrong on its own. `tests/live/test_live_control.py` (20 tests) runs `Control` against a fake capture: the four ops, `attach` idempotent on the phrase, the same phrase aimed at another output refused, a malformed phrase refused *without the value in the message*, and `fps` defaulting to the daemon's when the broker sends none. `tests/sessions/unit/test_stream_verbs.py` (24 tests) runs the broker against a fake daemon, and its subjects are the decisions rather than the plumbing: no container output is a refusal and never a fallback to the main output; a re-`enable` returns the same phrase and a different one is exit 8; a down daemon turns `attached` false rather than failing a read; the sweep touches a watched session and leaves an unwatched one alone; teardown detaches before the container is destroyed. `tests/unit/test_mcp_*` grow the two tools, the two resources and the three `session_start` arguments, with the read-only half asserted *absent* as a tool by name -- re-adding one should be a failure, not a silent widening. `checks/vm-stream.nix` is the new guest check, and it exists for the two things none of the above can say: 1. **That `grim` captures a real compositor from inside a unit.** It reads actual JPEGs off `/<phrase>/stream.mjpg` with `checks/lib/mjpeg_read.py`, a hand-rolled multipart reader that shares no code with the server -- a parser that did could not tell you the boundary was wrong. It framed on SOI/EOI rather than `Content-Length` for the same reason. This is what caught the unit's missing `XDG_RUNTIME_DIR`: `frames: [25827, 25827, 25827]` now, ten capture failures and a stream that gave up before. 2. **That a connected viewer moves the idle clock.** That needs the broker's real sweep, on a real timer, with a client genuinely still there a minute later -- hence `--hold`, and hence a check that spends ~90 seconds twice. The negative control runs first, because if the sweep touched an unwatched stream the positive case would pass for the wrong reason. Measured in the guest: with nobody watching `580s -> 502s` remaining, with a viewer `499s -> 596s`. It also asserts the phrase is in neither the audit log nor a 404 body, that the old URL stops serving after `disable`, that `disable` then `enable` rotates it, that `kill` takes the stream down and leaves the daemon up, and that `start --stream` yields an id and a URL that serves frames. No clock is injected anywhere in this check. An injected one would prove the arithmetic, which the unit tests already do, and not the daemon. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011ZeH65zd6pFTgBSu52W9PE
373 lines
16 KiB
Nix
373 lines
16 KiB
Nix
# checks.<system>.vm-stream -- the per-session live stream, end to end, with a
|
|
# real HTTP viewer inside the guest.
|
|
#
|
|
# The session machine plus `sessions.stream.enable = true`, which runs
|
|
# `hhstream-live` and hands the broker `/run/hhstream/live.sock`. Then:
|
|
# `hhstream session stream enable` on a live session, and a client that reads
|
|
# actual JPEGs off `/<phrase>/stream.mjpg`.
|
|
#
|
|
# WHAT THIS PROVES THAT THE UNIT TESTS CANNOT.
|
|
#
|
|
# `tests/live/test_live_control.py` and `tests/sessions/unit/test_stream_verbs.py`
|
|
# already run both ends of `live.sock` against each other with a fake capture.
|
|
# What they cannot say is that `grim` captures a real compositor from inside a
|
|
# *unit* -- which is the one failure this feature has already been bitten by
|
|
# once elsewhere in this tree (`docs/TODO/capture-binary-and-the-unit-path.md`:
|
|
# a unit's PATH has no `grim`, and the symptom is an empty frame area with no
|
|
# error). So the capture command is passed explicitly by the module and this
|
|
# check reads the bytes that come out of it.
|
|
#
|
|
# It also proves the one thing no in-process test can: that a *connected
|
|
# viewer* moves the idle clock. That needs the broker's real sweep, on a real
|
|
# timer, with a client that is genuinely still there a minute later -- which is
|
|
# why `hhs-mjpeg-read --hold` exists and why this check spends real time.
|
|
#
|
|
# THE TWO REAL-TIME WAITS ARE THE POINT OF THEIR SUBTESTS. The sweep runs once
|
|
# a minute (`hhstreamd.manager.STREAM_SWEEP_S`), and both the positive case and
|
|
# its negative control have to outlast one pass. No clock is injected anywhere:
|
|
# an injected one would prove the arithmetic, which the unit tests already do,
|
|
# and not the daemon.
|
|
#
|
|
# Habits copied from the other session checks: every wait is bounded, each
|
|
# subtest asserts once, and a failure dumps the compositor's diagnostics and
|
|
# the broker's journal before it propagates.
|
|
{
|
|
testers,
|
|
hhstream,
|
|
hhsModule,
|
|
testTools,
|
|
hyprland,
|
|
foot,
|
|
jq,
|
|
grim,
|
|
procps,
|
|
coreutils,
|
|
dejavu_fonts,
|
|
width ? 1280,
|
|
height ? 720,
|
|
sessionUser ? "hermes",
|
|
port ? 9988,
|
|
}:
|
|
testers.runNixOSTest (
|
|
{ pkgs, ... }:
|
|
let
|
|
tools = pkgs.callPackage ./lib/session-tools.nix { inherit hhstream; };
|
|
guest = import ./lib/session-machine.nix {
|
|
inherit
|
|
hhsModule
|
|
hhstream
|
|
hyprland
|
|
testTools
|
|
foot
|
|
jq
|
|
procps
|
|
grim
|
|
coreutils
|
|
dejavu_fonts
|
|
;
|
|
sessionTools = tools;
|
|
};
|
|
prelude = import ./lib/session-prelude.nix {
|
|
inherit hhstream coreutils sessionUser;
|
|
};
|
|
|
|
# The viewer. Plain python3 and a hand-rolled multipart reader, so the
|
|
# framing this repository serves is read by something that does not share
|
|
# code with it -- the same choice `tests/assert_stream.py` makes about the
|
|
# frame socket, for the same reason.
|
|
viewer = pkgs.runCommandLocal "hhs-mjpeg-read" { } ''
|
|
install -Dm755 ${./lib/mjpeg_read.py} $out/bin/hhs-mjpeg-read
|
|
sed -i '1s|.*|#!${pkgs.python3}/bin/python3|' $out/bin/hhs-mjpeg-read
|
|
$out/bin/hhs-mjpeg-read --help > /dev/null
|
|
'';
|
|
in
|
|
{
|
|
name = "hhstream-vm-stream";
|
|
|
|
# Boot, plus two sweep windows of real waiting (~70 s each) and a session
|
|
# launch. Capped so a hang is a failure rather than a wedged builder.
|
|
globalTimeout = 2400;
|
|
|
|
nodes.machine = {
|
|
imports = [
|
|
(guest {
|
|
inherit width height sessionUser;
|
|
apps.foot = {
|
|
exec = "${foot}/bin/foot";
|
|
argsAllow = [ ];
|
|
};
|
|
})
|
|
];
|
|
|
|
services.hyprland-headless-stream.sessions.stream = {
|
|
enable = true;
|
|
# Loopback: the viewer runs inside the guest. Reaching it from outside
|
|
# is `vm-stream-firewall`'s subject, and opening a port here would make
|
|
# two checks assert the same thing badly.
|
|
bind = "127.0.0.1";
|
|
inherit port;
|
|
};
|
|
|
|
environment.systemPackages = [ viewer ];
|
|
};
|
|
|
|
testScript =
|
|
prelude
|
|
+ ''
|
|
# `machine.sleep` wants a `timedelta`: a bare number is deprecated as
|
|
# `secs=` and refused outright by the driver's type check as
|
|
# `duration=`. MEASURED, as `nix build` failing before the VM even
|
|
# booted: "Expected `timedelta | None`, found `Literal[75]`".
|
|
import datetime as dt
|
|
|
|
READ = "${viewer}/bin/hhs-mjpeg-read"
|
|
PORT = ${toString port}
|
|
LIVE_SOCKET = "/run/hhstream/live.sock"
|
|
#: `hhstreamd.manager.STREAM_SWEEP_S`, plus room for one round trip.
|
|
SWEEP_WINDOW_S = 75
|
|
|
|
|
|
def expires_in(session_id):
|
|
"""The session's own idle deadline, as the broker reports it."""
|
|
|
|
return cli_json(f"status {session_id} --json")["session"]["expires_in_s"]
|
|
|
|
|
|
def stream_json(args=""):
|
|
return cli_json(f"stream {args} --json".strip())
|
|
|
|
|
|
start_all()
|
|
bring_up("the compositor and broker")
|
|
|
|
with subtest("the as-user wrapper propagates exit status"):
|
|
machine.succeed(as_user(TRUE), timeout=120)
|
|
machine.fail(as_user(FALSE), timeout=120)
|
|
|
|
with subtest("the stream daemon came up and owns its socket, not the broker's"):
|
|
try:
|
|
machine.wait_for_unit("hhstream-live.service", timeout=180)
|
|
machine.wait_for_file(LIVE_SOCKET, timeout=120)
|
|
restarts = machine.succeed(
|
|
"systemctl show -p NRestarts --value hhstream-live.service"
|
|
).strip()
|
|
assert restarts == "0", (
|
|
f"hhstream-live has restarted {restarts} time(s): it is "
|
|
"crash-looping, not running"
|
|
)
|
|
# The risk the plan names by name: a second RuntimeDirectory on
|
|
# /run/hhstream would have deleted the broker's socket.
|
|
machine.succeed(f"test -S {CONTROL}")
|
|
machine.succeed(f"test -S {LIVE_SOCKET}")
|
|
# And it is listening on the port it was given, with no stream
|
|
# on it: running the daemon turns nothing on.
|
|
machine.wait_until_succeeds(
|
|
f"ss -ltn | grep -q 127.0.0.1:{PORT}", timeout=60
|
|
)
|
|
assert stream_json("list")["streams"] == []
|
|
except Exception:
|
|
dump("the stream daemon did not come up")
|
|
raise
|
|
|
|
with subtest("a session with no stream reports none"):
|
|
session_id = machine.succeed(cli("start foot"), timeout=300).strip()
|
|
print(f"session {session_id}")
|
|
assert stream_json(f"status {session_id}")["stream"] is None
|
|
|
|
with subtest("enabling mints a phrase and the daemon publishes the output"):
|
|
try:
|
|
url = machine.succeed(
|
|
cli(f"stream enable {session_id}"), timeout=180
|
|
).strip()
|
|
print(f"stream url: {url}")
|
|
assert url.startswith(f"http://127.0.0.1:{PORT}/"), url
|
|
|
|
status = stream_json(f"status {session_id}")["stream"]
|
|
assert status["attached"] is True, status
|
|
assert status["output"] == f"hh-{session_id}", status
|
|
assert status["url"] == url, status
|
|
# The phrase is in the URL and in the record, and nowhere else.
|
|
assert status["phrase"] in url
|
|
except Exception:
|
|
dump("enabling the stream", session_id)
|
|
raise
|
|
|
|
with subtest("the phrase is not in the audit log"):
|
|
# It is the stream's only credential, and the log is read by
|
|
# whoever can read logs (CONTRACT-sessions.md §3.5).
|
|
phrase = stream_json(f"status {session_id}")["stream"]["phrase"]
|
|
events = machine.succeed(
|
|
"cat /var/lib/hhstream/events.jsonl || true", timeout=60
|
|
)
|
|
assert "stream_enable" in events, "the enable was not audited at all"
|
|
assert phrase not in events, "the phrase reached the audit log"
|
|
|
|
with subtest("a real client reads JPEGs off the stream"):
|
|
# The subtest the unit tests cannot write: `grim` capturing a real
|
|
# compositor, from inside a unit, through the module's own capture
|
|
# command.
|
|
try:
|
|
out = machine.succeed(
|
|
as_user(f"{READ} {url}stream.mjpg --frames 3"), timeout=180
|
|
)
|
|
first = json.loads(out.strip().splitlines()[0])
|
|
print(f"viewer read: {first}")
|
|
assert first["status"] == 200, first
|
|
assert "multipart/x-mixed-replace" in first["content_type"], first
|
|
assert len(first["frames"]) == 3, first
|
|
# Real frames of a real desktop, not a 1x1 placeholder.
|
|
assert all(size > 1024 for size in first["frames"]), first["frames"]
|
|
except Exception:
|
|
dump("reading the stream", session_id)
|
|
raise
|
|
|
|
with subtest("a wrong phrase is a 404 and is never echoed back"):
|
|
wrong = "wrong-words-entirely-here-00000"
|
|
status, body = machine.execute(
|
|
as_user(f"{READ} http://127.0.0.1:{PORT}/{wrong}/stream.mjpg --frames 1"),
|
|
timeout=120,
|
|
)
|
|
assert status != 0, f"a wrong phrase was served: {body}"
|
|
assert wrong not in body, "the wrong phrase was echoed back"
|
|
|
|
with subtest("an enabled stream nobody watches keeps nothing alive"):
|
|
# The negative control, and it has to come first: if the sweep
|
|
# touched an unwatched stream, the positive case below would pass
|
|
# for the wrong reason.
|
|
try:
|
|
assert stream_json("list")["streams"][0]["viewers"] == 0
|
|
before = expires_in(session_id)
|
|
machine.sleep(duration=dt.timedelta(seconds=SWEEP_WINDOW_S))
|
|
after = expires_in(session_id)
|
|
print(f"with no viewer: {before:.0f}s -> {after:.0f}s remaining")
|
|
assert after < before, (
|
|
"the idle deadline did not move closer over a whole sweep "
|
|
"window with nobody watching, so something is touching this "
|
|
"session"
|
|
)
|
|
except Exception:
|
|
dump("the unwatched stream", session_id)
|
|
raise
|
|
|
|
with subtest("a connected viewer is activity and moves the idle deadline"):
|
|
# The one exception to "touch is the only mark of activity"
|
|
# (CONTRACT-sessions.md §4). The viewer holds its connection open
|
|
# across a sweep, because a client that read and left would make
|
|
# the sweep correct and untested.
|
|
try:
|
|
before = expires_in(session_id)
|
|
machine.succeed(
|
|
background(
|
|
"hhs-viewer",
|
|
f"{READ} {url}stream.mjpg --frames 2 "
|
|
f"--hold {SWEEP_WINDOW_S} --timeout {SWEEP_WINDOW_S + 60}",
|
|
user=USER,
|
|
),
|
|
timeout=120,
|
|
)
|
|
watching = wait_for(
|
|
lambda: stream_json("list")["streams"][0]["viewers"] > 0,
|
|
timeout=60,
|
|
what="the broker to see a connected viewer",
|
|
)
|
|
assert watching
|
|
after = wait_for(
|
|
lambda: expires_in(session_id) > before,
|
|
timeout=SWEEP_WINDOW_S + 60,
|
|
what="the viewer sweep to reset the idle clock",
|
|
)
|
|
print(f"with a viewer: {before:.0f}s -> {expires_in(session_id):.0f}s")
|
|
|
|
events = machine.succeed("cat /var/lib/hhstream/events.jsonl")
|
|
assert "stream_viewer_activity" in events
|
|
assert stream_json(f"status {session_id}")["stream"]["phrase"] not in events
|
|
except Exception:
|
|
dump("the watched stream", session_id)
|
|
raise
|
|
|
|
with subtest("re-enabling returns the same url and starts no second capture"):
|
|
again = machine.succeed(cli(f"stream enable {session_id}"), timeout=180).strip()
|
|
assert again == url, f"the phrase rotated: {again} != {url}"
|
|
assert len(stream_json("list")["streams"]) == 1
|
|
|
|
with subtest("a phrase that is not this session's is refused"):
|
|
# Of the generated SHAPE and not this session's phrase -- the two
|
|
# refusals are different exits, and a fixture that is merely
|
|
# malformed would take the shape path (exit 2) and never reach the
|
|
# broker. MEASURED: the first fixture here was
|
|
# `other-words-go-here-99999`, which fails on `go` being two
|
|
# letters -- the CLI refused it on shape and this subtest read
|
|
# exit 2 where it wanted 8.
|
|
status, out = machine.execute(
|
|
cli(f"stream enable {session_id} --phrase other-words-going-here-99999"),
|
|
timeout=120,
|
|
)
|
|
# Exit 8 (BUSY): rotating under somebody holding the old URL is the
|
|
# one outcome neither answer can be mistaken for.
|
|
assert status == 8, f"expected exit 8, got {status}: {out}"
|
|
assert stream_json(f"status {session_id}")["stream"]["url"] == url
|
|
|
|
with subtest("a malformed phrase is refused without echoing it"):
|
|
status, out = machine.execute(
|
|
cli(f"stream enable {session_id} --phrase hunter2"), timeout=120
|
|
)
|
|
assert status == 2, f"expected exit 2, got {status}: {out}"
|
|
assert "hunter2" not in out, "the refusal echoed the phrase back"
|
|
|
|
with subtest("disabling stops the capture and forgets the phrase"):
|
|
machine.succeed(cli(f"stream disable {session_id}"), timeout=180)
|
|
assert stream_json(f"status {session_id}")["stream"] is None
|
|
assert stream_json("list")["streams"] == []
|
|
status, _ = machine.execute(
|
|
as_user(f"{READ} {url}stream.mjpg --frames 1"), timeout=120
|
|
)
|
|
assert status != 0, "the old URL still serves after disable"
|
|
|
|
with subtest("disable then enable rotates the phrase"):
|
|
rotated = machine.succeed(cli(f"stream enable {session_id}"), timeout=180).strip()
|
|
assert rotated != url, "the phrase did not rotate"
|
|
url = rotated
|
|
|
|
with subtest("kill takes the stream down with the session"):
|
|
# The teardown *order* is unit-tested (a capture aimed at a removed
|
|
# output fails every time); what is asserted here is the outcome
|
|
# both halves have to agree on.
|
|
try:
|
|
machine.succeed(cli(f"kill {session_id}"), timeout=300)
|
|
assert stream_json("list")["streams"] == []
|
|
status, _ = machine.execute(
|
|
as_user(f"{READ} {url}stream.mjpg --frames 1"), timeout=120
|
|
)
|
|
assert status != 0, "the stream outlived its session"
|
|
# And the daemon is still up: a session ending is not its
|
|
# problem.
|
|
machine.succeed("systemctl is-active hhstream-live.service")
|
|
except Exception:
|
|
dump("killing a streaming session", session_id)
|
|
raise
|
|
|
|
with subtest("start --stream does it in one call"):
|
|
# The usual way an agent sets one up, and the reason `start` takes
|
|
# the flag at all: a stream needs the container output that
|
|
# starting creates.
|
|
try:
|
|
out = machine.succeed(cli("start foot --stream"), timeout=300)
|
|
lines = [line for line in out.strip().splitlines() if line]
|
|
streamed_id, streamed_url = lines[0], lines[1]
|
|
print(f"{streamed_id} -> {streamed_url}")
|
|
assert streamed_url.startswith(f"http://127.0.0.1:{PORT}/")
|
|
read = json.loads(
|
|
machine.succeed(
|
|
as_user(f"{READ} {streamed_url}stream.mjpg --frames 2"),
|
|
timeout=180,
|
|
).strip().splitlines()[0]
|
|
)
|
|
assert len(read["frames"]) == 2, read
|
|
machine.succeed(cli(f"kill {streamed_id}"), timeout=300)
|
|
except Exception:
|
|
dump("start --stream")
|
|
raise
|
|
'';
|
|
}
|
|
)
|