Compare commits

...
Author SHA1 Message Date
HermenixandClaude Opus 5 01460016e7 containers: a headless output and a persistent workspace per session
A session was a process group, so every application landed on whichever
workspace the compositor happened to show: one session's capture caught
another's windows, and a pointer aimed at one could land in the other.

This is the pure half of the fix. containers.py computes the argv lists
that create a session's container (headless output, numbered workspace
bound to it as default and persistent, an anchored window rule pinning
the session's class there, then the mode) and the one that tears it down,
plus the lowest-free allocator for the 1000+slot band and the
quarantine backstop for windows that match no container. It runs
nothing: no subprocess, no socket, no file, no clock, and no import of
ipc.py, so every decision that can be tested without a compositor is
asserted in a unit test.

Validation is a control, not style: these strings reach a compositor
that acts on them, and a rule body is comma-separated, so a session id
outside [a-z0-9-], a class outside [A-Za-z0-9._+-], a non-positive or
fractional size and a blank quarantine name all raise ValueError naming
the value. Releasing a workspace the allocator never handed out raises
rather than no-opping, because the quiet version hands the same
workspace to two sessions minutes later.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QS2JvrgmvXmimKFcqqDVoa
2026-10-02 00:02:40 +00:00
Hermenix 98374b8457 docs: the plan for workspaces and pointer input, and the frozen sessions contract
The plan is the researched design (one compositor, one output+workspace per
session, serialised output-bound pointer input under a lease). The contract is
the freeze point for a parallel build: file ownership per slice, the exact
Python seams, the wire format, and the names and paths nobody may invent.

Kept separate from docs/CONTRACT.md, which governs the frame socket and is
untouched by this work.
2026-10-01 23:58:09 +00:00
HermenixandClaude Opus 5 7a81f9f334 checks/vm-sessions: size the negative control's window for its own cleanup
`a session touched inside the window is not reaped` was racing itself. The
idle window restarts at the *last* touch, so everything the subtest does
after the loop -- the `status --json` round trip, `kill -0`, and the `kill`
that clears the session away -- has to finish inside one whole window. With
`--idle-timeout 6s` and a `systemd-run --wait` round trip costing several
seconds in this guest, it did not: the session was reaped 6.3 s after the
last touch, legitimately, and the cleanup `kill` arrived ~13.5 s after it
and returned 4 ("not in a usable state"). The daemon was right and the test
was wrong.

The window is now 45 s, which is the margin the cleanup gets. The assertion
is unchanged in meaning and stricter in degree: nine touches 15 s apart --
a third of a window -- keep the session alive for 135 s, three whole
windows of real time, so a `touch` that did not reset the deadline would
have let the reaper fire two windows before the loop ended. Nothing in the
cleanup stamps activity (`registry.touch` has exactly one caller,
`manager.touch`), so the margin is slack and not a hidden touch.

Two details keep it honest rather than lucky: the loop ends with a touch
instead of a trailing sleep, so the cleanup gets a whole window and not a
window less one gap; and it runs under `set -e`, so a `touch` that fails --
exit 4 against an already-reaped session -- fails the loop where it happens
instead of being swallowed by the sleep that used to end each iteration.

Verified both ways: the check passes (258 s; cleanup finished 4.1 s after
the last touch, against the 45 s window), and with `manager.touch`
temporarily not calling `registry.touch` it goes red here, on the touch
that follows the reap, with the diagnostics dump naming the failure mode.
That break is reverted.

The positive control (`the idle reaper fires for real`, 5 s) needs no such
margin and is unchanged: it waits for `reaped`, which is terminal and held
for the tombstone TTL, so time passing only confirms it, and it runs no
cleanup that needs a usable session.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8R9yNgHeHMnY4nDQwaLtu
2026-10-01 12:49:18 +00:00
Hermenix f51197afca module: one tmpfiles definition, merged
Adding the broker's state directory as a second assignment to
systemd.tmpfiles.settings does not evaluate at all (the option takes one
definition), which broke every check that enables sessions. Both reasons to
write a tmpfiles entry now live in one lib.mkMerge.
2026-10-01 12:30:04 +00:00
Hermenix fc5112aa6a module: let the broker's state be pointed off the system disk
The unit used systemd's StateDirectory=, which can only ever mean
/var/lib/<name> — on this kind of machine that is the disposable root. The
records the broker adopts and events.jsonl, the audit trail of every accepted
command, are precisely the things that should outlive a reinstall, so they are
now an option (`sessions.stateDir`) and the directory is declared with tmpfiles
so its owner and mode are still declarative.
2026-10-01 12:29:35 +00:00
Hermenix 30f456c07e integrate the session broker: one CLI, a module unit, and a VM check
M0 of the sessions design, wired into the flake and gated by a real check.

What is here now:

- `src/hhstreamd/` — the broker: session registry, lifecycle verbs, process-group
  teardown, app allowlist, audit log, idle reaper on CLOCK_BOOTTIME.
- `src/hhstream_session/` — the CLI handlers, registered into the one front door.
- `src/hhstream/cli.py` — `hhstream status|shot|run|record|session …`, so the
  frame verbs and the session namespace are one binary.
- `nix/module.nix` — `sessions.*` options and the `hhstream-sessions.service`
  unit: the operator's own account (never a service account, because launched
  applications should see that account's home), the compositor's group as a
  supplementary group (without it a launched GUI client cannot open the Wayland
  socket), the generated app allowlist, and `--compositor-runtime-dir` so
  applications are handed the compositor's session rather than the broker's.
- `checks/vm-sessions.nix` — 18 subtests on a real compositor in a VM.

Two bugs found by driving it, both fixed:

- BUG-01: launched applications could not reach the compositor at all — the
  broker spawned them with its own inherited environment, which names the login
  session. `hhstreamd/compositor.py` discovers the socket and builds the child
  environment; a directory with no usable socket is a startup refusal, not a
  warning.
- BUG-02: `kill` refused to signal a live application and said nothing was
  wrong. The identity gate hashed `/proc/<pid>/cmdline`, which every `execve`
  replaces — and on NixOS applications are reached through wrapper scripts that
  do exactly that. The gate now rests on `starttime` (+ boot id, + session id
  where it is immutable) and records command-line drift as evidence of nothing.

KillMode=process is set on the unit on purpose: the default control-group kill
would take the applications down with the broker, which would make adoption and
a broker restart pointless.
2026-10-01 12:23:16 +00:00
Hermenix 9dc9e0da9a examples/agent-eyes: use the setuid sg wrapper, not whatever PATH offers
There are two `sg` on this box: the setuid wrapper in /run/wrappers/bin and a
plain symlink to shadow's sg in the system profile. A login shell has the
wrapper first and works; a cron job's PATH reaches the plain one, which cannot
switch groups and dies with "setgroups: Operation not permitted" — which reads
as "chromium exited immediately" once a consumer discards its stderr. The
helper now looks for a setuid `sg` explicitly and says so if there is none.
2026-10-01 04:00:01 +00:00
Hermenix 09903c5676 examples: agent-eyes subcommand helper (status/shot/run/record)
One front door to the headless session, for reuse by any consumer: it
discovers whichever wayland-N socket exists, re-execs itself under
sg hhstream when the login predates the group membership, and encodes
recordings at the measured frame rate so clips play at true speed.
2026-10-01 03:18:31 +00:00
Hermenix 2b32153c8b examples: README — measured capture costs, v3 window rules, hyprctl IPC caveat 2026-09-30 19:43:32 +00:00
Hermenix 1664ccf710 examples: follow the stream's format; wall guard checks URL, page text, and end of recording 2026-09-30 19:40:39 +00:00
Hermenix f667654f01 examples: sandboxed chromium by default, with a --no-sandbox fallback 2026-09-30 19:35:03 +00:00
Hermenix 36541d22f4 examples: windowed browser by default, so the session's desktop is visible 2026-09-30 19:21:30 +00:00
Hermenix bfb333e9ff examples: document the demo; strip ffmpeg metadata so Blossom accepts the mp4 2026-09-30 19:15:33 +00:00
Hermenix 1ce39ba210 examples: agent-eyes-demo.py -- drive a browser on the headless screen and film it 2026-09-30 18:47:24 +00:00
Hermenix a77eaa96d3 devShell: add chromium and the hhstream client for the eyes demo 2026-09-30 18:44:58 +00:00
HermenixandClaude Opus 5 6510e315d4 module: make outputs monitor rules for real connectors, and own the seat flag
A machine running this module has a GPU. aquamarine's headless backend has
no allocator of its own -- `CHeadlessBackend::drmFD()` returns -1 and
`CBackend::start()` takes the allocator from the first backend with a drm fd
-- so the session always rides on the DRM backend and always has the host's
own connectors. `HEADLESS-1` is a name no connector will ever have, which
made the shipped default a monitor rule Hyprland silently ignored, and made
`vm-hyprland`'s assertion on that name unsatisfiable.

`outputs` now means "monitor rules applied to whatever outputs the machine
has". `name` is a regex against the real connector names and defaults to the
empty string -- Hyprland's own wildcard form, as in its shipped
`monitor=,preferred,auto,1` -- so `enable = true` alone gives a session at a
known size on any host. Nothing else about the option changed.

SEATD_VTBOUND=0 moves from checks/lib/gpu-session.nix into the compositor
unit, where it belongs: libseat finds no seatd socket and no logind session
for a system service, falls back to its embedded seatd, and that seat is
VT-bound by default -- so it opens /dev/tty0, which the service user cannot,
and every device open is refused. That is true on any host, not just in the
test. gpu-session.nix keeps only the two things that really are the test
machine's business: a render node and Mesa.

vm-hyprland now asserts over `hyprctl -j monitors` that at least one monitor
exists at the configured mode, with no monitor name anywhere in the check.
It also gains a wallpaper, because `--reject-blank` could not otherwise mean
anything: the session the module configures by default is a flat black
field, which grim captures as an all-zero 1280x720 RGBA PNG of exactly 3653
bytes -- `blank_png_size()` to the byte. Runs that passed that assertion were
reading Hyprland's startup notification overlay, which expires after about
fifteen seconds. With `misc:force_default_wallpaper` on, the frame is 85267
bytes with all 921600 pixels non-zero and is byte-identical at +15s, +30s and
+60s.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M5GEju6XhoRXDWvEiaAX2r
2026-09-28 21:51:24 +00:00
Hermenix 231fa317cb module: discover the Wayland socket instead of assuming wayland-0 2026-09-28 21:24:27 +00:00
52 changed files with 11839 additions and 108 deletions

No files matched your search

+2 -1
View File
@@ -42,8 +42,9 @@ at the bottom. The package builds; it does not yet run.
| `unit` | the framing, the capture backend, the server's client handling, the client — in-process, against the `nix/package.nix` build | no |
| `vm-basic` | the NixOS module brings up both units and publishes a contract-conformant socket, with a **fake** capture backend | yes |
| `vm-hyprland` | the same, capturing the **real** headless compositor through `grim` | yes |
| `vm-sessions` | the **session broker**: a real GUI application launched into the compositor as a tracked session, seen by the compositor's own IPC, killed as a tree, reaped on its idle deadline (and *not* reaped when touched), and adopted back after the broker restarts | yes |
All three at once:
All of them at once:
```bash
nix flake check -L
+4 -1
View File
@@ -19,7 +19,9 @@ Two parts:
imports = [ inputs.hhstream.nixosModules.default ];
services.hyprland-headless-stream = {
enable = true;
outputs = [ { name = "HEADLESS-1"; width = 1280; height = 720; refresh = 30; } ];
# Optional: the default is one 1280x720@30 rule with an empty name, which
# is Hyprland's wildcard and applies to whatever outputs the GPU has.
outputs = [ { width = 1920; height = 1080; refresh = 60; } ];
};
}
```
@@ -38,5 +40,6 @@ connects, and keeps running when one disconnects.
| `docs/CONTRACT.md` | the wire protocol and the module surface — the interface other people build against |
| `docs/ARCHITECTURE.md` | how the pieces fit: compositor, capture, server, client |
| `README.dev.md` | how to build, test and hack on it |
| `examples/README.md` | the demo that films the screen: `agent-eyes-demo.py` |
Status: work in progress — the module and the test suite are being built.
+5
View File
@@ -3,6 +3,7 @@
# unit -- pytest over tests/unit/, against the nix/package.nix build
# vm-basic -- nixosTest with a fake capture backend: module + wire contract
# vm-hyprland -- nixosTest with the real compositor and grim
# vm-sessions -- nixosTest with the real compositor and the session broker
#
# Everything the guests need lives in checks/lib/test-tools.nix.
{
@@ -25,4 +26,8 @@ in
vm-hyprland = pkgs.callPackage ./vm-hyprland.nix {
inherit hhstream hhsModule testTools;
};
vm-sessions = pkgs.callPackage ./vm-sessions.nix {
inherit hhstream hhsModule testTools;
};
}
+5 -36
View File
@@ -11,14 +11,15 @@
#
# and systemd restarts it until the test's timeout expires. `CBackend::create()`
# is aquamarine's, and reading aquamarine 0.15.1 rather than guessing at it, it
# needs three separate things that a bare test machine does not provide. Each is
# one of the blocks below, and each was confirmed in the guest by removing it
# again and watching the compositor die on the next line down.
# needs three separate things. Two of them are hardware the bare test machine
# does not have and are the blocks below; the third -- a seat that is not bound
# to a VT -- is true of this session on any host, so it lives in nix/module.nix
# as `SEATD_VTBOUND=0` on the compositor unit rather than here.
#
# A note for whoever hits this next: none of it is visible in the journal until
# you also set `debug:disable_logs = false`, because Hyprland mutes aquamarine
# by default and the abort is all that survives. That is how the three causes
# below were separated.
# were separated.
{ ... }:
{
# 1. A DRM render node.
@@ -49,36 +50,4 @@
# ERR from aquamarine ]: Couldn't open a GBM device at fd 21
# CRIT from aquamarine ]: Cannot open backend: no allocator available
hardware.graphics.enable = true;
# 3. A seat that is not bound to a VT.
#
# This one is not really the test machine's business, and it is flagged rather
# than fixed: it belongs in nix/module.nix, which this slice may not edit.
#
# Only the DRM backend ever yields a buffer allocator --
# `CHeadlessBackend::drmFD()` returns -1 unconditionally (aquamarine
# src/backend/Headless.cpp), and CBackend::start() takes the allocator from the
# first implementation with a drm fd. So "headless" here does not mean "no
# GPU": the headless backend still rides on the DRM backend's allocator, and if
# the DRM backend fails there is none, whatever /dev/dri holds.
#
# The DRM backend opens /dev/dri/card0 through libseat. For this unit libseat
# finds no /run/seatd.sock and no logind session (it is a system service, not a
# login), so it falls back to its embedded seatd -- whose seat is VT-bound by
# default. It then tries to open /dev/tty0, which the unprivileged `hhstream`
# user cannot, so the client never becomes active and every device open is
# refused:
#
# ERR ]: [libseat] [common/terminal.c:162] Could not open target tty: Permission denied
# ERR ]: [libseat] [seatd/seat.c:557] Could not open VT for client
# ERR ]: libseat: Couldn't open device at /dev/dri/card0
# ERR ]: drm: Skipping device /sys/devices/.../drm/card0, not a KMS device
# ERR ]: drm: Found no gpus to use, cannot continue
# ERR ]: DRM Backend failed
#
# SEATD_VTBOUND=0 (seatd/server.c) asks that embedded seatd for a seat with no
# VT at all, which is the truth about this session: there is no terminal to
# switch away from. A headless session run as a system user will need this on
# any machine, not just in the test.
systemd.services.hyprland-headless.environment.SEATD_VTBOUND = "0";
}
+6 -2
View File
@@ -24,7 +24,9 @@ testers.runNixOSTest {
nodes.machine = {
# gpu-session.nix is what makes the compositor able to start at all; see
# that file for the three things a bare test machine is missing.
# that file for the two pieces of hardware a bare test machine is missing
# (the third thing it used to carry, SEATD_VTBOUND=0, is in nix/module.nix
# now -- it is a property of the session, not of the test machine).
imports = [ hhsModule ./lib/gpu-session.nix ];
services.hyprland-headless-stream = {
@@ -35,9 +37,11 @@ testers.runNixOSTest {
inherit socketPath;
frameRate = fps;
format = "png";
# The fake capture fixes the frame size, so the monitor rule does not
# matter here -- but it must not be a name no connector has either, or
# this file would keep advertising a shape that does nothing.
outputs = [
{
name = "HEADLESS-1";
inherit width height;
refresh = fps;
}
+100 -25
View File
@@ -3,9 +3,13 @@
# compositor through grim.
#
# This is the check expected to be flaky, so it is deliberately impatient: every
# wait has a bounded timeout, nothing is retried in a loop, and any failure dumps
# the compositor's log and `hyprctl monitors` before it propagates. A flake you
# can read is worth more than a flake that eventually goes green.
# wait has a bounded timeout, no assertion is retried, and any failure dumps the
# compositor's log and `hyprctl monitors` before it propagates. A flake you can
# read is worth more than a flake that eventually goes green.
#
# There is exactly one waiting loop, and it waits for the compositor to finish
# starting (see "the session settles ..." below) rather than for an assertion to
# pass. Every assertion after it runs once.
{
testers,
hhstream,
@@ -25,7 +29,9 @@ testers.runNixOSTest {
nodes.machine = {
# gpu-session.nix is what makes the compositor able to start at all; see
# that file for the three things a bare test machine is missing.
# that file for the two pieces of hardware a bare test machine is missing
# (the third thing it used to carry, SEATD_VTBOUND=0, is in nix/module.nix
# now -- it is a property of the session, not of the test machine).
imports = [ hhsModule ./lib/gpu-session.nix ];
services.hyprland-headless-stream = {
@@ -35,14 +41,42 @@ testers.runNixOSTest {
inherit socketPath;
frameRate = fps;
format = "png";
# No `name`, so the module's wildcard rule applies to whatever connector
# the virtio GPU gives this guest. The check must not know that name: on
# QEMU it is `Virtual-1`, on a real host it is whatever the DRM backend
# reports, and a rule naming a connector that does not exist is silently
# ignored rather than an error.
outputs = [
{
name = "HEADLESS-1";
inherit width height;
refresh = fps;
}
];
# No captureCommand: the module's default (grim) is the point of this check.
# Put a picture on the screen, so that `--reject-blank` below is an
# assertion and not a formality.
#
# The session the module configures by default is a flat field: no logo,
# no splash, no wallpaper, `background_color = 0x000000`. Measured in this
# guest, grim captures that as 1280x720 RGBA in which every byte is zero,
# encoding to *exactly* 3653 bytes -- which is `blank_png_size()` in
# tests/assert_stream.py to the byte. A flat field is by construction
# indistinguishable from a zero-filled buffer, so against the default
# config `--reject-blank` cannot pass, and any run in which it appears to
# is reading something transient (the startup notification overlay lives
# for about fifteen seconds and is 9739 bytes).
#
# Turning Hyprland's own wallpaper back on costs no new package and gives
# the compositor real content to composite: measured 85267 bytes, all
# 921600 pixels non-zero, byte-identical at +15s, +30s and +60s. That is
# what makes "the frame is not blank" mean "grim captured composited
# pixels".
extraConfig = ''
misc:force_default_wallpaper = 1
misc:disable_hyprland_logo = false
misc:disable_splash_rendering = false
'';
};
# hyprctl and grim are here for the diagnostics dump and for the manual
@@ -67,9 +101,21 @@ testers.runNixOSTest {
};
testScript = ''
import json
SOCKET = "${socketPath}"
WIDTH, HEIGHT = ${toString width}, ${toString height}
# The module pins XDG_RUNTIME_DIR; the instance signature under it is the
# only part that is not known ahead of time. Root may connect to the
# compositor's IPC socket even though it belongs to `hhstream`.
HYPRCTL_MONITORS = (
"set -eu; "
"sig=$(ls -1 /run/hyprland-headless-stream/hypr | head -n1); "
"XDG_RUNTIME_DIR=/run/hyprland-headless-stream"
" HYPRLAND_INSTANCE_SIGNATURE=$sig hyprctl -j monitors"
)
def dump(label):
print(f"########## diagnostics: {label} ##########")
@@ -87,17 +133,27 @@ testers.runNixOSTest {
dump("hyprland-headless.service did not start")
raise
with subtest("the compositor reports the configured headless output"):
with subtest("the compositor reports a monitor at the configured mode"):
try:
# Proves the compositor is really running a 1280x720 HEADLESS-1
# before we blame the capture for a wrong-sized frame.
out = machine.succeed("hhs-hyprland-diagnostics", timeout=120)
print(out)
assert "HEADLESS-1" in out, (
"hyprctl never reported a HEADLESS-1 monitor; see the dump above"
# Proves the compositor is really driving a WIDTHxHEIGHT output
# before we blame the capture for a wrong-sized frame. Asked over
# hyprctl's JSON so the assertion is on the mode, never on a
# monitor name: the name belongs to the host's connector.
monitors = json.loads(machine.succeed(HYPRCTL_MONITORS, timeout=120))
print(json.dumps(monitors, indent=2, sort_keys=True))
assert monitors, "hyprctl reported no monitors at all"
print(
"monitors: "
+ ", ".join(
f"{m['name']} {m['width']}x{m['height']}@{m.get('refreshRate')}"
for m in monitors
)
)
assert f"{WIDTH}x{HEIGHT}" in out, (
f"hyprctl never reported a {WIDTH}x{HEIGHT} mode; see the dump above"
assert any(
m["width"] == WIDTH and m["height"] == HEIGHT for m in monitors
), (
f"no monitor is at the configured {WIDTH}x{HEIGHT}; the wildcard "
"monitor rule did not take"
)
except Exception:
dump("compositor output not as configured")
@@ -112,20 +168,39 @@ testers.runNixOSTest {
raise
with subtest("grim can capture the session at all"):
# Separates "grim cannot see the compositor" from "the server is wrong",
# which are the two failures that look identical from the socket.
# Diagnostic, not an assertion: it separates "grim cannot see the
# compositor" from "the server is wrong", which are the two failures
# that look identical from the socket. The session's XDG_RUNTIME_DIR is
# the module's private one, not /run/user/<uid>, and the socket name is
# whatever the compositor bound -- so discover both, as the unit does.
# The grim path is the store one: a transient systemd unit gets a
# minimal PATH, not the guest's environment.systemPackages.
status, out = machine.execute(
"systemd-run --pipe --wait --uid=hhstream"
" --setenv=XDG_RUNTIME_DIR=/run/hyprland-headless-stream"
" --setenv=WAYLAND_DISPLAY=$(cd /run/hyprland-headless-stream"
" && ls -1 wayland-* | grep -v '\.lock$' | head -n1)"
" sh -c '${grim}/bin/grim -t png - > /tmp/direct.png' 2>&1",
timeout=120,
)
print(f"direct grim exit {status}: {out}")
print(machine.execute("stat -c 'direct.png: %s bytes' /tmp/direct.png")[1])
with subtest("the session settles into a frame with a picture in it"):
# A bounded wait for the *end of startup*, in the same spirit as
# wait_for_unit -- not a retry until something goes green. The Wayland
# socket is bound before the wallpaper texture has been uploaded and
# composited, so the very first frames off the socket can still be the
# empty field. Everything below this point is a hard assertion, made
# once, on a session that has finished coming up.
try:
print(
machine.succeed(
"systemd-run --pipe --wait --uid=hhstream"
" --setenv=XDG_RUNTIME_DIR=/run/user/$(id -u hhstream)"
" grim -t png /tmp/direct.png 2>&1 || true",
timeout=120,
)
machine.wait_until_succeeds(
f"hhs-assert-stream --socket {SOCKET} --frames 1 --timeout 30"
" --require-png --reject-blank",
timeout=120,
)
print(machine.execute("stat -c '%s bytes' /tmp/direct.png")[1])
except Exception:
dump("direct grim capture failed")
dump("the session never rendered anything but a blank field")
raise
with subtest("a frame off the socket is a real PNG of the configured size"):
+684
View File
@@ -0,0 +1,684 @@
# checks.<system>.vm-sessions -- the session broker, on a real machine, against
# the real compositor.
#
# Same machine as `vm-hyprland` (compositor + gpu-session.nix), plus
# `sessions.enable = true` and one allowlisted victim application: `foot`, a
# real Wayland client with no GPU demands. Nothing here reads the frame socket:
# the source of truth for "the application is on the screen" is **the compositor
# itself**, asked over its own IPC, because the frame publisher could be happy
# about a screen with nothing on it.
#
# Habits copied from vm-hyprland, deliberately: every wait is bounded, each
# subtest makes its assertions once, and a failure dumps the compositor's
# diagnostics *and* the broker's journal, registry and the application's own
# captured output before it propagates.
#
# Every wait in the script is one of five kinds, and each waits for an *event
# that must happen* rather than for an assertion to come good:
#
# 1. the compositor's unit and the broker's unit coming up (wait_for_unit);
# 2. the compositor's and the broker's sockets appearing (wait_for_file);
# 3. the compositor mapping the client foot just connected with;
# 4. a process finishing dying after a kill;
# 5. the idle reaper firing, and the restarted broker answering again.
#
# The two real-time sleeps are the idle-reaper subtests, and they are the point
# of those subtests: no clock is injected anywhere, because an injected clock
# would prove the arithmetic (the unit tests already do) and not the daemon.
# Those two sleeps are sized against the cost of a `systemd-run` round trip in
# this guest and not against the arithmetic, because an idle window has to
# outlast the subtest's own cleanup as well as its loop -- see the negative
# control ("a session touched inside the window is not reaped") for the
# measurement that fixes the numbers.
#
# ─────────────────────────────────────────────────────────────────────────────
# KNOWN DEFECTS this check is pointed at, as of 2026-10-01.
#
# Nothing below is worked around here. No WAYLAND_DISPLAY is injected into the
# unit, no ExecStart is overridden, and no assertion is softened to let a
# known-broken path through: a check that accommodated the daemon would go
# green while the capability it exists to prove stayed broken.
#
# * BUG-01 and the unknown `--compositor-runtime-dir` flag -- **fixed**. This
# check found both on its first run. The module passed the daemon a flag its
# parser did not define, so hhstreamd exited 2/INVALIDARGUMENT before it
# ever listened (restart counter past 40 by the time the test gave up), and
# even past that, a launched application was handed the daemon's own private
# per-session XDG_RUNTIME_DIR with no WAYLAND_DISPLAY and died inside a
# second. `hhstreamd/compositor.py` and `--compositor-runtime-dir` are the
# fix, and this check is what holds them honest.
#
# Worth keeping from that run: `wait_for_unit` *passed* against the crash
# loop, because the unit is `Type=simple` and systemd calls it active as
# soon as it has forked, and each attempt was slow enough (~3 s) that the
# default start limit never tripped. That is why the unit subtest asserts on
# the socket and on `NRestarts` rather than trusting `wait_for_unit` alone.
#
# * BUG-02 -- **open**, being fixed separately. `procmon.verify` reports *any*
# cmdline change as `pid_reuse_detected`, and `SessionManager.teardown`
# treats that name as "do not signal, and do not sweep either", so a session
# whose application exec-replaced its own argv -- a wrapper script, most of
# them -- is tombstoned `killed` while the process keeps running. The kill
# and reaper subtests below assert that the pid, its named children, its
# process group and its session id are all gone, and they stay that way: if
# BUG-02 reaches this check's victim, the right outcome is a red test, not a
# softer assertion. (The victim here is `${foot}/bin/foot`, a plain ELF in
# nixpkgs rather than a wrapper, so the trigger may not fire through it --
# which is a fact about foot's packaging and not a reason to trust the path.)
#
# * The `/proc` session-id sweep cannot reach a PTY child -- **found by this
# check, open**. DESIGN.md §7 #7 says a child that called `setsid()` once is
# still caught, because it keeps the session id, and only a double
# `setsid()` escapes. Measured in this guest, that is not so: foot is
# pid/pgid/sid 1042 and the shell it runs is pid/pgid/sid **1056**. A
# terminal emulator must `setsid()` to give its child a controlling
# terminal, and `setsid()` changes the session id -- the very key the sweep
# searches on. So one `setsid()` is enough to leave both the process group
# `killpg` signals and the session the sweep walks, and any app that opens a
# PTY (a terminal, `script`, `tmux`, `ssh`) leaks its child by construction.
#
# In practice foot's shell still dies, because closing the PTY master sends
# it SIGHUP -- but that is the kernel's tty semantics, not this daemon's
# containment, and a child that ignores SIGHUP or holds no tty would
# survive. That is why the subtest below walks the **process tree** and
# names the pids it expects to be gone, rather than trusting an empty
# process group to mean "nothing survived".
#
# * A broker restart kills every session it was managing -- **found by this
# check, open, and it is a one-line module fix**. `systemctl restart
# hhstream-sessions` tombstones every live session as `exited`
# (`exit_detected_by: "proc_check"`, `last_error: "gone"`), and the new
# daemon logs `adopted 0, tombstoned 1`. The application is not gone because
# the daemon let go of it; it is gone because the *unit* killed it.
# `systemd.services.hhstream-sessions` sets no `KillMode=`, so it gets the
# default `control-group`: on stop systemd signals every process still in
# the unit's cgroup, and a launched app is a child of the daemon and so
# lives in that cgroup.
#
# This defeats DESIGN.md §2.4 and the whole adopt-on-startup path.
# `hhstreamd/launcher.py` explains at length why it uses a plain `Popen`
# rather than asyncio's subprocess transport -- precisely so that a daemon
# restart is not destructive -- and notes that "no unit file setting can fix
# it because the kill comes from inside this process". The converse is what
# bites here: the daemon's own care is undone from outside by the unit's
# default kill mode. `KillMode = "process"` on that unit is the fix.
# systemd.kill(5) warns that `process` lets children escape the service
# manager's lifecycle, which is exactly the trade this design already made:
# the registry, `killpg` and the `/proc` sweep are the lifecycle here.
#
# The restart subtests below are left asserting survival, and therefore red,
# until that lands.
# ─────────────────────────────────────────────────────────────────────────────
{
testers,
hhstream,
testTools,
hhsModule,
hyprland,
foot,
jq,
procps,
coreutils,
dejavu_fonts,
width ? 1280,
height ? 720,
fps ? 30,
# The account the broker and every application it launches run as. The module
# refuses to create it (launched applications run as a real account), so the
# guest below declares it.
sessionUser ? "hermes",
# Long enough that nothing in this test is reaped by surprise, short enough
# that the number is visibly this test's and not the module's 3 h default.
idleTimeout ? 600,
controlSocket ? "/run/hhstream/control.sock",
}:
testers.runNixOSTest {
name = "hhstream-vm-sessions";
# Boot plus compositor startup plus ~2.5 min of real idle-clock waiting (five
# seconds for the reaper to fire, three 45 s windows for the touched session
# that must outlive them). Capped so a hang is a failure rather than a wedged
# builder.
globalTimeout = 1800;
nodes.machine = {
imports = [ hhsModule ./lib/gpu-session.nix ];
services.hyprland-headless-stream = {
enable = true;
package = hhstream;
hyprlandPackage = hyprland;
# No `name`: the wildcard rule applies to whatever connector the virtio
# GPU gives this guest (see vm-hyprland.nix for why the check must not
# know that name).
outputs = [
{
inherit width height;
refresh = fps;
}
];
sessions = {
enable = true;
user = sessionUser;
inherit idleTimeout;
apps.foot = {
exec = "${foot}/bin/foot";
# The daemon refuses unlisted arguments, and this check never passes
# any: `foot` with no argv opens a window running the account's shell,
# which is all the victim has to do.
argsAllow = [ ];
};
};
};
# The module deliberately does not create this user. `video` and `render`
# are what any client of this session needs to reach the GPU; the broker's
# unit adds the compositor's own group on top (SupplementaryGroups=), which
# is what lets a launched client open the Wayland socket.
users.users.${sessionUser} = {
isNormalUser = true;
extraGroups = [
"video"
"render"
];
};
# foot resolves "monospace" through fontconfig and refuses to start with no
# usable font. A NixOS test machine has `fonts.enableDefaultPackages =
# false`, so without this the victim application fails for a reason that has
# nothing to do with the broker. One font, not the default set.
fonts.packages = [ dejavu_fonts ];
environment.systemPackages = [
testTools # hhs-hyprland-diagnostics
hhstream # the CLI under test
hyprland # hyprctl: the compositor is the source of truth here
foot
jq # the in-guest predicates the bounded waits poll
procps # ps, for "the process group and the session id are gone"
];
virtualisation = {
memorySize = 2048;
cores = 2;
diskSize = 4096;
graphics = false;
};
};
testScript = ''
import json
USER = "${sessionUser}"
CONTROL = "${controlSocket}"
CONTROL_DIR = "/run/hhstream"
IDLE_TIMEOUT = ${toString idleTimeout}.0
COMPOSITOR_RUNTIME = "/run/hyprland-headless-stream"
# The module pins the compositor's XDG_RUNTIME_DIR; the instance signature
# under it is the only part not known ahead of time. Root may connect to
# the IPC socket even though it belongs to `hhstream`. Same shape as
# vm-hyprland.nix, which asks this for monitors.
HYPRCTL = (
"set -eu; "
f"sig=$(ls -1 {COMPOSITOR_RUNTIME}/hypr | head -n1); "
f"XDG_RUNTIME_DIR={COMPOSITOR_RUNTIME}"
" HYPRLAND_INSTANCE_SIGNATURE=$sig hyprctl -j"
)
HHSTREAM = "${hhstream}/bin/hhstream"
TRUE = "${coreutils}/bin/true"
FALSE = "${coreutils}/bin/false"
ID = "${coreutils}/bin/id"
def as_user(command):
"""`command` (absolute path plus argv, never a shell) run as USER.
A transient unit rather than `su`, because that is what the existing
checks already do for the compositor's account, and because it gives a
clean argv with no login shell in the middle. `--pipe --wait`
propagates the command's own exit status, which the first subtest
proves before anything depends on it.
"""
return (
"systemd-run --quiet --collect --pipe --wait"
f" --uid={USER} --setenv=HOME=/home/{USER} -- {command}"
)
def cli(args):
"""`hhstream session …` as USER, against the control socket."""
return as_user(f"{HHSTREAM} session --socket {CONTROL} {args}")
def gone(pid, pgid, sid, children=()):
"""A shell predicate: this pid, each pid in `children`, the process
group and the session id are all absent from /proc.
`children` is named explicitly rather than left to the process-group
check because "kill really kills, including children" is the assertion,
and a predicate that only looked at pgid/sid would also pass against a
teardown that never signalled anything if the group happened to be
empty. Built by concatenation so awk's braces do not have to be escaped
through an f-string.
"""
awk = "awk '$2 == " + str(pgid) + " || $3 == " + str(sid) + "'"
parts = [
"! kill -0 " + str(victim) + " 2>/dev/null"
for victim in (pid, *children)
]
parts.append('[ -z "$(ps -eo pid=,pgid=,sid= | ' + awk + ')" ]')
return " && ".join(parts)
def dump(label, session_id=None):
print(f"########## diagnostics: {label} ##########")
status, out = machine.execute("hhs-hyprland-diagnostics", timeout=300)
print(out)
print(f"----- hhs-hyprland-diagnostics exit {status} -----")
commands = [
"systemctl --no-pager --full status hhstream-sessions.service",
"journalctl --no-pager --boot --lines=200 --unit hhstream-sessions.service",
"systemctl show -p ExecStart --value hhstream-sessions.service",
f"ls -la {CONTROL_DIR} {CONTROL_DIR}/sessions /var/lib/hhstream"
" /var/lib/hhstream/sessions /var/lib/hhstream/logs",
"cat /var/lib/hhstream/sessions/*.json",
"ps -eo pid,ppid,pgid,sid,user,stat,comm,args --sort=pid",
cli("list --all"),
]
if session_id:
commands += [
cli(f"status {session_id}"),
# The application's own captured output: the only place a
# failed launch explains itself.
cli(f"logs {session_id} --lines 200"),
]
for command in commands:
status, out = machine.execute(f"{command} 2>&1", timeout=180)
print(f"----- {command} (exit {status}) -----")
print(out)
status, out = machine.execute(f"{HYPRCTL} clients 2>&1", timeout=180)
print(f"----- hyprctl clients (exit {status}) -----")
print(out)
print("########## end diagnostics ##########")
start_all()
with subtest("the as-user wrapper propagates exit status"):
# Everything below reads `machine.succeed`/`machine.fail` through
# `as_user`. If the wrapper swallowed the exit status, every one of
# those assertions would be vacuous, so prove it here first -- the same
# reason checks/lib/test-tools.nix self-tests its helpers at build time.
machine.succeed(as_user(TRUE), timeout=120)
machine.fail(as_user(FALSE), timeout=120)
print(machine.succeed(as_user(ID), timeout=120))
with subtest("the compositor comes up"):
# Not the thing under test, but every assertion below needs a session
# for the applications to be launched into.
try:
machine.wait_for_unit("hyprland-headless.service", timeout=300)
machine.wait_for_file(f"{COMPOSITOR_RUNTIME}/hypr", timeout=120)
except Exception:
dump("the compositor never started")
raise
with subtest("the broker's unit comes up and listens"):
# `wait_for_unit` on its own is not enough here, and this is measured
# rather than assumed: the unit is `Type=simple` with no readiness
# protocol, so systemd calls it active the instant it has forked. A
# daemon that exits two milliseconds later and is restarted every
# second therefore passes `wait_for_unit` -- which is exactly what the
# ExecStart below does today. The honest test of "it came up" is the
# socket it exists to listen on, plus a restart count of zero.
try:
machine.wait_for_unit("hhstream-sessions.service", timeout=180)
machine.wait_for_file(CONTROL, timeout=120)
machine.succeed("systemctl is-active hhstream-sessions.service")
restarts = machine.succeed(
"systemctl show -p NRestarts --value hhstream-sessions.service"
).strip()
assert restarts == "0", (
f"the broker has restarted {restarts} time(s) already: it is "
"crash-looping, not running"
)
except Exception:
dump("hhstream-sessions.service did not come up and listen")
raise
with subtest("the control socket has the expected owner and mode"):
try:
print(machine.succeed(f"stat -c 'owner=%U group=%G mode=%a type=%F' {CONTROL}"))
print(machine.succeed(f"stat -c 'owner=%U group=%G mode=%a' {CONTROL_DIR}"))
machine.succeed(f'[ "$(stat -c %F {CONTROL})" = socket ]')
machine.succeed(f'[ "$(stat -c %U {CONTROL})" = {USER} ]')
# The unit sets no Group=, so the socket's group is the account's
# own primary group. Asked of the guest rather than hardcoded: the
# module's contract is "the account that drives the CLI owns it",
# not "the group is called users".
machine.succeed(f'[ "$(stat -c %G {CONTROL})" = "$(id -gn {USER})" ]')
machine.succeed(f'[ "$(stat -c %a {CONTROL})" = 660 ]')
machine.succeed(f'[ "$(stat -c %a {CONTROL_DIR})" = 750 ]')
machine.succeed(f'[ "$(stat -c %U {CONTROL_DIR})" = {USER} ]')
except Exception:
dump("the control socket is missing or not as specified")
raise
with subtest("start prints a bare session id on stdout"):
# The contract: stdout is the id and nothing else, so $(...) is usable.
# Human detail goes to stderr, which the driver sends to the test log.
try:
out = machine.succeed(cli("start foot"), timeout=180)
assert out.endswith("\n"), f"start's stdout is not one line: {out!r}"
session_id = out.strip()
assert session_id.startswith("foot-"), (
f"stdout is not a session id for the app that was started: {out!r}"
)
assert len(session_id.split()) == 1, (
f"stdout carries more than the bare id: {out!r}"
)
print(f"session id: {session_id}")
except Exception:
dump("start did not print a session id")
raise
with subtest("the session reports ready, with a live pid"):
try:
reply = json.loads(machine.succeed(cli(f"status --json {session_id}"), timeout=120))
session = reply["session"]
print(json.dumps(session, indent=2, sort_keys=True))
assert session["state"] == "ready", f"state is {session['state']!r}, not ready"
pid, pgid, sid = session["pid"], session["pgid"], session["sid"]
assert isinstance(pid, int) and pid > 1, f"implausible pid {pid!r}"
# start_new_session(): the kill handle is sid == pgid == pid.
assert pgid == pid and sid == pid, f"pid {pid}, pgid {pgid}, sid {sid} disagree"
assert session["idle_timeout_s"] == IDLE_TIMEOUT, (
f"idle timeout is {session['idle_timeout_s']!r}, not the module's "
f"sessions.idleTimeout ({IDLE_TIMEOUT})"
)
machine.succeed(f"kill -0 {pid}")
except Exception:
dump("the session did not report ready", locals().get("session_id"))
raise
with subtest("the compositor reports a client whose pid is the session's"):
# THE assertion of this check: the compositor, over its own IPC, is the
# only thing that can say an application is really on this screen. The
# frame publisher is not consulted, and neither class nor title is
# asserted on -- only the pid, which is the one fact that ties the
# client on the screen to the session the broker handed back.
#
# One bounded wait, for the compositor to map a client that has only
# just connected: `ready` means "the process is alive", never "a window
# exists". Everything after it is a single assertion on a session that
# has finished coming up.
try:
machine.wait_until_succeeds(
f"{HYPRCTL} clients | jq -e --argjson pid {pid} 'any(.pid == $pid)' >/dev/null",
timeout=120,
)
clients = json.loads(machine.succeed(f"{HYPRCTL} clients", timeout=120))
print(f"compositor clients by pid: {[c.get('pid') for c in clients]}")
assert [c.get("pid") for c in clients].count(pid) == 1, (
f"the compositor does not report exactly one client with pid {pid}"
)
except Exception:
dump("the compositor never saw the launched application", session_id)
raise
with subtest("list reports the session with the right pid and state"):
try:
table = machine.succeed(cli("list"), timeout=120)
print(table)
rows = [line.split() for line in table.splitlines()[1:] if line.strip()]
mine = [row for row in rows if row[0] == session_id]
assert len(mine) == 1, f"list does not show {session_id} exactly once"
app, state, listed_pid = mine[0][1], mine[0][2], mine[0][3]
assert app == "foot", f"list calls the app {app!r}"
assert state == "ready", f"list calls the state {state!r}"
assert listed_pid == str(pid), f"list reports pid {listed_pid}, not {pid}"
except Exception:
dump("list did not report the session", session_id)
raise
with subtest("status --json reports the same pid and state"):
try:
reply = json.loads(machine.succeed(cli(f"status --json {session_id}"), timeout=120))
assert reply["ok"] is True, reply
assert reply["id"] == session_id, reply
assert reply["state"] == "ready", reply
assert reply["session"]["pid"] == pid, reply["session"]
assert reply["session"]["app"] == "foot", reply["session"]
assert reply["session"]["live"] is True, reply["session"]
assert reply["session"]["ready_signal"] == "spawned", reply["session"]
except Exception:
dump("status --json did not report the session", session_id)
raise
with subtest("the application really has a child of its own"):
# Captured before the kill, so "including children" is an assertion
# about named pids afterwards and not an inference from an empty
# process group. The window being mapped (above) means the child
# already exists, so this needs no wait.
#
# The walk is over **ppid**, not over the session id, and that is the
# whole point. Measured in this guest: foot is pid/pgid/sid 1042 and
# the shell it runs is pid/pgid/sid 1056 -- a terminal emulator must
# call setsid() to get a controlling terminal for its child, so that
# child leaves both the process group the daemon signals *and* the
# session id the daemon sweeps. So a pgid/sid-only check would pass
# with the child still running -- it is absent from both keys whether
# it is alive or dead -- and capturing "children" by session id finds
# nothing to name. The process tree is the only honest source.
try:
table = machine.succeed("ps -eo pid=,ppid=", timeout=120)
by_parent: dict[int, list[int]] = {}
for line in table.splitlines():
bits = line.split()
if len(bits) == 2:
by_parent.setdefault(int(bits[1]), []).append(int(bits[0]))
children: list[int] = []
frontier = [pid]
while frontier:
for child in by_parent.get(frontier.pop(), []):
if child not in children:
children.append(child)
frontier.append(child)
print(f"descendants of {pid}: {children}")
assert children, (
f"{pid} has no descendant at all: the victim never started its "
"child, so this check cannot say anything about killing children"
)
print(
machine.succeed(
"ps -o pid,ppid,pgid,sid,comm "
+ " ".join(f"-p {p}" for p in (pid, *children)),
timeout=120,
)
)
except Exception:
dump("the launched application never had a child", session_id)
raise
with subtest("kill ends the session"):
try:
reply = json.loads(machine.succeed(cli(f"kill --json {session_id}"), timeout=180))
print(json.dumps(reply, indent=2, sort_keys=True))
assert reply["ok"] is True, reply
assert reply["session"]["state"] == "killed", reply["session"]
# Not asserted: `stragglers`. It counts what the /proc sweep had to
# SIGKILL after the process group died, which is the sweep working
# rather than the kill failing. The next subtest is the assertion.
print(f"stragglers swept by session id: {reply['session'].get('stragglers')}")
except Exception:
dump("kill did not end the session", session_id)
raise
with subtest("the pid, its children, its process group and its session id are gone"):
try:
# Bounded, and only for the kernel to finish a teardown the daemon
# has already performed: `kill` returns after the process group was
# signalled, the grace spent and the session id swept. It is not a
# retry of the assertion -- if anything is still there after this,
# something escaped.
machine.wait_until_succeeds(gone(pid, pgid, sid, children), timeout=60)
print(machine.succeed("ps -eo pid,pgid,sid,stat,comm --sort=pid"))
except Exception:
dump("something survived the kill", session_id)
raise
with subtest("the compositor no longer lists the client"):
try:
machine.wait_until_succeeds(
f"{HYPRCTL} clients | jq -e --argjson pid {pid} 'all(.pid != $pid)' >/dev/null",
timeout=60,
)
clients = json.loads(machine.succeed(f"{HYPRCTL} clients", timeout=120))
print(f"compositor clients by pid: {[c.get('pid') for c in clients]}")
assert pid not in [c.get("pid") for c in clients]
except Exception:
dump("the compositor still lists the killed client", session_id)
raise
with subtest("the idle reaper fires for real"):
# No injected clock anywhere: the daemon's own CLOCK_BOOTTIME deadline,
# a five-second timeout, and real time passing in the guest.
#
# `start --json` rather than start-then-status, so the pid is captured
# in the same call that created the session: with a five-second window
# a second round trip could land after the reaper had already fired.
try:
reaped = json.loads(machine.succeed(cli("start --json --idle-timeout 5s foot"), timeout=180))
reaped_id, reaped_pid = reaped["id"], reaped["pid"]
reaped_pgid, reaped_sid = reaped["pgid"], reaped["sid"]
print(f"short-lived session {reaped_id}: pid {reaped_pid}, timeout "
f"{reaped['idle_timeout_s']}s")
assert reaped["idle_timeout_s"] == 5.0, reaped["idle_timeout_s"]
assert reaped["state"] == "ready", reaped["state"]
# Nothing touches it. The only wait is for the deadline to arrive.
machine.wait_until_succeeds(
cli(f"status --json {reaped_id}")
+ " | jq -e '.session.state == \"reaped\"' >/dev/null",
timeout=90,
)
record = json.loads(machine.succeed(cli(f"status --json {reaped_id}"), timeout=120))["session"]
assert record["state"] == "reaped", record["state"]
assert record["live"] is False, record
except Exception:
dump("the idle reaper never fired", locals().get("reaped_id"))
raise
with subtest("the reaped session's application is gone"):
try:
machine.wait_until_succeeds(gone(reaped_pid, reaped_pgid, reaped_sid), timeout=60)
except Exception:
dump("the reaper left the application running", reaped_id)
raise
with subtest("a session touched inside the window is not reaped"):
# The other half of the clock. Without this the subtest above would be
# satisfied by a daemon that killed everything after five seconds
# whatever the caller did, which proves nothing about `touch`.
#
# The numbers below are sized so that the subtest's own cleanup cannot
# race the deadline it is asserting about. The window restarts at the
# *last* touch, so everything after the loop -- the status round trip,
# `kill -0`, and the `kill` that clears the session away -- has to
# finish inside one whole window. A `systemd-run --wait` round trip
# costs several seconds in this guest, and three of them do not fit in
# six seconds: with a six-second window the session was legitimately
# reaped underneath its own cleanup and `kill` returned 4 ("not in a
# usable state"). The window is therefore tens of seconds, and that
# whole window is the margin the cleanup gets.
#
# What is asserted is unchanged, and stricter in degree. Touches are
# KEPT_INTERVAL apart -- a third of the window -- and the loop runs for
# KEPT_TOUCHES * KEPT_INTERVAL seconds, three whole windows of real
# time: a `touch` that did not reset the deadline would have let the
# reaper fire two windows before the loop ended, and the assertion
# below would read `reaped`. Two details keep it honest rather than
# lucky: the loop ends with a touch instead of a sleep, so the cleanup
# gets a whole window and not a window less one gap; and it runs under
# `set -e`, so a `touch` that fails fails the loop here instead of
# being discovered later as a reap.
KEPT_WINDOW = 45
KEPT_INTERVAL = 15
KEPT_TOUCHES = 9
try:
kept = json.loads(
machine.succeed(
cli(f"start --json --idle-timeout {KEPT_WINDOW}s foot"), timeout=180
)
)
kept_id, kept_pid = kept["id"], kept["pid"]
print(f"touched session {kept_id}: pid {kept_pid}, timeout {kept['idle_timeout_s']}s")
assert kept["idle_timeout_s"] == float(KEPT_WINDOW), kept["idle_timeout_s"]
machine.succeed(
"set -e; for i in "
+ " ".join(str(n) for n in range(1, KEPT_TOUCHES + 1))
+ f"; do sleep {KEPT_INTERVAL}; "
+ cli(f"touch {kept_id}")
+ " >/dev/null; done",
timeout=KEPT_TOUCHES * (KEPT_INTERVAL + 30) + 60,
)
record = json.loads(machine.succeed(cli(f"status --json {kept_id}"), timeout=120))["session"]
print(json.dumps(record, indent=2, sort_keys=True))
assert record["state"] == "ready", (
f"a session touched every {KEPT_INTERVAL}s for "
f"{KEPT_TOUCHES * KEPT_INTERVAL}s, inside a {KEPT_WINDOW}s window, "
f"is {record['state']!r}"
)
assert record["live"] is True, record
machine.succeed(f"kill -0 {kept_pid}")
# Leave nothing behind for the restart subtest to trip over.
machine.succeed(cli(f"kill {kept_id}"), timeout=180)
except Exception:
dump("a touched session was reaped anyway", locals().get("kept_id"))
raise
with subtest("a session survives a restart of the broker"):
try:
survivor = json.loads(machine.succeed(cli("start --json foot"), timeout=180))
survivor_id, survivor_pid = survivor["id"], survivor["pid"]
print(f"survivor {survivor_id}: pid {survivor_pid}")
machine.succeed("systemctl restart hhstream-sessions.service", timeout=180)
machine.wait_for_unit("hhstream-sessions.service", timeout=180)
# The new daemon rebinds the socket and adopts the live records
# before it answers; one bounded wait for it to answer at all,
# rather than for the answer we want.
machine.wait_until_succeeds(cli("list") + " >/dev/null", timeout=120)
record = json.loads(
machine.succeed(cli(f"status --json {survivor_id}"), timeout=120)
)["session"]
print(json.dumps(record, indent=2, sort_keys=True))
assert record["pid"] == survivor_pid, (
f"re-adopted as pid {record['pid']}, not {survivor_pid}"
)
assert record["state"] == "ready", f"re-adopted as {record['state']!r}"
assert record["adopted"] is True, "the record does not say it was adopted"
assert record["pgid"] == survivor["pgid"], record
assert record["sid"] == survivor["sid"], record
machine.succeed(f"kill -0 {survivor_pid}")
except Exception:
dump("the session did not survive the restart", locals().get("survivor_id"))
raise
with subtest("the re-adopted session is still killable"):
try:
reply = json.loads(machine.succeed(cli(f"kill --json {survivor_id}"), timeout=180))
assert reply["ok"] is True, reply
assert reply["session"]["state"] == "killed", reply["session"]
machine.wait_until_succeeds(
gone(survivor_pid, survivor["pgid"], survivor["sid"]), timeout=60
)
machine.succeed("systemctl is-active hhstream-sessions.service")
except Exception:
dump("the re-adopted session could not be killed", locals().get("survivor_id"))
raise
'';
}
+1
View File
@@ -70,6 +70,7 @@ So the requirements are:
| `tests/unit` | the framing, the capture wrapper and the server's client handling, with no compositor and no GPU |
| `checks.vm-basic` | the module actually starts both units, the socket appears with the right owner/mode, the Hello parses and two frames arrive — with a fake capture program |
| `checks.vm-hyprland` | the real thing: headless Hyprland comes up in the VM and `grim` captures a frame of the configured size |
| `checks.vm-sessions` | the session broker side: a real application runs as a tracked session on the compositor, the compositor's IPC sees its window, the idle reaper fires for real (and does not fire when the session is touched), and a broker restart adopts its sessions back |
`vm-hyprland` is the only check that can fail for reasons outside this repo, so
it is kept separate from `vm-basic` on purpose: when it fails, the compositor's
+300
View File
@@ -0,0 +1,300 @@
# CONTRACT — sessions build (sessions-v1)
This file is the interface for the **session** work: the broker, containers,
compositor IPC and pointer input. It is the sibling of `docs/CONTRACT.md`, which
freezes the *frame-socket* wire format; that one is untouched by this build.
**Immutable without a version bump.** Every slice codes against this and nothing
else. If something here is wrong or missing, **stop and report it in your final
message** — the parent edits this file, not you. Do not widen your slice to work
around a gap.
Version: sessions-v1 (2026-10-01).
---
## 1. What is being built, in one paragraph
A headless Hyprland desktop is driven by an agent. A broker (`hhstreamd`) starts
allowlisted applications as **sessions** and tracks them. This build gives each
session its **own output and workspace** (a spatial container), teaches the
broker to **talk to the compositor** (IPC), and adds **serialised pointer input**
aimed at one session, under a **lease** whose bookmark is stored with the
session.
Read `docs/PLAN-workspaces-and-pointer.md` only if your brief tells you to. It
describes the whole system; you are building one part.
---
## 2. File ownership — the rule that makes four sessions possible
| Slice | Owns (only these) |
| --- | --- |
| **A — stage0-rename** | `nix/module.nix`, `src/hhstream/cli.py` (the two socket-path defaults only), `examples/agent-eyes` |
| **B — ipc-client** | `src/hhstreamd/ipc.py`, `tests/ipc/**` |
| **C — containers** | `src/hhstreamd/containers.py`, `tests/containers/**` |
| **D — pointer** | `src/hhstreamd/pointer.py`, `src/hhstream_pointer/**`, `tests/pointer/**`, `pyproject.toml`, `nix/package.nix` |
| **parent (integration)** | `src/hhstreamd/{protocol,registry,manager,launcher,compositor,procmon}.py`, `checks/**`, `docs/**`, `src/hhstream/cli.py` (everything but the path defaults), `nix/module.nix` (after A merges) |
Two slices never edit the same file. A shared-file change you believe you need is
a **report**, not an edit.
**Never touch:** `/etc/nixos` (outside this repo), anything under
`/var/lib/hermes/nixos`, any systemd unit, any running service. Never run
`nixos-rebuild`, `nixosTest`, `nix build .#checks.*`, or launch a GUI
application. The parent serialises all of that on a 2-core machine.
---
## 3. Python seams (exact signatures — code to these)
### 3.1 `hhstreamd/ipc.py` (B)
```python
class CompositorUnavailable(Exception):
"""No request socket, or the compositor refused. Callers degrade, never guess."""
@dataclass(frozen=True)
class Monitor:
name: str; x: int; y: int; width: int; height: int; scale: float
active_workspace_id: int | None
@dataclass(frozen=True)
class Workspace:
id: int; name: str; monitor: str | None; windows: int; is_special: bool
@dataclass(frozen=True)
class Client:
address: str; pid: int; workspace_id: int | None; monitor: str | None
class_: str; initial_class: str; title: str; stable_id: str; visible: bool
@dataclass(frozen=True)
class CursorPos:
x: int; y: int
@dataclass(frozen=True)
class Event:
name: str # "openwindow", "workspace", "monitorremoved", ...
data: dict # parsed payload; keys are whatever the event carried
class HyprlandIPC:
@classmethod
def discover(cls, runtime_dir: Path, signature: str | None = None) -> "HyprlandIPC": ...
# raises CompositorUnavailable. Signature from the argument, else
# $HYPRLAND_INSTANCE_SIGNATURE, else the single directory under
# <runtime_dir>/hypr/. More than one and no signature = CompositorUnavailable.
def __init__(self, request_socket: Path, event_socket: Path | None = None) -> None: ...
def version(self) -> str: ...
def monitors(self) -> list[Monitor]: ...
def workspaces(self) -> list[Workspace]: ...
def clients(self) -> list[Client]: ...
def cursorpos(self) -> CursorPos: ...
def query(self, command: str) -> str: ... # raw request socket, for anything not wrapped
def events(self) -> Iterator[Event]: ... # blocking; raises CompositorUnavailable on EOF
def close(self) -> None: ...
def parse_event(line: str) -> Event: ... # "name>>a,b,c" -> Event
```
Rules: every method raises `CompositorUnavailable` rather than returning a
guess. `clients()` parses `clients -j`; `monitors()` parses `monitors -j`;
`workspaces()` parses `workspaces -j`. Stdlib only.
### 3.2 `hhstreamd/containers.py` (C)
**Pure logic. No I/O, no subprocess, no socket, no import of `ipc.py`.** It
computes command lines and parses nothing. Seams are primitives so this module
stands alone.
```python
@dataclass(frozen=True)
class ContainerPlan:
output: str # hh-<session-id>
workspace: int
mode: tuple[int, int] # width, height
create: list[list[str]] # full argv lists, in order, each ["hyprctl", ...]
teardown: list[list[str]]
def output_name(session_id: str) -> str: ...
# "hh-" + session_id. Ids are already [a-z0-9-]; assert and raise ValueError otherwise.
def contains_output(name: str) -> bool: ...
# True for names this broker creates (prefix "hh-"), so a start-up scan can
# find orphans.
def container_plan(session_id: str, workspace: int, width: int, height: int,
app_id: str) -> ContainerPlan: ...
# create: output create headless <name> ["hyprctl","output","create","headless",name]
# workspace rule ["hyprctl","keyword","workspace",
# f"{workspace}, monitor:{name}, default:true, persistent:true"]
# window rule ["hyprctl","keyword","windowrule",
# f"match:initial_class ^{app_id}$, workspace {workspace}, monitor {name}"]
# output mode ["hyprctl","keyword","monitor", f"{name},{width}x{height},auto,1"]
# teardown: output remove <name> ["hyprctl","output","remove",name]
def app_id_for(session_id: str) -> str: ...
# the bare "hh-<session-id>" the launcher sets on the app; "^hh-<session-id>$"
# is built in container_plan.
def quarantine_plan(quarantine_workspace: str) -> ContainerPlan | None: ...
# the backstop: every window matching no container goes to
# special:quarantine. Returns the keyword argv, or None if disabled.
class WorkspaceAllocator:
def __init__(self, base: int, count: int) -> None: ...
def allocate(self) -> int: ... # lowest free slot in [base, base+count); raises NoSlots
def release(self, workspace: int) -> None: ... # unknown id -> KeyError (a bug, not a no-op)
def in_use(self) -> list[int]: ...
class NoSlots(Exception): ...
```
### 3.3 `hhstreamd/pointer.py` (D)
Lease state machine + coordinate maths. **No Wayland, no socket, no I/O.** The
clock is injected so expiry is testable without sleeping.
```python
class LeaseBusy(Exception): # another session holds it
def __init__(self, holder: str) -> None: ...
class LeaseInvalid(Exception): # not held, or held by someone else / stale lease_id
@dataclass(frozen=True)
class Bookmark:
x: float; y: float # 0..1, output-local, normalised
output: str
at: str # ISO-8601 UTC
@dataclass(frozen=True)
class LeaseInfo:
session_id: str; lease_id: str; acquired_at: float; expires_at: float
hard_deadline: float
@dataclass(frozen=True)
class LeaseOutcome:
bookmark: Bookmark | None; reason: str; buttons_lifted: list[str]
class PointerLease:
def __init__(self, ttl_s: float = 30.0, max_s: float = 300.0,
clock: Callable[[], float] = time.monotonic) -> None: ...
def acquire(self, session_id: str) -> str: ... # -> lease_id; raises LeaseBusy
def refresh(self, session_id: str, lease_id: str) -> None: ... # raises LeaseInvalid
def release(self, session_id: str, lease_id: str, reason: str) -> LeaseOutcome: ...
def state(self) -> dict: ... # {"holder": str|None, "since": float|None,
# "lease_id": str|None, "expires_at": float|None}
def expire(self) -> LeaseInfo | None: ...
# force-release if past ttl or hard cap; returns what was released.
# The caller (broker) lifts buttons on the driver side.
def press(self, session_id: str, button: str) -> None: ...
def unpress(self, session_id: str, button: str) -> list[str]: ...
def pressed(self) -> list[str]: ...
def window_to_absolute(point: tuple[int, int], window: tuple[int, int, int, int],
monitor: tuple[int, int, int, int]) -> tuple[float, float]:
# point is window-local pixels; window is (x, y, w, h) global; monitor is
# (x, y, w, h) global. Returns normalised 0..1 in the MONITOR's frame,
# clamped to [0, 1]. Raises ValueError if the point is outside the window.
```
### 3.4 `src/hhstream_pointer/**` (D) — the driver process
A separate process, because a Wayland client needs its own connection and a
blocked driver must not stall the broker.
- Entry point `hhstream-pointer` (add to `pyproject.toml`:
`hhstream-pointer = "hhstream_pointer.__main__:main"`).
- Command line the module will pass, exactly:
`hhstream-pointer --socket /run/hhstream/pointer.sock --compositor-runtime-dir /run/hypr-stream`
- Uses **pywayland** (packaged as `python3Packages.pywayland`) and the **wlr
virtual pointer XML** from `pkgs.wlr-protocols`
(`share/wayland-protocols/wlr-unstable/zwlr-virtual-pointer-unstable-v1.xml`).
Compile the protocol at build time or vendor the generated module under your
directory — your choice, stated in your report. `pyproject.toml` and
`nix/package.nix` are yours precisely so this is possible.
- Binds `zwlr_virtual_pointer_manager_v1` **version 2** and uses
`create_virtual_pointer_with_output(seat, output, id)` so coordinates are
output-local. If only version 1 is offered, **report it — never silently fall
back** to a global pointer, which would break aiming.
- Private socket protocol with the broker, newline-delimited JSON:
```json
{"op": "warp", "output": "hh-x", "x": 0.42, "y": 0.61} -> {"ok": true}
{"op": "button", "button": "left", "state": "down"} -> {"ok": true}
{"op": "frame"} -> {"ok": true}
{"op": "pressed"} -> {"ok": true, "buttons": ["left"]}
{"op": "release_all"} -> {"ok": true, "lifted": ["left"]}
```
`{"ok": false, "error": "<text>"}` for anything rejected. The driver holds **no
lease state** — the broker owns the lease; the driver is an executor that must
never be trusted to serialise anything.
---
## 4. Wire format (broker control socket) — B, C and D do not implement this
Newline-delimited JSON over `/run/hhstream/control.sock`. Existing verbs are
unchanged. New verbs (integration implements; listed so slices know what their
code will be called with):
| Verb | Request (beyond `verb`, `request_id`) | Response |
| --- | --- | --- |
| `container.list` | — | `containers[]`, `orphans[]` |
| `container.reconcile` | `id?` | `container`, `diff` |
| `pointer.acquire` | `id`, `ttl_s?`, `restore?` | `lease_id`, `bookmark` |
| `pointer.move` | `id`, `lease_id`, `x`, `y` \| `dx`, `dy` | `bookmark` |
| `pointer.button` | `id`, `lease_id`, `button`, `state` | `buttons[]` |
| `pointer.click` | `id`, `lease_id`, `button?`, `count?` | `bookmark` |
| `pointer.release` | `id`, `lease_id` | `bookmark`, `released_by` |
| `pointer.state` | — | `holder`, `since`, `lease_id` |
### Error codes (extend `ExitCode`, do not renumber)
| 0 OK | 2 USAGE | 3 NO_SUCH_SESSION | 4 BAD_STATE | 5 DAEMON_UNREACHABLE |
| 6 POLICY | 7 TIMEOUT | 8 BUSY | **9 NO_LEASE (new)** | **10 COMPOSITOR (new)** |
Activity accounting is unchanged and load-bearing: **`touch` is the only mark of
activity**; `list`, `status`, `logs`, `gc`, and every `pointer.*` verb do **not**
reset the idle clock.
---
## 5. Names and paths (frozen)
| Thing | Value |
| --- | --- |
| Compositor runtime dir | `/run/hypr-stream` — **must be ≤ 25 chars** (socket paths must stay under `sun_path`'s 108-byte limit) |
| Compositor request socket | `<runtime dir>/hypr/<signature>/.socket.sock` |
| Compositor event socket | `<runtime dir>/hypr/<signature>/.socket2.sock` |
| Broker control socket | `/run/hhstream/control.sock` |
| Broker↔driver socket | `/run/hhstream/pointer.sock` |
| Broker state dir | `/var/lib/hermes/hhstream-sessions` |
| Session runtime dir | `/run/hhstream/sessions/<session-id>` |
| Container output name | `hh-<session-id>` |
| Container workspace band | `1000 + slot`, allocated from the registry's free list |
| Container mode | `1280x720` |
| Units | `hyprland-headless`, `hyprland-headless-stream`, `hhstream-sessions`, `hhstream-pointer` |
| Binaries | `hhstream`, `hhstreamd`, `hhstream-server`, `hhstream-client`, `hhstream-pointer` |
| Service account | `hermes`, supplementary group `hhstream` |
---
## 6. Rules of engagement
1. **Only your files** (§2). A shared-file change you need is a report.
2. **Unit tests only.** `nix develop -c python -m pytest tests/<yours> -q`.
Nothing that builds a VM, rebuilds the system, or touches a live service.
3. **Stdlib only**, except slice D which may add `pywayland`.
4. **Machine style:** module docstrings that say *why*, not what; comments carry
reasoning; no TODO placeholders; no speculative generality.
5. **Definition of done:** a commit on your branch; your tests pass; your final
message states (a) what you built, (b) the exact test command and its result,
(c) what you could **not** verify, (d) any contract gap you hit. A slice that
"finishes" without (b) is not done.
6. **Do not open by reading the whole repo.** Your brief plus this contract is
your context. Read the two or three files you actually touch.
7. If the contract is wrong, say so and stop. If your slice turns out to need
another slice's file, say so and stop. Both are successful outcomes.
+1 -1
View File
@@ -82,7 +82,7 @@ Namespace: `services.hyprland-headless-stream`
| `user` / `group` | str | `hhstream` | owner of the session and the socket |
| `frameRate` | int | `30` | capture target, becomes `fps` in Hello |
| `format` | enum `png`/`jpeg` | `png` | frame encoding |
| `outputs` | list of attrsets | one 1280×720@30 `HEADLESS-1` | virtual monitors: `name`, `width`, `height`, `refresh`, `scale` |
| `outputs` | list of attrsets | one 1280×720@30 rule with an empty `name` | monitor rules applied to the outputs the machine has: `name`, `width`, `height`, `refresh`, `scale`. `name` is a regex matched against the real connector names (`eDP-1`, `Virtual-1`, …) and **empty means every output**; a name no connector has is silently ignored. The compositor never creates an output — a session started by this module drives the host's GPU. |
| `extraConfig` | str | `""` | extra lines appended to `hyprland.conf` |
| `hyprlandPackage` | package | `pkgs.hyprland` | which compositor |
| `package` | package | the flake's own build | the frame server/client |
+662
View File
@@ -0,0 +1,662 @@
# Plan — per-session workspaces, outputs, and pointer input
Technical specification for the next milestone of the headless-Hyprland session
broker. Every claim about the compositor in here is either **[RAN]** on this
machine or read from the running build's source at tag `v0.56.2`; the evidence
markers and citations are in `REPORT.md` (same directory as the brief that
produced it).
Status: **proposal — not approved, nothing built.** Stage 0 is a prerequisite for
all of it and is nearly free.
---
## 1. Goals and non-goals
### Goals
| ID | Goal | Verified possible? |
| --- | --- | --- |
| G1 | A session owns a spatial container: listable, capturable in isolation, and torn down as a container — not just a process group | Yes — one headless output + one persistent workspace per session |
| G2 | A session's identifier is returned at start and is sufficient later to observe, keep alive, kill, and now *target input at* | Yes — `session start` already prints the bare id; the record gains container fields |
| G3 | Mouse input aimed at one session, without disturbing another session's contents | Yes — output-bound virtual pointer, serialised by a broker-wide lease |
| G4 | Several sessions coexist, isolated enough that one cannot disturb another's container | Partly — spatial isolation yes; *adversarial* isolation needs Stage 4; crash isolation needs Stage 5 and this machine cannot afford it |
### Non-goals (deliberate, for now)
- **Concurrent pointer input in one compositor.** Structurally impossible: one
`wl_seat`, one `m_pointerPos`. A second cursor requires a second compositor.
- **Per-tenant compositors on this machine.** Hyprland consumes 96.6% of a core
continuously at `debug:vfr = false` with software rendering on a 2-core box.
Stage 5 is specified but gated on the CPU question.
- **Typing.** Keyboard injection uses the same seat and the same serialisation
question; it is specified as an extension of the pointer lease (§6.7) but not
in the first three stages.
- **A general window-management API** (move/resize/stack). The session owns a
container; it does not manage windows inside it.
---
## 2. Architecture
```
┌──────────────────────────────┐
frame clients ──▶│ hyprland-headless-stream │ JPEG frames of ONE output
│ (frame server, stream.sock) │
└──────────────┬───────────────┘
│ grim -o <output>
┌──────────────────────────────┴──────────────────────────────┐
│ hyprland-headless.service — ONE compositor, N outputs │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ output hh-A │ │ output hh-B │ │ output main │ │
│ │ ws 101 │ │ ws 102 │ │ ws 1 │ │
│ │ [app A] │ │ [app B] │ │ wallpaper │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
└───────▲──────────────────▲───────────────────▲──────────────┘
│ request socket │ socket2 (events) │ virtual pointer
│ hyprctl / IPC │ │ (output-bound)
┌───────┴──────────────────┴───────────────────┴──────────────┐
│ hhstream-sessions.service — the broker (as `hermes`) │
│ registry │ launcher │ ipc client │ pointer service │ reaper │
└───────▲─────────────────────────────────────────────────────┘
│ control socket (JSON request/response)
┌───────┴─────────────────────────────────────────────────────┐
│ hhstream CLI — status│shot│run│record│session … │
└─────────────────────────────────────────────────────────────┘
```
Four moving parts change; the frame server is untouched except for which output
it captures.
### 2.1 Component responsibilities
| Component | Owns | Must not |
| --- | --- | --- |
| `hyprland-headless` (compositor) | seats, outputs, workspaces, windows, the one cursor | know about sessions |
| `hhstreamd` (broker) | the registry, session lifetime, containers, the pointer lease, the audit log | guess compositor state; act on a container it did not create without verifying |
| `hhstream` (CLI) | argument parsing, exit codes, human formatting | hold state; talk to the compositor directly |
| frame server | encoding and publishing frames of a nominated output | know which output belongs to whom |
---
## 3. Stage 0 — rename the runtime directory (prerequisite)
**Problem.** `$XDG_RUNTIME_DIR` is `/run/hyprland-headless-stream` (29 chars).
Hyprland's IPC sockets are `$XDG_RUNTIME_DIR/hypr/$HIS/.socket.sock` and
`…/.socket2.sock`, and the instance signature alone is ~30 chars, so the path
exceeds `sun_path`'s 108-byte limit. The compositor logs
`ERR ]: Socket2 path is too long. (2) IPC will not work.` **[RAN]**
**Consequence today.** The *request* socket works by luck of residual truncation
(it is silently a different, shorter path both sides agree on); the *event*
socket does not work at all. A generated comment in the Hyprland config asserts
the IPC is misnamed and that `hyprctl clients` cannot connect — that is wrong in
cause and conclusion; `hyprctl version|monitors|workspaces|clients|devices|
layers|instances|cursorpos|status` all answer correctly against the live
compositor **[RAN]**.
**Change.** Rename to `/run/hypr-stream` (15 chars) everywhere:
- `nix/module.nix`: `RuntimeDirectory` for both units, the published socket
`default`, and the broker's `--compositor-runtime-dir`.
- `/etc/nixos/hyprland-stream.nix`: `socketPath`, `sessions.stateDir` is
unaffected.
- The frame server's `HOME`/`WorkingDirectory`.
- The `hhstream` CLI's default frame socket.
- Remove the incorrect comment claiming IPC is broken, and replace it with the
measured reason for the name length (**hard requirement: ≤ 25 chars**, so
socket paths stay under `sun_path`).
**Gate.** In a VM: both sockets exist at their real names; `hyprctl` answers;
a socket2 client receives `openwindow`/`workspace` events for a window it
created. **Rollback:** the rename is config-only; reverting restores today's
behaviour exactly (including the broken event socket).
**Risk.** Anything holding the old path. Enumerated above; there is nothing else
on this machine.
---
## 4. Stage 1 — the broker speaks IPC
### 4.1 IPC client (`src/hhstreamd/ipc.py`)
One client, two sockets:
| Socket | Use | Failure behaviour |
| --- | --- | --- |
| `hypr/$HIS/.socket.sock` | request/response queries | retry once, then degrade (§4.4) |
| `hypr/$HIS/.socket2.sock` | event stream | subscribe; on loss, reconcile by re-querying |
Instance discovery: `HYPRLAND_INSTANCE_SIGNATURE` if set, else the single
`$XDG_RUNTIME_DIR/hypr/<sig>` directory. Store the signature in the session
record so a restart re-attaches to the same instance rather than guessing.
Commands used (all **[RAN]** as available on this build):
| Purpose | Command |
| --- | --- |
| liveness / version | `version` |
| outputs and their rects | `monitors` (with per-output `x,y,width,height,scale`) |
| workspaces and their monitors | `workspaces` |
| windows with class, title, workspace, monitor, `stableId` | `clients -j` |
| cursor position (diagnostics only) | `cursorpos` |
Events consumed from socket2: `openwindow`, `closewindow`, `movewindow`,
`movewindowv2`, `activewindow`, `activewindowv2`, `workspace`, `workspacev2`,
`focusedmon`, `monitoradded`, `monitorremoved`, `urgent`.
### 4.2 Reconciler
An event stream that can break is not a source of truth. So:
1. Every mutation the broker performs is followed by a **read-back** of the
affected fact (`clients -j` filtered to the session's container).
2. On socket2 reconnect, a full reconcile runs: for each live session, compare
the record against `clients -j` and repair or report.
3. Reconciliation is the *only* writer of `window`/`workspace`/`output` fields
from events; events merely schedule a reconcile of one session.
### 4.3 Record fields this fills
`workspace`, `window`, `foreground`, plus new `container` (§7). `foreground` is
defined as the **monitor-active-workspace join**, not the client's `visible`
field, so it means "a human or the frame server is actually looking at this".
### 4.4 Degraded mode
If no IPC is reachable: sessions still start, stop, reap and are tracked by
process identity — exactly today's behaviour. `workspace`/`window` stay `null`
(never guessed), `container: "none"`, and one line is logged at startup and
reported in `status`. Lifecycle verbs never depend on IPC.
### 4.5 Teardown cross-check
Today teardown is process-group only. From here: after killing, verify from the
compositor (no client with this session's app-id remains) and report a
discrepancy as `survivors: [...]` rather than declaring success. Two independent
sources — `pidfd`/`/proc` and the compositor — must agree.
---
## 5. Stage 2 — one output and one workspace per session
### 5.1 Container creation (on `start`)
1. `hyprctl output create headless hh-<session-id>` **[UNVERIFIED — experiment 3]**
(strongly implied: Hyprland requests the headless backend as
`AQ_BACKEND_REQUEST_MANDATORY`; `dispatchOutput` checks
`getImplementations()` for `AQ_BACKEND_HEADLESS`).
2. Generate a synthetic config for that output via `hyprctl keyword` (immediate,
instance-scoped, no reload):
- `workspace = <n>, monitor:hh-<id>, default:true, persistent:true`
- `windowrule = match:initial_class ^hh-<id>$, workspace <n>, monitor hh-<id>`
- the output's mode, taken from the container's size (default 1280×720).
3. Launch the app with a matching identity so the rule binds: the app is
started in a **`systemd-run --user` scope? No** — see §5.4 — the app is
launched as today (plain `Popen`), with `--class`/`--app-id` set per app in
the policy where the app supports it, and `XDPP_APP_ID`-style environment
where it does not.
4. Read back: the client's `workspace`, `monitor` and `stableId`; write them to
the record.
Workspace numbering: containers get workspaces from a reserved band
(1000 + slot) so nothing collides with the operator's own workspaces; the slot
is allocated from a free list held in the registry and released on teardown.
### 5.2 The catch-all window rule
The current config has `windowrule = match:class .*, float on, size 1040 560,
center on` — justified by the belief that IPC could not be queried. With Stage 1
that belief is gone, so it is replaced by:
- per-container rules (`match:initial_class ^hh-<id>$`), and
- a **quarantine backstop**: `match:class .*, workspace special:quarantine`.
A window that matches no container lands in a hidden special workspace and is
reported by `session list --unmatched`, instead of appearing on whatever
output happened to be focused.
### 5.3 Teardown (on `kill`, reap, or gc)
1. Send the configured signal to the process group; wait `grace`; escalate.
2. `hyprctl output remove hh-<id>` — removes the output, and with it the
container. Windows on it are destroyed by the compositor when their surface
is destroyed.
3. Release the workspace number and the pointer bookmark.
4. Record a tombstone with `container: "removed"` and the reason.
Reversible in one step: `output remove` is the container's deletion, and it is
also the recovery path — a broker that has lost its registry can enumerate
outputs named `hh-*` and converge (an orphan `hh-*` output with no session is
either re-adopted or removed, never left behind).
### 5.4 Per-session capture
- `grim -o hh-<id> out.jpg` captures one container in isolation.
- The frame server keeps publishing the **main** output for backwards
compatibility (`hhstream shot`), and gains `--output hh-<id>` to publish or
capture a specific container.
- `grim -T <stableId>` (a window on a non-visible workspace) is **unverified**
(experiment 1). The design does not depend on it: containers are outputs, so
`grim -o` is always available. `-T` is a convenience if it works.
### 5.5 Container sizing and cost
Each output is a real render target. At `vfr = false` the compositor renders
every output every frame **[UNVERIFIED — experiment 2]**. Therefore:
- `max_sessions` gains a **container budget** (`max_containers`, default 2 on
this machine, to be re-derived from the measurement).
- The measurement decides whether `vfr = true` is adopted (screencopy may drive
rendering on demand, making it free performance). If `vfr` stays false, the
container budget is a hard CPU decision, not a preference.
---
## 6. Stage 3 — the pointer lease and driver
### 6.1 What the compositor provides
- `zwlr_virtual_pointer_manager_v1` **version 2** is a live global **[RAN]**.
- `create_virtual_pointer_with_output(seat, output, id)` — Hyprland stores
`m_boundOutput` on the device and maps normalised absolute motion into that
output's rectangle (`CPointerManager::warpAbsolute`, `PointerManager.cpp:905-920`)
**[SRC]**.
- `motion_absolute(time, x, y, w, h)`, `button(time, btn, state)`, `frame()`.
- **No per-device position exists** (`IPointer.hpp`: only `m_boundOutput`,
`m_connected`, flip flags, scroll factor) and **no query** for the cursor
position. Position is one global `m_pointerPos`.
### 6.2 The lease — state machine
```
acquire (free) release / ttl / crash
FREE ───────────────▶ HELD(lease) ─────────────────────────▶ FREE
▲ │ ▲
│ force-release │ │ events (move/click) refresh TTL
└───────────────────────┘ │
└── any verb without the lease → BUSY (8)
```
| Transition | Trigger | Side effects |
| --- | --- | --- |
| acquire | `session pointer acquire <id>` | take lock; warp from bookmark (or output centre); start TTL |
| refresh | any gesture verb with a valid lease | extend TTL |
| release | `session pointer release <id>` | write bookmark; lift any pressed buttons; free lock; stamp reason |
| timeout | TTL (default 30 s) with no verb, or hard cap (default 300 s) | force-release; lift buttons; event `pointer_lease_timeout` |
| crash | broker restart | leases are stale by definition → free; bookmarks intact |
Rules:
- **One holder.** The lock is broker-wide because the cursor is one. A second
`acquire` returns `BUSY` (8) with the current holder's id.
- **Gesture atomicity.** The lease must span warp → press → move → release. The
broker will not interleave another session's events, and it *refuses*
`pointer move` for a session that does not hold the lease.
- **Never leave a button down.** On release/timeout with any button pressed, the
broker sends the button-up and records `buttons_lifted: [BTN_LEFT]`.
- **Bookmark written on every move and on release**, so a crash leaves the last
real position.
### 6.3 Bookmark representation
```json
"bookmark": {"x": 0.42, "y": 0.61, "output": "hh-<id>", "at": "2026-10-01T…Z"}
```
Output-local **normalised** coordinates, because that is the frame
`motion_absolute` already speaks: the value survives output resizes, layout
changes and container renumbering, and needs no conversion on the way in.
### 6.4 Aiming
Converting a window-local pixel target to `motion_absolute` inputs:
1. Get the client's box from `clients -j` (or the container's rect if aiming at
the output).
2. Map box-local `(px, py)` → global → subtract the output's `(x, y)` from
`monitors` → divide by the output's width/height.
3. Emit `motion_absolute(t, X, Y, X_extent, Y_extent)` then `frame()`.
4. Settle: `animations = false` is already set and is now a **correctness**
requirement — a composition animation between warp and click moves the target
under the cursor.
### 6.5 Driver placement
The driver is a **separate small process**, `hhstream-pointer`, not part of the
broker's event loop: a Wayland client needs a connection of its own, and a
blocked driver must not stall session lifecycle. The broker owns the lease and
speaks to the driver over a private socket; the driver owns the Wayland
connection and the per-session `zwlr_virtual_pointer_v1` objects. If the driver
dies, the broker force-releases the lease.
### 6.6 What the lease does not protect
Nothing in the compositor stops *any* client from creating its own virtual
pointer and moving the shared cursor — the global is advertised to everyone. The
lease disciplines our sessions; it does not enforce against a hostile one. Stage
4 is what makes it enforceable (the compositor's own global filter can deny
input injection to a sandboxed session).
### 6.7 Keyboard (specified, later)
Same seat, same serialisation. `zwp_virtual_keyboard_manager_v1` exists **[RAN]**.
The lease gains a `keyboard` flag so a typing burst is atomic; no new mechanism
is needed. Not in Stages 0–3.
---
## 7. Data model — the session record (schema 3)
Today (`schema: 2`) the record has identity, lifecycle, and reserved-but-empty
spatial fields. Schema 3 adds the container and the pointer, and makes the
existing `null`s meaningful:
```json
{
"schema": 3,
"id": "foot-v7r4li",
"app": "foot",
"argv": ["…"],
"pid": 1288, "pgid": 1288, "sid": 1288,
"pid_starttime": 8100, "boot_id": "…",
"exit_detected_by": "pidfd",
"state": "ready", // spawned|ready|exited|killed|reaped|failed
"created_at": "…", "created_boottime": 81.0,
"ready_signal": "spawned",
"idle_timeout_s": 10800.0, "last_activity_at": "…", "last_activity_boottime": 106.5,
"tombstoned_boottime": null,
"home": "/var/lib/hermes", "ephemeral_home": false,
"runtime_dir": "/run/hhstream/sessions/foot-v7r4li",
"log_path": "/var/lib/hermes/hhstream-sessions/logs/foot-v7r4li.log",
"container": { // NEW — null until Stage 2
"output": "hh-foot-v7r4li",
"workspace": 1003,
"mode": {"width": 1280, "height": 720},
"created_by": "broker",
"state": "owned" // owned|degraded|removed|orphan
},
"window": {"stable_id": "…", "class": "foot", "title": "…"}, // Stage 1
"workspace": 1003, // Stage 1
"foreground": false, // Stage 1 (monitor-active join)
"pointer": { // NEW — Stage 3, per-session bookmark
"bookmark": {"x": 0.42, "y": 0.61, "output": "hh-foot-v7r4li", "at": "…"},
"last_released_at": "…", "released_by": "lease|timeout|crash|kill"
},
"command_count": 8, "adopted": false, "last_error": null,
"signature": "efb5…" // NEW — HYPRLAND_INSTANCE_SIGNATURE
}
```
Invariants:
- `container: null` means "no spatial container" (degraded or pre-Stage-2), never
"unknown but probably something".
- `pointer` is written on release and on every move; a missing `pointer` is
"never used", not "position unknown".
- Every field the broker cannot verify stays `null`. No field is ever guessed.
---
## 8. Wire protocol (broker control socket)
Newline-delimited JSON request/response over the existing control socket. Every
request gains a `request_id` (client-supplied) for idempotency; the broker keeps
the last N completed ids per session and returns the recorded answer rather than
repeating a mutation.
### 8.1 Verbs
| Verb | Request | Response | Stage |
| --- | --- | --- | --- |
| `session.start` | `app`, `argv[]`, `idle_timeout_s?`, `home?`, `ephemeral_home?`, `container?` | `session` (record) | existing (+container) |
| `session.list` | `all?` | `sessions[]` | existing |
| `session.status` | `id`, `events?` | `session`, `events[]` | existing |
| `session.touch` | `id` | `session` | existing |
| `session.kill` | `id`, `grace?`, `signal?` | `session`, `survivors[]` | existing (+cross-check) |
| `session.gc` | — | `removed[]`, `orphans[]` | existing |
| `session.adopt` | `pid`, `app?`, `idle_timeout_s?` | `session` | existing |
| `session.logs` | `id`, `lines?` | `text` | existing |
| `container.list` | — | `containers[]` (incl. orphans) | **2** |
| `container.reconcile` | `id?` | `container`, `diff` | **2** |
| `pointer.acquire` | `id`, `ttl_s?`, `restore?` | `lease_id`, `bookmark` | **3** |
| `pointer.move` | `id`, `lease_id`, `x`, `y` \| `dx`, `dy` | `bookmark` | **3** |
| `pointer.button` | `id`, `lease_id`, `button`, `state` | `buttons[]` | **3** |
| `pointer.click` | `id`, `lease_id`, `button?`, `count?` | `bookmark` | **3** |
| `pointer.release` | `id`, `lease_id` | `bookmark`, `released_by` | **3** |
| `pointer.state` | — | `held_by`, `since`, `bookmark` | **3** |
### 8.2 Error model
Existing exit/status codes are kept and extended:
| Code | Name | Meaning | New? |
| --- | --- | --- | --- |
| 0 | OK | | |
| 2 | USAGE | malformed request | |
| 3 | NO_SUCH_SESSION | unknown id | |
| 4 | BAD_STATE | e.g. kill on a reaped session | |
| 5 | DAEMON_UNREACHABLE | socket missing/refused | |
| 6 | POLICY | app or argument not allowlisted; container budget exhausted | |
| 7 | TIMEOUT | wait_ready/grace expired | |
| 8 | BUSY | **pointer lease held by another session** | |
| 9 | NO_LEASE | **verb requires the lease and this session does not hold it** | new |
| 10 | COMPOSITOR | **IPC or output operation failed; session survives in degraded mode** | new |
`COMPOSITOR` is distinct from a session failure on purpose: losing the
compositor must not look like losing the session.
### 8.3 Activity accounting (unchanged, restated because it is load-bearing)
`touch` is the only mark of activity. `list`, `status`, `logs`, `gc`, and **all
`pointer.*` verbs** do *not* reset the idle clock — a session that is being
clicked in must still be kept alive explicitly, or a driven-but-idle session
would never be reaped.
---
## 9. CLI surface
```
hhstream status
hhstream shot [--out FILE] [--output NAME] # --output NEW
hhstream run [--] CMD… # unmanaged
hhstream record [--seconds N] [--out FILE] [--output NAME]
hhstream session start APP [-- ARGV…] [--json]
[--idle-timeout D] [--wait-ready D] [--home P | --ephemeral-home]
[--no-container] # NEW: skip the container
hhstream session list [--all] [--json]
hhstream session status ID [--events N] [--json]
hhstream session touch ID
hhstream session kill ID [--grace D] [--signal S]
hhstream session logs ID [--lines N] [--follow]
hhstream session gc
hhstream session adopt --pid PID [--app A] [--idle-timeout D]
hhstream session container ID [--json] # NEW
hhstream session reconcile [ID] # NEW
hhstream session pointer acquire ID [--ttl D] [--no-restore] # NEW
hhstream session pointer move ID --to X,Y | --by DX,DY # NEW
hhstream session pointer click ID [--button left|right|middle] # NEW
hhstream session pointer down ID [--button B] # NEW
hhstream session pointer up ID [--button B] # NEW
hhstream session pointer release ID # NEW
hhstream session pointer state [--json] # NEW
```
Human output keeps the current conventions: the bare identifier on stdout for
`start`, detail on stderr, `--json` for the protocol object. Machine-facing
consumers always use `--json`; human text is never parsed.
`list` gains two columns and one honest gap marker:
```
ID APP STATE WS OUTPUT POINTER IDLE EXPIRES
foot-v7r4li foot ready 1003 hh-foot-v7r4li free 0:12 2:59:48
chromium-abc chromium ready 1004 hh-chromium-abc held:foot 0:03 2:59:57
```
---
## 10. Policy (TOML) — new keys
```toml
[broker]
max_sessions = 4
max_containers = 2 # NEW — CPU-bounded, from experiment 2
degraded_ok = true # NEW — allow sessions with no container/ IPC
[compositor]
signature = "auto" # NEW — or a pinned HYPRLAND_INSTANCE_SIGNATURE
runtime_dir = "/run/hypr-stream" # NEW — must be ≤25 chars
workspace_base = 1000 # NEW — reserved band for containers
container_mode = "1280x720" # NEW
quarantine_enabled = true # NEW
[pointer]
lease_ttl_s = 30 # NEW — idle TTL before force-release
lease_max_s = 300 # NEW — hard cap for a single lease
settle_ms = 60 # NEW — wait after warp before pressing
button_default = "left"
restore_on_acquire = true # NEW — warp from bookmark
[apps.<name>]
exec = "…"
args_allow = [ … ] # unchanged: per-argument fnmatch, no shell
app_id = "hh-<session>" # NEW — identity the window rule matches
```
---
## 11. NixOS module changes
Options added to `services.hyprland-headless-stream`:
| Option | Type | Default | Purpose |
| --- | --- | --- | --- |
| `sessions.containers.enable` | bool | `true` | per-session output+workspace |
| `sessions.containers.max` | int | `2` | CPU budget (measure first) |
| `sessions.containers.mode` | str | `"1280x720"` | container size |
| `sessions.containers.workspaceBase` | int | `1000` | reserved band |
| `sessions.pointer.enable` | bool | `false` | start `hhstream-pointer` |
| `sessions.pointer.leaseTtl` | int (s) | `30` | |
| `sessions.pointer.leaseMax` | int (s) | `300` | |
Unit changes:
- `hyprland-headless.service`, `hyprland-headless-stream.service`,
`hhstream-sessions.service`: `RuntimeDirectory` → `hypr-stream` (Stage 0).
- New `hhstream-pointer.service`: same user `hermes`, supplementary group
`hhstream` (it needs the compositor's Wayland socket), `After=` the
compositor, `Restart=on-failure`, `KillMode=process` (same reasoning as the
broker: it must not take sessions down when it restarts).
- tmpfiles: the pointer service's private runtime dir; the state dir already
exists.
- Assertion: `runtimeDirectory` length ≤ 25, failing evaluation rather than
producing a compositor with a broken event socket.
---
## 12. Isolation (Stage 4) and per-tenant compositors (Stage 5)
**Stage 4 — sandbox a session's application.** Launch it through a
`wp_security_context_manager_v1` listener with `sandbox_engine` and
`app_id`/`instance_id` set per session, and hand it the restricted socket instead
of the compositor's. The compositor's global filter then denies it: screen
capture, input injection, enumerating other sessions' windows, layer-shell
overlays, clipboard reads. This is what gives the pointer lease teeth (§6.6).
Still not achievable: protection from a compositor crash, `xdg_activation`
focus-stealing, and a stray window landing on another container.
**Stage 5 — per-tenant compositors.** `hyprland-session@.service` template;
headless backend (never nested, never DRM); per-instance `RuntimeDirectory`
≤25 chars; ideally a per-tenant user; per-session socket resolution in the broker
instead of one global `--compositor-runtime-dir`. Gives truly concurrent
pointers, crash isolation, kernel-level separation. **Gated on the CPU
measurement** — at 96.6% of a core for one compositor on a 2-core box, this is
not affordable today.
---
## 13. Failure modes
| Failure | Detection | Behaviour |
| --- | --- | --- |
| Compositor restarts | socket2 EOF; `version` fails | sessions are dead (their clients died with it) → mark `container: orphan`, reaper reports; broker re-attaches, does not pretend |
| Broker restarts | unit restart | sessions live on (`KillMode=process`, plain `Popen`); adoption re-reads `clients -j` and re-binds containers by output name |
| Output creation fails | `hyprctl output create` error | session starts **without** a container, `container: "degraded"`, exit code 10 from `container` verbs, lifecycle unaffected |
| socket2 path too long | compositor log line | refuse to start in container mode; assert in the module (Stage 0 prevents it) |
| A window matches no container | `openwindow` + no rule match | quarantined to a hidden workspace; reported by `container.list` as orphan |
| Pointer driver dies | socket EOF | force-release the lease with reason `driver_gone`; sessions unaffected |
| Lease holder goes silent | TTL | force-release after `lease_ttl_s`; buttons lifted; audit event |
| Stuck button | button state in the lease | lifted on release/timeout; `buttons_lifted` recorded |
| Registry lost, containers present | start-up scan | enumerate `hh-*` outputs and adopt or remove; never leave a container unattended |
| `grim -o` fails on a container | capture error | frame server falls back to the main output and reports degraded, rather than publishing a black frame |
---
## 14. Testing and acceptance gates
Same discipline as today: **unit tests plus a real NixOS VM check per stage, all
green before any switch**, and each check must be shown to fail when its
mechanism is broken (mutation test), because twice now a check passed for the
wrong reason.
| Stage | Unit | VM check | Key assertions |
| --- | --- | --- | --- |
| 0 | socket-path length is derived, not hardcoded | extend `vm-hyprland` | both sockets exist at real names; a socket2 subscriber receives `openwindow` for a window the test creates; `hyprctl` answers |
| 1 | IPC parse/format against recorded fixtures; reconciler diff logic | `vm-ipc` | `workspace`/`window`/`foreground` are filled and match `clients -j`; teardown cross-check reports a deliberately-orphaned window as a survivor |
| 2 | workspace-slot allocator; rule generation | `vm-containers` | one output per session, named `hh-*`; a second window of the same app lands in the same container; an unmatched window is quarantined; `grim -o` captures only that container; teardown removes the output |
| 3 | lease state machine (TTL, hard cap, force-release, button lift); coordinate conversion | `vm-pointer` | acquire warps from the bookmark; a click lands where aimed (verified by reading back the *app's* state, not pixels); a second acquire returns BUSY; timeout releases and lifts buttons; a broker restart leaves the lock free and the bookmark intact |
| 4 | policy filter expectations | `vm-sandbox` | a sandboxed app cannot bind a virtual pointer or capture the screen (both fail with a protocol error) |
Every check runs with an injected clock for the TTL logic, and with real time
for at least one reap/expiry assertion — the existing `vm-sessions` does exactly
this and it is why the race in the negative control was found.
---
## 15. Open questions and experiments
1. **Window capture off-screen** (`grim -T <stableId>`) — unverified; the design
does not depend on it (containers are outputs).
2. **Container cost at `vfr = false`** — gates `max_containers` and Stage 5.
Procedure: create one headless output, measure the compositor's CPU for a
minute, remove it; repeat with `vfr = true` and check the frame server still
receives frames. Both mutate a live desktop, which is why they were not run.
3. **Does `hyprctl output create headless` succeed on this instance?** Implied by
source, unverified.
4. **Decision, operator:** is serialised pointer input acceptable, or is
concurrent input a hard requirement? Only the latter forces Stage 5.
5. **Decision, operator:** the container budget (`max_containers`) if experiment
2 shows a per-output cost that matters.
6. **Decision, operator:** whether `hhstream pointer` verbs are part of the
session-lease API now, or the lease is internal until input injection is
proven useful end-to-end.
---
## 16. Recommended sequencing
1. **Stage 0** — rename, measured reason in the comment. Hours, unblocks
everything, worth doing alone.
2. **Stage 1** — IPC client + reconciler + record fields. Small.
3. **Experiment 2** (cost) — because it sets `max_containers` and whether Stage
5 is ever reachable.
4. **Stage 2** — containers. Small-to-medium.
5. **Stage 3** — pointer lease + driver. Small, given the lease is specified
here.
6. **Stage 4** — sandboxing, when adversarial isolation is wanted.
7. **Stage 5** — only on evidence.
Each stage ships with its check, its rollback (revert the commit; containers and
outputs are removed by `gc`), and an entry in the audit trail.
+116
View File
@@ -0,0 +1,116 @@
# examples
## `agent-eyes` — the short commands
The everyday front door to the session: one subcommand each, no environment
juggling, and it re-execs itself under `sg hhstream` when the login predates the
group membership.
```bash
examples/agent-eyes status # services, socket, group access, liveness probe
examples/agent-eyes shot --out /tmp/s.jpg # one frame of the screen
examples/agent-eyes run -- chromium ... # launch a GUI app INTO the session
examples/agent-eyes record --seconds 10 --out clip.mp4
```
- `shot` defaults to `/var/lib/hermes/public/agent-eyes/latest.jpg`, which the
local file server already publishes, so the newest frame is always one
browser refresh away.
- `run` sets `XDG_RUNTIME_DIR`, `WAYLAND_DISPLAY` (it discovers whichever
`wayland-N` exists rather than assuming `wayland-0`) and `XDG_SESSION_TYPE`,
then `exec`s the command. Everything it launches runs as *your* user, not the
service user.
- `record` reads frames off the socket for the given wall-clock seconds and
encodes at the **measured** rate, so the clip plays at true speed. It strips
metadata for the media stores, same as the demo.
Both `shot` and `record` want nothing but `hhstream-client`, `python3` and
`ffmpeg`, which are all on the system PATH; only `chromium` needs `nix develop`.
## `agent-eyes-demo.py`
Puts a real browser on the headless screen, drives it, and films the result by
reading frames off the published socket — the consumer side of
[`docs/CONTRACT.md`](../docs/CONTRACT.md).
```bash
nix develop
sg hhstream -c 'python3 examples/agent-eyes-demo.py --query "Ari Sosnovsky"'
```
Output is an MP4 (default `agent-eyes.mp4`): the Google homepage, the query
typed in a letter at a time, then the results page — captured from the
compositor, not from a screencast API.
Everything else is a flag: `--seconds`, `--capture-fps`, `--output`,
`--work-dir`, `--keep-work`, `--debug-port`, `--clear-screen`. `--help` is
accurate.
### What it checks rather than assumes
- the session is up and the socket is readable (a clear message if your user is
not in the stream's group);
- the browser actually reached a results page — it presses Enter, then waits for
the URL to change, and submits the form directly if Enter did nothing;
- it *fails* on Google's `/sorry/` bot wall instead of encoding a video of a
CAPTCHA, because a video that looks like a working demo and is not one is
worse than no video.
### Three things that will bite you
1. **Your shell needs the group.** The socket is `0660 hhstream:hhstream` inside
a `0750` directory; a browser also has to reach the session's Wayland socket
in the same directory. Add yourself to `hhstream` (this is what
`/etc/nixos/hyprland-stream.nix` does for `hermes`) and log in again — or use
`sg hhstream -c '...'` for the shell you already have open.
2. **Google rate-limits you.** The first search from a fresh profile usually
passes; the third in a minute gets `unusual traffic`. It is the IP, not the
browser. Wait a few minutes and re-run.
3. **The frame server was the bottleneck, and it was PNG.** Measured with `grim`
at 1280×720 on this software-rendered VM: PNG ~326 ms/frame (a ~3 fps
ceiling), PNG `-l 1` ~105 ms, **JPEG q80 46 ms**. The session therefore
streams JPEG and the demo records ~22 fps. The script never assumes the
format — it takes the extension from the Hello and hands it to ffmpeg — so
either setting works. Leave `--capture-fps` at its default of 30 so nothing
is thrown away: the encoder uses the *measured* rate, so the video plays at
real speed either way, and decimating only makes it choppier.
### It films a desktop, not a browser
The browser is **windowed by default** (1040×560, floating, centred), so the
session's wallpaper, gaps and window borders are all in shot — that is the
difference between "a screen recording of a browser" and "here is the machine I
look at". `--fullscreen` gives the old full-bleed kiosk window.
The desktop furniture itself lives in `/etc/nixos/hyprland-stream.nix` (wallpaper
via `swaybg`, `gaps_out = 36`, a catch-all float window rule), not in this repo:
it is this machine's taste, not part of the kit.
Two notes for anyone editing that file:
- Window rules are **v3** syntax here (`windowrule = match:class .*, float on`).
The v2 spelling (`float, class:^(chromium)$`) parses as an unknown field and
Hyprland draws the error *across the screen*, which lands in the recording.
Always check an edit with `Hyprland --verify-config --config <file>` — it does
not need a display, so it costs seconds instead of a rebuild.
- `hyprctl` **cannot connect** to this session: the compositor names its IPC
socket `.socket.so`/`.socket.s` instead of `.socket.sock`, so nothing using the
documented path works. Do not plan on `hyprctl` for window control — measure
what you need from the frames instead.
### Posting the result to a chat
Xiph/Blossom media stores reject files that carry metadata. `encode()` in the
script already strips it with `-map_metadata -1 -fflags +bitexact -flags:v
+bitexact`; without those flags a Buzz relay answers
`422 media contains metadata or a non-canonical metadata channel` and the upload
fails. Uploading from the command line, with the same script-encoded file:
```bash
buzz upload file --file agent-eyes.mp4 # -> {"url": ".../media/<sha256>.mp4"}
buzz media get -o check.mp4 "<url>" # fetch it back and compare sha256
```
`buzz upload file` takes `--file <path>`, not a positional argument, and exits
**0** with a JSON error on stdout when the relay refuses the type — read the
output, never the exit status.
+179
View File
@@ -0,0 +1,179 @@
#!/usr/bin/env bash
# agent-eyes — work the headless Hyprland session from the outside.
#
# agent-eyes status is the session up, can I reach it?
# agent-eyes shot [--out FILE] grab one frame of the screen
# agent-eyes run [--] CMD [ARGS...] launch a GUI app INTO the session
# agent-eyes record --seconds N [--out F] film whatever happens
#
# The session itself is a system service (hyprland-headless +
# hyprland-headless-stream, see the flake's NixOS module). This script is the
# *consumer* side of docs/CONTRACT.md: it never starts a compositor.
#
# Access: the runtime directory is 0750 and the socket 0660, both group
# `hhstream`. If this process cannot read them it re-execs itself under
# `sg hhstream -c ...`, which is the only reliable fix for a login that
# predates the group membership.
set -euo pipefail
SOCKET=${AGENT_EYES_SOCKET:-/run/hyprland-headless-stream/stream.sock}
RUNDIR=$(dirname "$SOCKET")
GROUP=${AGENT_EYES_GROUP:-hhstream}
CLIENT=${AGENT_EYES_CLIENT:-hhstream-client}
SHOT_DIR=${AGENT_EYES_SHOT_DIR:-/var/lib/hermes/public/agent-eyes}
die() { printf 'agent-eyes: %s\n' "$*" >&2; exit 1; }
have_group() { [ -x "$RUNDIR" ] && [ -r "$SOCKET" ]; }
# Only the setuid wrapper can switch groups, and PATH order decides which `sg`
# you get. A login shell has /run/wrappers/bin first and works; a cron job's
# PATH reaches the non-setuid copy in the system profile instead, which dies
# with "setgroups: Operation not permitted" — and, with stderr going to
# /dev/null, that looks like "chromium exited immediately". So pick the wrapper
# deliberately rather than trusting PATH.
sg_bin() {
local c
for c in /run/wrappers/bin/sg "$(command -v sg 2>/dev/null || true)"; do
[ -n "$c" ] && [ -x "$c" ] && [ -u "$c" ] && { printf '%s\n' "$c"; return 0; }
done
return 1
}
# One self re-exec, so every branch below may assume group access.
if ! have_group; then
SG=$(sg_bin) || die "cannot join group $GROUP: no setuid 'sg' found (looked in /run/wrappers/bin)"
exec "$SG" "$GROUP" -c "$(printf '%q ' "$0" "$@")"
fi
wayland_socket() {
local s
for s in "$RUNDIR"/wayland-?; do
[ -S "$s" ] && { printf '%s\n' "$s"; return 0; }
done
return 1
}
usage() { sed -n '2,9p' "$0" | sed 's/^# \{0,1\}//'; }
cmd_status() {
local svc
echo "services:"
for svc in hyprland-headless hyprland-headless-stream; do
printf ' %-28s %s\n' "$svc" "$(systemctl is-active "$svc" 2>&1 || true)"
done
printf 'socket: %s\n' "$SOCKET"
printf 'access: %s\n' "$([ -r "$SOCKET" ] && echo "readable (group $GROUP)" || echo DENIED)"
local wl
wl=$(wayland_socket) || wl=""
printf 'wayland: %s\n' "${wl:-none found}"
printf 'probe: '
if "$CLIENT" --socket "$SOCKET" --count 1 --timeout 5 --out /dev/null 2>&1 | tail -1; then
:
else
echo "no frame — compositor or capture tool is down"
return 1
fi
}
cmd_shot() {
local out="$SHOT_DIR/latest.jpg"
while [ $# -gt 0 ]; do
case $1 in
--out) out=${2:?--out needs a path}; shift 2 ;;
*) die "shot: unknown argument '$1'" ;;
esac
done
mkdir -p "$(dirname "$out")"
"$CLIENT" --socket "$SOCKET" --count 1 --out "$out" >/dev/null
printf '%s\n' "$out"
}
# Launch a GUI client into the running compositor. Window rules, wallpaper and
# gaps are whatever the session's hyprland.conf says; a new window just appears.
cmd_run() {
[ $# -gt 0 ] || die "run: needs a command (e.g. agent-eyes run chromium)"
[ "${1:-}" = "--" ] && shift
local wl
wl=$(wayland_socket) || die "run: no wayland socket in $RUNDIR"
XDG_RUNTIME_DIR="$RUNDIR" \
WAYLAND_DISPLAY="${wl##*/}" \
XDG_SESSION_TYPE=wayland \
exec "$@"
}
cmd_record() {
local seconds=10 out=agent-eyes.mp4 keep=0
while [ $# -gt 0 ]; do
case $1 in
--seconds) seconds=${2:?--seconds needs a number}; shift 2 ;;
--out) out=${2:?--out needs a path}; shift 2 ;;
--keep) keep=1; shift ;;
*) die "record: unknown argument '$1'" ;;
esac
done
command -v ffmpeg >/dev/null || die "record: ffmpeg not on PATH"
local dir
dir=$(mktemp -d "${TMPDIR:-/tmp}/agent-eyes-rec.XXXXXX")
local got
got=$(python3 - "$SOCKET" "$dir" "$seconds" <<'PY'
import json, os, socket, sys, time
sock_path, outdir, seconds = sys.argv[1], sys.argv[2], float(sys.argv[3])
s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
s.connect(sock_path)
f = s.makefile("rb")
hello = json.loads(f.readline())
ext = "jpg" if hello["format"].endswith("jpeg") else "png"
def read_exactly(n):
buf = b""
while len(buf) < n:
chunk = f.read(n - len(buf))
if not chunk:
raise EOFError
buf += chunk
return buf
start = time.monotonic()
n = 0
while True:
line = f.readline()
if not line:
break
size = int(line.strip())
payload = read_exactly(size)
n += 1
with open(os.path.join(outdir, "frame-%05d.%s" % (n, ext)), "wb") as out:
out.write(payload)
if time.monotonic() - start >= seconds:
break
elapsed = time.monotonic() - start
# The encoder gets the *measured* rate, so the video plays at true speed even
# though a software-rendered VM tops out well under the advertised fps.
print("%d %d %.3f %s" % (n, round(n / elapsed) if elapsed else 1, elapsed, ext))
PY
)
set -- $got
local frames=$1 fps=$2 elapsed=$3 ext=$4
[ "$frames" -gt 0 ] || die "record: read no frames in ${seconds}s"
ffmpeg -hide_banner -loglevel error -y \
-framerate "$fps" -i "$dir/frame-%05d.$ext" \
-c:v libx264 -pix_fmt yuv420p -vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" \
-map_metadata -1 -fflags +bitexact -flags:v +bitexact \
"$out"
printf '%s (%s frames in %ss -> %s fps)\n' "$out" "$frames" "$elapsed" "$fps"
[ "$keep" = 1 ] || rm -rf "$dir"
}
[ $# -gt 0 ] || { usage; exit 1; }
sub=$1; shift
case $sub in
status|st) cmd_status "$@" ;;
shot) cmd_shot "$@" ;;
run) cmd_run "$@" ;;
record|rec) cmd_record "$@" ;;
help|-h|--help) usage ;;
*) die "unknown subcommand '$sub' (try: status, shot, run, record, help)" ;;
esac
+505
View File
@@ -0,0 +1,505 @@
#!/usr/bin/env python3
"""Agent Eyes demo: put a browser on the headless screen, drive it, film it.
This is the *consumer* side of `docs/CONTRACT.md`: it reads frames off the
published socket (not from a screenshot API of its own) while a real browser,
running inside the headless Hyprland session, does something visible. The output
is an MP4 you can post anywhere.
Why it exists: "Agent Eyes" is this machine's ability to look at a screen it can
drive. Everything else in this repo is the plumbing that makes the screen exist;
this is the smallest honest demonstration that a human can watch.
nix develop
sg hhstream -c 'python3 examples/agent-eyes-demo.py --query "Ari Sosnovsky"'
Preconditions, all of them checked below rather than assumed:
* the headless session is up and `stream.sock` exists (services
`hyprland-headless` + `hyprland-headless-stream`);
* you can read it — that means being in the stream's group, hence `sg
hhstream` for an already-logged-in shell (`id` will not show it until you
log in again);
* nothing else is on that screen. Hyprland tiles, so a leftover window would
take half the frame. `--clear-screen` kills leftovers (needs sudo);
* `chromium` and `ffmpeg` on PATH — both come from the dev shell.
It talks to the browser over the DevTools protocol rather than synthesising
input events, so the typing is real text entry into the real page and the script
can *read back* what happened. That read-back is the point: the demo asserts the
result page loaded instead of just hoping the frames look right.
"""
from __future__ import annotations
import argparse
import json
import os
import pathlib
import shutil
import socket
import subprocess
import sys
import threading
import time
import urllib.error
import urllib.request
from typing import NoReturn
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent.parent / "src"))
from hhstream import protocol # noqa: E402 (path set above)
DEFAULT_SOCKET = "/run/hyprland-headless-stream/stream.sock"
DEFAULT_RUNTIME_DIR = "/run/hyprland-headless-stream"
def log(message: str) -> None:
print(f"[eyes] {message}", flush=True)
def fail(message: str) -> NoReturn:
print(f"[eyes] {message}", file=sys.stderr, flush=True)
raise SystemExit(1)
# --------------------------------------------------------------------------
# The session: where the screen is, and how to reach it
# --------------------------------------------------------------------------
def wayland_display(runtime_dir: str) -> str:
"""The compositor names its own socket (`wayland-N`), so ask the filesystem.
Hardcoding `wayland-0` is the trap the module documents at length: a
preserved runtime directory can leave `wayland-0` behind from a crashed run
and push the live session to `wayland-1`.
"""
for entry in sorted(pathlib.Path(runtime_dir).glob("wayland-*")):
if entry.is_socket() and not entry.name.endswith(".lock"):
return entry.name
fail(f"no Wayland socket in {runtime_dir} — is hyprland-headless running?")
def session_env(runtime_dir: str) -> dict:
display = wayland_display(runtime_dir)
log(f"session: XDG_RUNTIME_DIR={runtime_dir} WAYLAND_DISPLAY={display}")
return {
**os.environ,
"XDG_RUNTIME_DIR": runtime_dir,
"WAYLAND_DISPLAY": display,
"XDG_SESSION_TYPE": "wayland",
}
def check_readable(socket_path: str) -> None:
if not pathlib.Path(socket_path).exists():
fail(f"{socket_path} does not exist — is hyprland-headless-stream running?")
if not os.access(socket_path, os.R_OK | os.W_OK):
fail(
f"cannot use {socket_path} as {os.environ.get('USER', '?')}. The socket "
"is `0660 hhstream:hhstream` in a `0750` directory, so your user must "
"be in the `hhstream` group — run this under `sg hhstream -c '...'` if "
"the group was added since you logged in."
)
# --------------------------------------------------------------------------
# The browser
# --------------------------------------------------------------------------
def clear_screen(runtime_dir: str) -> None:
"""Kill leftover browsers on the shared screen (needs sudo)."""
log("--clear-screen: killing leftover chromium processes")
# The bracket keeps the pattern from matching `pkill`'s own command line —
# `pkill -f chromium` kills the shell that ran it, which is a confusing way
# to lose a session.
result = subprocess.run(
["sudo", "-n", "pkill", "-f", "[c]hromium"],
stdout=subprocess.DEVNULL, stderr=subprocess.PIPE,
)
if result.returncode not in (0, 1): # 1 == "nothing matched", which is fine
fail(f"could not clear the screen: {result.stderr.decode().strip()}")
time.sleep(1.0)
def _browser_argv(env: dict, *, url: str, port: int, home: pathlib.Path,
fullscreen: bool, unsandboxed: bool) -> list[str]:
profile = home / "profile"
return [
"chromium",
"--ozone-platform=wayland",
# The sandbox works on this machine, so it is on by default. When it
# does not, the fallback needs `--test-type` as well: without it Chromium
# paints "you are using an unsupported command-line flag: --no-sandbox"
# across the top of the page, and that banner would be in every frame of
# the recording.
*(["--no-sandbox", "--test-type"] if unsandboxed else []),
"--disable-dev-shm-usage",
# Windowed by default: the session's wallpaper, gaps and window borders
# are the point of filming a *desktop*, and a kiosk window hides all of
# them. `--fullscreen` brings back the full-bleed browser.
*(["--kiosk"] if fullscreen else []),
"--window-size=1040,560",
f"--user-data-dir={profile}",
"--no-first-run",
"--no-default-browser-check",
f"--remote-debugging-port={port}",
# Keep the debug port on loopback: nothing outside this machine has any
# business driving the browser.
"--remote-debugging-address=127.0.0.1",
url,
]
def start_browser(env: dict, *, url: str, port: int, home: pathlib.Path,
fullscreen: bool = False) -> subprocess.Popen:
home.mkdir(parents=True, exist_ok=True)
env = {**env, "HOME": str(home)}
argv = _browser_argv(env, url=url, port=port, home=home,
fullscreen=fullscreen, unsandboxed=False)
log(f"browser: starting chromium (debug port {port}, "
f"{'kiosk' if fullscreen else 'windowed'}) at {url}")
proc = subprocess.Popen(argv, env=env, stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL)
# A sandboxed Chromium can die on launch (no user namespaces, no setuid
# helper). Give it a moment, and fall back rather than filming an empty
# desktop and calling it a demo.
time.sleep(2.5)
if proc.poll() is not None:
log("browser: sandboxed launch exited; retrying with --no-sandbox")
argv = _browser_argv(env, url=url, port=port, home=home,
fullscreen=fullscreen, unsandboxed=True)
proc = subprocess.Popen(argv, env=env, stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL)
return proc
def cdp_target(port: int, timeout: float = 30.0) -> str:
"""Wait for the DevTools endpoint and return the page target's ws URL."""
deadline = time.monotonic() + timeout
last = ""
while time.monotonic() < deadline:
try:
with urllib.request.urlopen(f"http://127.0.0.1:{port}/json/list",
timeout=2) as response:
targets = json.load(response)
pages = [t for t in targets if t.get("type") == "page"]
if pages:
return pages[0]["webSocketDebuggerUrl"]
last = "endpoint up, no page target yet"
except (urllib.error.URLError, OSError, ValueError) as exc:
last = str(exc)
time.sleep(0.5)
fail(f"DevTools endpoint on port {port} never came up ({last})")
class DevTools:
"""A very small CDP client: send a method, wait for its reply."""
def __init__(self, ws_url: str) -> None:
import websocket # type: ignore[import-not-found] # from the dev shell
self._ws = websocket.create_connection(ws_url, timeout=20,
suppress_origin=True)
self._next_id = 0
def call(self, method: str, **params):
self._next_id += 1
ident = self._next_id
self._ws.send(json.dumps({"id": ident, "method": method, "params": params}))
while True:
message = json.loads(self._ws.recv())
if message.get("id") == ident:
if "error" in message:
raise RuntimeError(f"{method}: {message['error']}")
return message.get("result", {})
def evaluate(self, expression: str):
result = self.call("Runtime.evaluate", expression=expression,
returnByValue=True, awaitPromise=True)
return result.get("result", {}).get("value")
def close(self) -> None:
self._ws.close()
def is_walled(devtools: DevTools) -> bool:
"""Google's bot wall, however it presents itself.
Both checks are needed. The URL is the obvious tell (`/sorry/index?...`),
but the redirect can arrive *after* the results page has already painted —
so a guard that only runs right after the search passes and then films a
CAPTCHA for sixteen seconds. Hence: check the URL, check the page text, and
call this again once the recording has finished.
"""
url = devtools.evaluate("location.href") or ""
if "/sorry/" in url:
return True
text = devtools.evaluate(
"(() => { const b = document.body; return b ? b.innerText.slice(0, 600) : ''; })()") or ""
return "unusual traffic" in text.lower()
def type_query(devtools: DevTools, query: str, per_char: float = 0.11) -> None:
"""Focus the search box and type into it, one character at a time.
`Input.insertText` goes through the renderer's normal text-entry path, so the
page sees exactly what a keyboard would produce (its autocomplete included)
and the characters appear on screen as they are sent — which is the whole
reason this is worth filming.
"""
focused = devtools.evaluate(
"(() => { const el = document.querySelector('textarea[name=q], input[name=q]');"
" if (!el) return false; el.focus(); return document.activeElement === el; })()"
)
if not focused:
fail("could not focus the search box — did the page load?")
log(f"typing {query!r}")
for char in query:
devtools.call("Input.insertText", text=char)
time.sleep(per_char)
def submit_and_wait(devtools: DevTools, timeout: float = 15.0) -> str:
"""Press Enter, then confirm the results page actually loaded."""
before = devtools.evaluate("location.href")
devtools.call("Input.dispatchKeyEvent", type="rawKeyDown", key="Enter",
code="Enter", windowsVirtualKeyCode=13, nativeVirtualKeyCode=13)
devtools.call("Input.dispatchKeyEvent", type="keyUp", key="Enter",
code="Enter", windowsVirtualKeyCode=13, nativeVirtualKeyCode=13)
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
time.sleep(0.25)
now = devtools.evaluate("location.href")
if now and now != before and "search" in now:
return now
# Enter did not navigate: submit the form directly rather than leaving a
# video of a page that never went anywhere.
log("Enter did not navigate; submitting the form directly")
devtools.evaluate(
"(() => { const f = document.querySelector('form');"
" if (f) { f.submit(); return true; } return false; })()"
)
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
time.sleep(0.25)
now = devtools.evaluate("location.href")
if now and now != before:
return now
fail("the search never navigated — nothing to film")
# --------------------------------------------------------------------------
# Recording: read the real frames off the socket
# --------------------------------------------------------------------------
class Recorder(threading.Thread):
"""Pull frames off the published socket and write them as a numbered series.
Deliberately not `hhstream-client`: that reads a fixed count and exits, and
here the frames have to arrive *while* something else happens.
"""
def __init__(self, socket_path: str, out_dir: pathlib.Path, target_fps: float) -> None:
super().__init__(daemon=True)
self.socket_path = socket_path
self.out_dir = out_dir
self.target_fps = target_fps
self.hello: dict = {}
# Frame extension follows the Hello, not an assumption: the stream can
# legitimately be JPEG, and the caller must not have to know which.
self.ext = "png"
self.count = 0
self.error: str | None = None
# NOT `_stop` / `_started`: `threading.Thread` owns both names. It sets
# `_started` itself as the thread boots — *before* `run()` — so a
# `wait()` on it returns instantly and every attribute `run()` is
# supposed to fill in is still empty. It also calls `self._stop()` on
# the way out, which an Event cannot answer to.
self._stop_event = threading.Event()
self._ready = threading.Event()
def run(self) -> None:
try:
conn = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
conn.settimeout(10.0)
conn.connect(self.socket_path)
except OSError as exc:
self.error = str(exc)
self._ready.set()
return
try:
with conn.makefile("rb") as stream:
# CONTRACT: the Hello can take a moment (it is written on the
# first successful capture), so give it the socket timeout.
self.hello = protocol.read_hello(stream)
self.ext = {"image/png": "png", "image/jpeg": "jpg"}.get(
str(self.hello.get("format", "image/png")), "png")
server_fps = float(self.hello.get("fps") or 30)
step = max(1, round(server_fps / self.target_fps))
self._ready.set()
index = 0
while not self._stop_event.is_set():
payload = protocol.read_frame(stream)
if payload is None:
break
index += 1
if (index - 1) % step:
continue # decimate to the target rate
self.count += 1
(self.out_dir / f"frame-{self.count:05d}.{self.ext}").write_bytes(payload)
except (OSError, EOFError, ValueError) as exc:
self.error = str(exc)
finally:
conn.close()
self._ready.set()
def stop(self) -> None:
self._stop_event.set()
self.join(timeout=10)
def encode(frames: pathlib.Path, fps: float, out: pathlib.Path, ext: str = "png") -> None:
argv = [
"ffmpeg", "-hide_banner", "-loglevel", "error", "-y",
"-framerate", f"{fps:.4f}",
"-i", str(frames / f"frame-%05d.{ext}"),
"-c:v", "libx264", "-preset", "veryfast", "-crf", "23",
# yuv420p + faststart: what every player and every chat client wants.
"-pix_fmt", "yuv420p", "-movflags", "+faststart",
# Strip every metadata tag. Xiph/Blossom media stores reject files that
# carry them — Buzz's does, with `422 media contains metadata or a
# non-canonical metadata channel`, and the only thing to fix is ffmpeg's
# own `encoder`/`handler_name` tags. `bitexact` keeps them from being
# written back in.
"-map_metadata", "-1", "-fflags", "+bitexact", "-flags:v", "+bitexact",
"-an",
str(out),
]
log(f"encoding {out.name}")
result = subprocess.run(argv, stderr=subprocess.PIPE)
if result.returncode != 0:
fail(f"ffmpeg failed: {result.stderr.decode().strip()}")
def main() -> int:
parser = argparse.ArgumentParser(
prog="agent-eyes-demo.py",
description="Drive a browser on the headless screen and film it.",
)
parser.add_argument("--query", default="Ari Sosnovsky",
help="what to type into the search box")
parser.add_argument("--output", default="agent-eyes.mp4", metavar="PATH")
parser.add_argument("--socket", default=DEFAULT_SOCKET)
parser.add_argument("--runtime-dir", default=DEFAULT_RUNTIME_DIR)
parser.add_argument("--seconds", type=float, default=14.0,
help="how long to record (default: %(default)s)")
parser.add_argument("--capture-fps", type=float, default=15.0,
help="frames per second to keep (default: %(default)s)")
parser.add_argument("--debug-port", type=int, default=9222)
parser.add_argument("--fullscreen", action="store_true",
help="kiosk/full-bleed browser instead of a windowed one "
"(hides the session's wallpaper and window borders)")
parser.add_argument("--work-dir", type=pathlib.Path, default=None,
help="where frames land (default: a temp dir)")
parser.add_argument("--clear-screen", action="store_true",
help="kill leftover browsers first (needs passwordless sudo)")
parser.add_argument("--keep-work", action="store_true",
help="keep the frames and the browser profile")
args = parser.parse_args()
for tool in ("chromium", "ffmpeg"):
if not shutil.which(tool):
fail(f"{tool} is not on PATH — run this inside `nix develop`")
check_readable(args.socket)
env = session_env(args.runtime_dir)
if args.clear_screen:
clear_screen(args.runtime_dir)
work = args.work_dir or pathlib.Path(
os.environ.get("TMPDIR", "/tmp")) / "agent-eyes-demo"
frames = work / "frames"
if frames.exists():
shutil.rmtree(frames)
frames.mkdir(parents=True)
out = pathlib.Path(args.output).resolve()
out.parent.mkdir(parents=True, exist_ok=True)
browser = start_browser(env, url="https://www.google.com",
port=args.debug_port, home=work / "home",
fullscreen=args.fullscreen)
devtools = None
recorder = None
try:
devtools = DevTools(cdp_target(args.debug_port))
recorder = Recorder(args.socket, frames, args.capture_fps)
recorder.start()
if not recorder._ready.wait(timeout=15):
fail("the frame socket never answered")
if recorder.error:
fail(f"recording failed: {recorder.error}")
log(f"recording {args.seconds:.0f}s at {args.capture_fps:.0f}fps "
f"(server says {recorder.hello.get('fps')}fps, "
f"{recorder.hello.get('width')}x{recorder.hello.get('height')})")
began = time.monotonic()
time.sleep(1.5) # let the page settle on screen
type_query(devtools, args.query)
landed = submit_and_wait(devtools)
log(f"landed on {landed}")
# The wall can arrive as a redirect *after* the results paint, so give
# it a beat and then look again rather than trusting the first URL.
time.sleep(1.5)
if is_walled(devtools):
# A video that looks like a working demo and is not one is worse
# than no video.
fail(
"Google served its 'unusual traffic' wall instead of results "
"(rate-limited IP, not a broken browser). Wait a few minutes and "
"run again; the first search from a fresh profile usually passes."
)
devtools.evaluate("window.scrollBy(0, 320)") # show actual results
# Record for the requested wall-clock duration, so the video's length is
# what was asked for regardless of how long the page took to load.
remaining = args.seconds - (time.monotonic() - began)
if remaining > 0:
time.sleep(remaining)
# Final gate, after the whole recording: if the page ended up on the
# wall at any point, nothing is encoded.
if is_walled(devtools):
fail("Google walled the search during recording — no video written")
finally:
if recorder is not None:
recorder.stop()
if devtools is not None:
devtools.close()
browser.terminate()
try:
browser.wait(timeout=10)
except subprocess.TimeoutExpired:
browser.kill()
if recorder is None or recorder.count < 2:
fail("fewer than two frames arrived — nothing to encode")
fps = recorder.count / args.seconds
log(f"captured {recorder.count} frames ({fps:.2f} fps effective)")
encode(frames, fps, out, recorder.ext)
size = out.stat().st_size
log(f"wrote {out} ({size / 1024:.0f} KiB, {recorder.count} frames)")
if not args.keep_work:
shutil.rmtree(work, ignore_errors=True)
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,51 @@
# hhstreamd configuration — the app allowlist.
#
# This is the main security control (DESIGN.md §8 item 1). Without it,
# `hhstream session start /bin/sh` would make every other control decorative.
#
# The daemon reads this at startup from --config / $HHSTREAM_CONFIG, defaulting
# to /etc/hhstream/config.toml. A missing file is not an error: the allowlist is
# then empty and every `start` is refused with exit 6, naming the file it looked
# for. A misconfigured deployment must not look like a crash.
[defaults]
# The maximum number of live sessions. With no cgroup caps available (see
# DESIGN.md §3.3), this is the primary containment control against a looping
# agent spawning browsers, so it is a real setting and not a tuning knob.
max_sessions = 4
# Default idle timeout, in seconds. A session with no `touch` for this long is
# reaped. `touch` is the only thing that resets the clock.
idle_timeout_s = 10800 # 3h
# The largest idle timeout a caller may ask for.
max_idle_timeout_s = 86400 # 24h
# How long `kill` waits between SIGTERM and SIGKILL, by default.
grace_s = 10
# How long `start` will wait for readiness before returning exit 7. The session
# is left alive either way.
wait_ready_s = 15
# ---------------------------------------------------------------------------
# Apps. The key is the name a caller passes to `start`; it is never a path.
# ---------------------------------------------------------------------------
[apps.chromium]
# Absolute path, always. Resolution against PATH is refused.
exec = "/run/current-system/sw/bin/chromium"
# Caller-supplied arguments must each match one of these fnmatch patterns.
# An app with no `args_allow` accepts no caller arguments at all.
args_allow = ["--new-window", "--app=*", "--user-data-dir=*"]
[apps.foot]
exec = "/run/current-system/sw/bin/foot"
[apps.blender]
exec = "/run/current-system/sw/bin/blender"
# Fixed arguments, always prepended, never supplied by the caller.
args = ["--factory-startup"]
args_allow = ["*.blend"]
# A per-app default idle timeout, overridden by --idle-timeout.
idle_timeout_s = 7200
+11
View File
@@ -39,6 +39,17 @@
hyprland
grim
ffmpeg
# The client side of the contract (hhstream-client), so a shell can
# read the stream without a `nix build` first.
(callPackage ./nix/package.nix { })
# `websocket-client` drives Chromium over the DevTools protocol, which
# is how the demo types into the page and reads the result back.
python3.pkgs.websocket-client
# Something to *put on* the headless screen. Chromium is the only
# browser here that both runs on Wayland and can be driven over the
# DevTools protocol, which is how examples/agent-eyes-demo.py types
# into a page. Kept in the dev shell rather than installed globally.
chromium
];
shellHook = ''
export PYTHONPATH="$PWD/src''${PYTHONPATH:+:$PYTHONPATH}"
+14 -3
View File
@@ -9,14 +9,25 @@
extraConfig,
}: let
# `monitor = <name>, <width>x<height>@<refresh>, auto, <scale>` -- "auto"
# lets the compositor lay the virtual outputs out left to right.
# lets the compositor lay the outputs out left to right.
#
# The name is a *rule selector*, not an output the compositor will create: it
# is matched as a regex against the names the backend hands up (`eDP-1`,
# `Virtual-1`, ...). Hyprland never invents an output to satisfy a rule, so a
# name that no connector has is a rule that is silently ignored.
#
# An empty name is the wildcard form -- the same shape as Hyprland's own
# shipped `monitor=,preferred,auto,1` -- and applies to every output the
# machine actually has. That is what a session which does not know the host's
# connector names wants, and it is the module's default.
monitorLine = o: "monitor = ${o.name}, ${toString o.width}x${toString o.height}@${toString o.refresh}, auto, ${toString o.scale}";
in ''
# Generated by services.hyprland-headless-stream. Do not edit.
#
# There is no seat, no libinput device, no cursor and no human here: the only
# consumer of this session is the frame server. Everything below either
# creates the virtual outputs or switches off something that assumes a user.
# consumer of this session is the frame server. Everything below either sets
# the mode of the outputs this machine has or switches off something that
# assumes a user.
${lib.concatMapStringsSep "\n" monitorLine outputs}
+365 -39
View File
@@ -20,31 +20,86 @@ let
socketDir = builtins.dirOf cfg.socketPath;
# Hyprland refuses `--socket` unless it is paired with `--wayland-fd` (that
# pair is the socket-handover path), so the name is not ours to choose: the
# compositor calls wl_display_add_socket_auto, which takes the first free
# `wayland-N` starting at 0. The runtime directory is private to this
# session, so that is always `wayland-0`.
waylandDisplay = "wayland-0";
# pair is the socket-handover path), so the socket name is not ours to choose:
# the compositor calls wl_display_add_socket_auto, which takes the first free
# `wayland-N` starting at 0.
#
# N is therefore not fixed. The runtime directory is preserved across restarts
# (`RuntimeDirectoryPreserve=yes`), so a crashed compositor leaves its
# `wayland-0` behind and the next start binds `wayland-1` — measured in the VM
# test, where the frame server then waited forever for a socket that would
# never exist. Neither unit may hardcode the name:
#
# * the compositor clears leftover sockets first (the directory is private to
# this session, so nothing else can own them) — that alone makes it
# `wayland-0` again, but the units below do not depend on that;
# * the server discovers whichever socket is actually there.
clearStaleSockets = pkgs.writeShellScript "hhstream-clear-stale-sockets" ''
for stale in "$XDG_RUNTIME_DIR"/wayland-* "$XDG_RUNTIME_DIR"/wayland-*.lock; do
if [ -e "$stale" ]; then
${lib.getExe' pkgs.coreutils "rm"} -f "$stale"
fi
done
'';
# Exit 0 once *any* Wayland socket is bound, and print its name so callers that
# want it do not have to glob again.
findSession = pkgs.writeShellScript "hhstream-find-session" ''
for _ in $(${lib.getExe' pkgs.coreutils "seq"} 1 300); do
for candidate in "$XDG_RUNTIME_DIR"/wayland-*; do
if [ -S "$candidate" ]; then
echo "''${candidate##*/}"
exit 0
fi
done
${lib.getExe' pkgs.coreutils "sleep"} 0.1
done
echo "no Wayland socket in $XDG_RUNTIME_DIR after 30s; is the compositor up?" >&2
exit 1
'';
# The server needs WAYLAND_DISPLAY pointing at the socket the compositor
# actually bound. This cannot be an `Environment=` line, because the name is
# only known once the compositor is up — systemd's ExecStartPre cannot export
# into the main process.
execServer = pkgs.writeShellScript "hhstream-exec-server" ''
for candidate in "$XDG_RUNTIME_DIR"/wayland-*; do
if [ -S "$candidate" ]; then
export WAYLAND_DISPLAY="''${candidate##*/}"
break
fi
done
exec ${lib.getExe' cfg.package "hhstream-server"} "$@"
'';
hyprlandConf = pkgs.writeText "hyprland.conf" (import ./hyprland-conf.nix {
inherit lib;
inherit (cfg) outputs extraConfig;
});
# `BindsTo=` already stops the frame server when the compositor goes away,
# but the compositor is only *ready* once its Wayland socket is bound. The
# compositor unit is `Type=notify`, so this is a second line of defence for
# compositor builds that do not sd_notify.
waitForSession = pkgs.writeShellScript "hhstream-wait-for-session" ''
socket="$XDG_RUNTIME_DIR/$WAYLAND_DISPLAY"
for _ in $(${lib.getExe' pkgs.coreutils "seq"} 1 300); do
if [ -S "$socket" ]; then
exit 0
fi
${lib.getExe' pkgs.coreutils "sleep"} 0.1
done
echo "no Wayland socket at $socket after 30s; is the compositor up?" >&2
exit 1
# `BindsTo=` already stops the frame server when the compositor goes away, but
# the compositor is only *ready* once its Wayland socket is bound. The
# compositor unit is `Type=notify`, so `findSession` above is a second line of
# defence for compositor builds that do not sd_notify, and it is why this is
# not a plain `After=` ordering.
# The session broker's app allowlist, in the daemon's own config format. The
# daemon treats a missing file as an empty allowlist and refuses every `start`
# with the path it looked for, so generating this is what makes
# `sessions.enable = true` useful rather than decorative.
sessionsConfig = pkgs.writeText "hhstream-sessions.toml" ''
[defaults]
max_sessions = ${toString cfg.sessions.maxSessions}
idle_timeout_s = ${toString cfg.sessions.idleTimeout}
${lib.concatStringsSep "\n" (
lib.mapAttrsToList (name: app: ''
[apps.${name}]
exec = ${builtins.toJSON (toString app.exec)}
args_allow = [${
lib.concatStringsSep ", " (map (arg: builtins.toJSON arg) app.argsAllow)
}]
'') cfg.sessions.apps
)}
'';
serverArgs =
@@ -136,7 +191,16 @@ in {
outputs = lib.mkOption {
type = lib.types.listOf (lib.types.submodule {
options = {
name = lib.mkOption {type = lib.types.str;};
name = lib.mkOption {
type = lib.types.str;
default = "";
description = ''
Regex matched against the names of the outputs this machine has
(`eDP-1`, `Virtual-1`, ...). Empty is Hyprland's wildcard form and
matches every output; a name no connector has is a rule that is
silently ignored.
'';
};
width = lib.mkOption {
type = lib.types.ints.positive;
default = 1280;
@@ -160,14 +224,26 @@ in {
});
default = [
{
name = "HEADLESS-1";
name = "";
width = 1280;
height = 720;
refresh = 30;
scale = 1.0;
}
];
description = "Virtual monitors the compositor should create.";
description = ''
Monitor rules applied to the outputs the machine has. Each entry becomes
one `monitor = <name>, <width>x<height>@<refresh>, auto, <scale>` line.
These are *not* outputs the compositor creates: aquamarine's headless
backend has no allocator of its own and rides on the DRM backend, so a
session started by this module always has the GPU's own connectors and
whatever names the host gives them. The default therefore uses the empty
name -- Hyprland's wildcard, as in its own shipped
`monitor=,preferred,auto,1` -- so that `enable = true` alone gives a
session at a known size on any machine. Naming a connector that does not
exist is not an error; the rule is simply never applied.
'';
};
extraConfig = lib.mkOption {
@@ -193,6 +269,139 @@ in {
default = ["grim" "-t" "png" "-"];
description = "Command that writes one frame to stdout; overridable for tests.";
};
sessions = {
enable = lib.mkOption {
type = lib.types.bool;
default = false;
description = ''
Run the session broker (`hhstreamd`) that launches GUI applications
into this compositor as *tracked sessions*: started by name, kept
alive by an explicit `touch`, killed by name, and reaped when they go
quiet for {option}`sessions.idleTimeout`.
It is off by default because it is a different capability from
streaming frames: a frame server can be read by anyone in the group,
while this one starts processes on the machine.
Its CLI is `hhstream session …` on the same binary as the frame
commands; the daemon itself is the unit `hhstream-sessions.service`.
'';
};
user = lib.mkOption {
type = lib.types.str;
default = config.services.hyprland-headless-stream.user;
defaultText = lib.literalExpression "services.hyprland-headless-stream.user";
description = ''
The account the broker runs as, and the account every launched
application runs as. It must already exist: the module does not create
it, because its whole point is usually to be a real person's account
whose home directory and files the applications should see.
Set it to the account that will actually drive the CLI. The broker is
granted {option}`sessions.group` as a supplementary group so the
applications it launches can reach the compositor's socket.
'';
};
group = lib.mkOption {
type = lib.types.str;
default = config.services.hyprland-headless-stream.group;
defaultText = lib.literalExpression "services.hyprland-headless-stream.group";
description = ''
The group that owns the compositor's runtime directory. The broker gets
it as a supplementary group, and passes it to every application it
launches — without it a launched GUI client cannot open the Wayland
socket and dies at startup.
'';
};
idleTimeout = lib.mkOption {
type = lib.types.ints.positive;
default = 10800;
description = ''
Seconds a session may go without a `touch` before the broker kills it
and everything it spawned. The clock lives in the broker, not in the
CLI, so it keeps running after the CLI exits.
Only `touch` resets it: `list`, `status`, `logs` and `gc` deliberately
do not, or a monitoring loop would keep abandoned sessions alive
forever.
'';
};
maxSessions = lib.mkOption {
type = lib.types.ints.positive;
default = 4;
description = ''
Hard cap on live sessions. Without cgroup limits this is the main
containment control against a looping caller spawning applications
until the machine gives up.
'';
};
apps = lib.mkOption {
type = lib.types.attrsOf (lib.types.submodule {
options = {
exec = lib.mkOption {
type = lib.types.path;
description = "Absolute path to the executable. Never a shell command.";
};
argsAllow = lib.mkOption {
type = lib.types.listOf lib.types.str;
default = [];
description = ''
Arguments a caller may pass to this application. Anything else is
refused, so `exec` must be able to do its job with these and
nothing more. A trailing `*` matches a prefix
(`--user-data-dir=*`).
'';
};
};
});
default = {};
description = ''
The allowlist: which applications may be started as a session, and with
which arguments. This is the primary security control —
without it, `session start /bin/sh` would make every other control
decorative. An application that is not listed here cannot be started
at all.
'';
example = lib.literalExpression ''
{
chromium.exec = "''${pkgs.chromium}/bin/chromium";
chromium.argsAllow = [ "--new-window" "--user-data-dir=*" "https://*" ];
}
'';
};
configFile = lib.mkOption {
type = lib.types.nullOr lib.types.path;
default = null;
description = ''
Use this TOML allowlist instead of the one generated from
{option}`sessions.apps`, for configurations the option shape cannot
express.
'';
};
stateDir = lib.mkOption {
type = lib.types.str;
default = "/var/lib/hhstream";
description = ''
Where the broker keeps what outlives a broker restart: the live-record
mirrors it adopts on the way back up, tombstones of ended sessions, and
`events.jsonl`, the audit trail of every accepted command.
Point this at durable storage. The default is a system path, which on a
machine whose root is rebuilt from scratch — the usual shape of a
declaratively-managed server — means the audit trail dies with the
next reinstall. On such a machine this belongs on the same disk as the
rest of the state that cannot be rebuilt.
'';
};
};
};
config = lib.mkIf cfg.enable {
@@ -205,6 +414,16 @@ in {
assertion = lib.hasPrefix "/" cfg.socketPath;
message = "services.hyprland-headless-stream.socketPath must be an absolute path.";
}
{
assertion = !cfg.sessions.enable || lib.hasAttr cfg.sessions.user config.users.users;
message = ''
services.hyprland-headless-stream.sessions.user = "${cfg.sessions.user}" is not a
declared user. The module deliberately does not create it: launched applications
run as that account, so it is normally a real person's account whose home
directory and files they should see. Declare it (or point the option at an
existing account).
'';
}
];
# The flake's own build, as docs/CONTRACT.md says; `mkDefault` so a
@@ -212,6 +431,33 @@ in {
services.hyprland-headless-stream.package =
lib.mkDefault (pkgs.callPackage ./package.nix {});
# The broker's state directory and the moved socket directory are two
# different reasons to write a tmpfiles entry, and this option takes one
# definition — so they are merged here rather than assigned twice (which
# does not evaluate at all).
systemd.tmpfiles.settings = lib.mkMerge [
# The broker keeps the records it adopts on the way back up, its
# tombstones, and `events.jsonl`, the audit trail of every accepted
# command. It is declared here rather than left to the daemon so its owner
# and mode are declarative and survive a restart of the unit.
(lib.mkIf cfg.sessions.enable {
"10-hhstream-sessions".${cfg.sessions.stateDir}.d = {
user = cfg.sessions.user;
group = cfg.sessions.group;
mode = "0700";
};
})
# Only needed when the socket was moved out of the runtime directory that
# `RuntimeDirectory=` already creates for us.
(lib.mkIf (socketDir != runtimeDir) {
"10-hyprland-headless-stream".${socketDir}.d = {
user = cfg.user;
group = cfg.group;
mode = "0750";
};
})
];
users.users.${cfg.user} = {
isSystemUser = true;
group = cfg.group;
@@ -219,16 +465,6 @@ in {
};
users.groups.${cfg.group} = {};
# Only needed when the socket was moved out of the runtime directory that
# `RuntimeDirectory=` already creates for us.
systemd.tmpfiles.settings = lib.mkIf (socketDir != runtimeDir) {
"10-hyprland-headless-stream".${socketDir}.d = {
user = cfg.user;
group = cfg.group;
mode = "0750";
};
};
systemd.services.hyprland-headless = {
description = "Headless Hyprland session";
documentation = ["file://${hyprlandConf}"];
@@ -236,8 +472,8 @@ in {
# No WAYLAND_DISPLAY and no DISPLAY here on purpose: with either set the
# compositor would pick its nested backend and try to connect to a parent
# session. With no seat and no DRM device it falls back to headless,
# which is exactly what the virtual outputs in hyprland.conf want.
# session. Without them it drives DRM directly, and the monitor rules in
# hyprland.conf set the mode of whatever connectors it finds there.
environment =
sessionEnvironment
// {
@@ -250,6 +486,26 @@ in {
WLR_BACKENDS = "headless";
WLR_LIBINPUT_NO_DEVICES = "1";
WLR_RENDERER_ALLOW_SOFTWARE = "1";
# Ask libseat's embedded seatd for a seat with no VT.
#
# This unit is a system service, not a login, so libseat finds no
# /run/seatd.sock and no logind session and falls back to the seatd it
# links in -- whose seat is VT-bound by default. That seatd then opens
# /dev/tty0, which `cfg.user` cannot, so its client never goes active
# and every device open is refused:
#
# [libseat] Could not open target tty: Permission denied
# [libseat] Could not open VT for client
# libseat: Couldn't open device at /dev/dri/card0
# drm: Found no gpus to use, cannot continue
#
# and with the DRM backend gone there is no allocator, so
# `CBackend::create()` aborts -- the headless backend takes its
# allocator from the DRM one. SEATD_VTBOUND=0 (seatd/server.c) is the
# truth about this session on any host: there is no terminal to switch
# away from.
SEATD_VTBOUND = "0";
};
serviceConfig =
@@ -260,6 +516,10 @@ in {
Type = "notify";
NotifyAccess = "all";
TimeoutStartSec = 60;
# A preserved runtime directory can still hold the previous run's
# sockets, and that would push the new session to `wayland-1` (and
# leave the server waiting on a dead one).
ExecStartPre = "${clearStaleSockets}";
ExecStart = lib.escapeShellArgs [
(lib.getExe cfg.hyprlandPackage)
"--config"
@@ -280,7 +540,8 @@ in {
environment =
sessionEnvironment
// {
WAYLAND_DISPLAY = waylandDisplay;
# WAYLAND_DISPLAY is set by `execServer` at start, because the socket
# name is only known once the compositor has bound it.
PYTHONUNBUFFERED = "1";
};
@@ -288,11 +549,76 @@ in {
commonServiceConfig
// {
Type = "simple";
ExecStartPre = "${waitForSession}";
ExecStart = lib.escapeShellArgs (
[(lib.getExe' cfg.package "hhstream-server")] ++ serverArgs
);
ExecStartPre = "${findSession} >/dev/null";
ExecStart = lib.escapeShellArgs ([execServer] ++ serverArgs);
};
};
# ── the session broker ─────────────────────────────────────────────
#
# A different capability from the frame server, and a different account: this
# one starts processes, so it runs as the operator's own user and everything
# it launches runs as that user too.
systemd.services.hhstream-sessions = lib.mkIf cfg.sessions.enable {
description = "Session broker for the headless Hyprland desktop";
wantedBy = ["multi-user.target"];
after = ["hyprland-headless.service"];
bindsTo = ["hyprland-headless.service"];
environment = {
PYTHONUNBUFFERED = "1";
};
serviceConfig = {
Type = "simple";
User = cfg.sessions.user;
# The broker does not need the group itself -- but the applications it
# launches do, and they inherit its supplementary groups. Without it a
# GUI client cannot open the compositor's socket (0770, this group) and
# dies with "Failed to connect to Wayland display".
SupplementaryGroups = [cfg.sessions.group];
RuntimeDirectory = "hhstream";
RuntimeDirectoryMode = "0750";
WorkingDirectory = "/run/hhstream";
UMask = "0007";
Restart = "on-failure";
RestartSec = 1;
NoNewPrivileges = true;
# Sessions must outlive the broker, because a broker restart is an
# ordinary event (an upgrade, a crash, `systemctl restart`) and the whole
# point of the registry is to adopt what it finds afterwards.
#
# systemd's default is KillMode=control-group, which SIGTERMs every
# process in the unit's cgroup when the unit stops -- i.e. it kills the
# applications from outside, undoing the broker's own care (it spawns
# them with a plain Popen precisely so its own death does not reach them).
# KillMode=process signals only the broker; the applications are left
# running and are adopted on the way back up.
KillMode = "process";
ExecStart = lib.escapeShellArgs [
(lib.getExe' cfg.package "hhstreamd")
"--socket"
"/run/hhstream/control.sock"
"--runtime-dir"
"/run/hhstream"
"--state-dir"
cfg.sessions.stateDir
"--config"
(toString (
if cfg.sessions.configFile != null then cfg.sessions.configFile else sessionsConfig
))
# Where the compositor's own session lives. The broker discovers the
# `wayland-N` socket in there and hands XDG_RUNTIME_DIR / WAYLAND_DISPLAY
# to every application it launches; the daemon was built to fail loudly
# if this directory holds no usable socket.
"--compositor-runtime-dir"
runtimeDir
];
};
};
};
}
+4
View File
@@ -12,6 +12,10 @@ dependencies = []
[project.scripts]
hhstream-server = "hhstream.server:main"
hhstream-client = "hhstream.client:main"
# The one front door: frames/launch/record verbs plus the `session` namespace.
hhstream = "hhstream.cli:main"
# The session daemon. A separate process on purpose: it outlives every CLI call.
hhstreamd = "hhstreamd.__main__:main"
[tool.setuptools.packages.find]
where = ["src"]
+345
View File
@@ -0,0 +1,345 @@
"""``hhstream`` — the one front door to this machine's headless Hyprland session.
Four verbs that work whether or not the session daemon is running, and one
namespace that needs it:
hhstream status is the session up, and can I reach it?
hhstream shot one frame of the screen
hhstream run -- CMD... launch a GUI app into the session (unmanaged)
hhstream record film the screen for N seconds
hhstream session ... managed application sessions (start/list/touch/kill)
``run`` and ``session start`` both put an application on the screen; they are
deliberately different. ``run`` is the primitive: it execs and forgets, nothing
tracks the process, and stopping it is the caller's problem. ``session start``
registers it, reports its pid, keeps its own clock, can be killed by name, and is
reaped if it goes quiet for too long.
Access: the frame socket lives in a directory owned by the compositor's group
(mode 0750). If this process cannot read it, the CLI re-execs itself through the
*setuid* ``sg`` wrapper — deliberately, because a non-setuid ``sg`` earlier on
PATH fails with "setgroups: Operation not permitted", which looks like every
other failure.
"""
from __future__ import annotations
import argparse
import os
import pathlib
import shlex
import shutil
import subprocess
import sys
import time
from typing import Iterator
from . import client as frames
from . import protocol
#: The frame socket the compositor's publisher serves.
DEFAULT_FRAME_SOCKET = frames.DEFAULT_SOCKET
ENV_FRAME_SOCKET = "HHSTREAM_FRAME_SOCKET"
#: The group that owns the session's runtime directory.
DEFAULT_GROUP = "hhstream"
ENV_GROUP = "HHSTREAM_GROUP"
#: The units `status` reports on. Overridable so the CLI is not welded to this
#: machine's naming, but the defaults are the ones the module ships.
DEFAULT_UNITS = ("hyprland-headless.service", "hyprland-headless-stream.service")
ENV_UNITS = "HHSTREAM_UNITS"
FFMPEG = "ffmpeg"
# ── access ─────────────────────────────────────────────────────────────────
def setuid_sg() -> str | None:
"""The ``sg`` that can actually switch groups.
Two exist on a NixOS box: the setuid wrapper in /run/wrappers/bin and a
plain symlink to shadow's copy in the system profile. Only the wrapper
works; a login shell happens to find it first, a cron job does not.
"""
candidates = ["/run/wrappers/bin/sg", shutil.which("sg")]
for path in candidates:
if not path or not os.path.isfile(path) or not os.access(path, os.X_OK):
continue
if os.stat(path).st_mode & 0o4000:
return path
return None
def have_access(socket_path: str) -> bool:
directory = pathlib.Path(socket_path).parent
return os.access(directory, os.X_OK) and os.access(socket_path, os.R_OK)
def ensure_access(socket_path: str, group: str, argv: list[str]) -> None:
"""Re-exec through ``sg`` when the socket is out of reach. Returns if fine."""
if have_access(socket_path):
return
sg = setuid_sg()
if sg is None:
sys.exit(
f"hhstream: cannot reach {socket_path} and no setuid 'sg' was found "
f"(looked in /run/wrappers/bin) to join group {group!r}"
)
os.execv(sg, [sg, group, "-c", " ".join(shlex.quote(a) for a in argv)])
def wayland_socket(runtime_dir: str) -> pathlib.Path | None:
"""Whichever ``wayland-N`` the compositor bound — never assume ``-0``."""
root = pathlib.Path(runtime_dir)
for candidate in sorted(root.glob("wayland-*")):
if candidate.name.endswith(".lock"):
continue
if candidate.is_socket():
return candidate
return None
# ── helpers ────────────────────────────────────────────────────────────────
def read_frames(socket_path: str, *, count: int, timeout: float) -> tuple[dict, list[bytes]]:
return frames.read_stream(socket_path, count=count, timeout=timeout)
def frame_extension(hello: dict) -> str:
return "jpg" if str(hello.get("format", "")).endswith("jpeg") else "png"
def stream_frames(socket_path: str, seconds: float, timeout: float = 10.0) -> Iterator[bytes]:
"""Yield frames off one connection until `seconds` of wall time have passed.
One connection on purpose: the publisher drops frames for a slow reader
rather than queueing them, and reconnecting per frame would make us the slow
reader.
"""
import socket as _socket
conn = _socket.socket(_socket.AF_UNIX, _socket.SOCK_STREAM)
conn.settimeout(timeout)
try:
conn.connect(socket_path)
with conn.makefile("rb") as stream:
protocol.read_hello(stream)
started = time.monotonic()
while time.monotonic() - started < seconds:
payload = protocol.read_frame(stream)
if payload is None:
return
yield payload
finally:
conn.close()
def env_for_session(runtime_dir: str) -> dict[str, str]:
"""The environment a GUI client needs to attach to the running compositor."""
wayland = wayland_socket(runtime_dir)
if wayland is None:
sys.exit(f"hhstream: no wayland socket in {runtime_dir} — is the compositor up?")
return {
"XDG_RUNTIME_DIR": runtime_dir,
"WAYLAND_DISPLAY": wayland.name,
"XDG_SESSION_TYPE": "wayland",
}
# ── verbs ──────────────────────────────────────────────────────────────────
def cmd_status(args: argparse.Namespace) -> int:
units = [u for u in (os.environ.get(ENV_UNITS) or "").split(",") if u] or list(DEFAULT_UNITS)
print("services:")
for unit in units:
state = subprocess.run(
["systemctl", "is-active", unit], capture_output=True, text=True
).stdout.strip() or "unknown"
print(f" {unit:<34} {state}")
print(f"frame: {args.frame_socket}")
print(f"access: {'readable' if have_access(args.frame_socket) else 'DENIED'}")
runtime_dir = str(pathlib.Path(args.frame_socket).parent)
wayland = wayland_socket(runtime_dir)
print(f"wayland: {wayland if wayland else 'none found'}")
print("probe: ", end="")
try:
hello, payloads = read_frames(args.frame_socket, count=1, timeout=args.timeout)
except OSError as exc:
print(f"no frame — {exc}")
return 1
if not payloads:
print("no frame — the publisher closed without sending one")
return 1
print(f"{len(payloads[0])} bytes, {hello['width']}x{hello['height']} {hello['format']}")
return 0
def cmd_shot(args: argparse.Namespace) -> int:
hello, payloads = read_frames(args.frame_socket, count=1, timeout=args.timeout)
if not payloads:
print("hhstream: the publisher sent no frame", file=sys.stderr)
return 1
out = pathlib.Path(args.out or f"hhstream-frame.{frame_extension(hello)}")
out.parent.mkdir(parents=True, exist_ok=True)
out.write_bytes(payloads[0])
print(out)
return 0
def cmd_run(args: argparse.Namespace) -> int:
argv = list(args.argv or [])
if not argv:
print("hhstream: run needs a command, e.g. hhstream run -- myapp", file=sys.stderr)
return 2
env = dict(os.environ)
env.update(env_for_session(str(pathlib.Path(args.frame_socket).parent)))
os.execvpe(argv[0], argv, env)
return 0 # unreachable with execvpe
def cmd_record(args: argparse.Namespace) -> int:
if shutil.which(FFMPEG) is None:
print(f"hhstream: {FFMPEG} is not on PATH", file=sys.stderr)
return 1
out = pathlib.Path(args.out or "hhstream-recording.mp4")
work = pathlib.Path(args.work_dir) if args.work_dir else pathlib.Path(
os.environ.get("TMPDIR", "/tmp")
) / f"hhstream-rec-{os.getpid()}"
work.mkdir(parents=True, exist_ok=True)
hello, first = read_frames(args.frame_socket, count=1, timeout=args.timeout)
ext = frame_extension(hello)
if first:
(work / f"frame-{1:05d}.{ext}").write_bytes(first[0])
count = len(first)
started = time.monotonic()
for payload in stream_frames(args.frame_socket, max(0.0, args.seconds - 0.05)):
count += 1
(work / f"frame-{count:05d}.{ext}").write_bytes(payload)
elapsed = max(time.monotonic() - started, 0.001)
if count == 0:
print(f"hhstream: read no frames in {args.seconds}s", file=sys.stderr)
return 1
# Encode at the *measured* rate: a software-rendered VM tops out well under
# the advertised fps, so anything else would play back at the wrong speed.
fps = max(1, round(count / elapsed))
subprocess.run(
[
FFMPEG, "-hide_banner", "-loglevel", "error", "-y",
"-framerate", str(fps),
"-i", str(work / f"frame-%05d.{ext}"),
"-c:v", "libx264", "-pix_fmt", "yuv420p",
"-vf", "scale=trunc(iw/2)*2:trunc(ih/2)*2",
# Media stores reject files carrying metadata; strip it here so a
# recording is uploadable without a second pass.
"-map_metadata", "-1", "-fflags", "+bitexact", "-flags:v", "+bitexact",
str(out),
]
)
print(f"{out} ({count} frames in {elapsed:.2f}s -> {fps} fps)")
if not args.keep:
shutil.rmtree(work, ignore_errors=True)
return 0
# ── parser ─────────────────────────────────────────────────────────────────
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
prog="hhstream",
description=(
"Work this machine's headless Hyprland session: look at it, put an "
"application on it, manage that application as a session."
),
epilog=(
"'run' launches and forgets. 'session start' launches and tracks — "
"use it when something must be found, kept alive and killed later."
),
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument(
"--frame-socket",
default=os.environ.get(ENV_FRAME_SOCKET, DEFAULT_FRAME_SOCKET),
metavar="PATH",
help=f"the compositor's frame socket (default: ${ENV_FRAME_SOCKET} or "
f"{DEFAULT_FRAME_SOCKET})",
)
parser.add_argument(
"--timeout", type=float, default=10.0, metavar="SECONDS", help="per-read socket timeout"
)
parser.add_argument(
"--group",
default=os.environ.get(ENV_GROUP, DEFAULT_GROUP),
metavar="NAME",
help=f"group that owns the session's runtime directory (default: {DEFAULT_GROUP})",
)
verbs = parser.add_subparsers(dest="command", metavar="COMMAND")
verbs.add_parser("status", help="are the units up, and can this process reach the socket?")
shot = verbs.add_parser("shot", help="write one frame of the screen")
shot.add_argument("--out", metavar="FILE", help="where to write it (default: ./hhstream-frame.*)")
run = verbs.add_parser("run", help="launch a GUI app into the session (unmanaged)")
run.add_argument("argv", nargs="*", metavar="-- CMD...", help="the command, after a literal --")
record = verbs.add_parser("record", help="film the screen for N seconds")
record.add_argument("--seconds", type=float, default=10.0, metavar="N")
record.add_argument("--out", metavar="FILE", help="output mp4 (default: ./hhstream-recording.mp4)")
record.add_argument("--work-dir", metavar="DIR", help="where frames go (default: a temp dir)")
record.add_argument("--keep", action="store_true", help="keep the frames after encoding")
# The managed namespace. It brings its own --socket (the *control* socket,
# which is a different socket from the frame one) and its own exit codes.
from hhstream_session import register as register_session
register_session(verbs)
return parser
def main(argv: list[str] | None = None) -> int:
parser = build_parser()
args = parser.parse_args(argv)
if args.command is None:
parser.print_help(sys.stderr)
return 2
# Every verb except the session namespace speaks to the frame socket, so the
# group check happens here, once. The session handlers do their own.
if args.command in ("status", "shot", "run", "record"):
# Re-exec the ORIGINAL command line, not a reconstruction: `sys.executable`
# plus the arguments would drop this script's own path and hand Python a
# verb as a filename ("can't open file 'status'").
argv = list(getattr(sys, "orig_argv", None) or [sys.executable, *sys.argv])
ensure_access(args.frame_socket, args.group, argv)
handler = {
"status": cmd_status,
"shot": cmd_shot,
"run": cmd_run,
"record": cmd_record,
}.get(args.command)
if handler is not None:
return handler(args)
# session … — argparse already selected the handler when it parsed
return int(getattr(args, "func")(args))
if __name__ == "__main__": # pragma: no cover
raise SystemExit(main())
+508
View File
@@ -0,0 +1,508 @@
"""``hhstream session …`` — the CLI handlers, as a drop-in for someone else's CLI.
The integration contract is one function::
from hhstream_session import register
register(subparsers) # subparsers from argparse's add_subparsers()
...
args = parser.parse_args()
raise SystemExit(args.func(args))
``register`` adds a single ``session`` subcommand with its own verb tree, sets
``func`` (and ``hhstream_session_handler``, for hosts that dispatch on their own
attribute) to a callable taking the parsed namespace and returning an **exit
code from the frozen table** in ``DESIGN.md`` §1.6. The registrant needs to
know nothing about the daemon beyond the socket path, which comes from
``--socket`` or ``$HHSTREAM_SOCKET``.
Nothing here is imported until a ``session`` verb actually runs: the module
imports only :mod:`hhstreamd.protocol` (framing and exit codes) and the
standard library, so a host CLI pays no startup cost and gains no new failure
mode by registering it.
**The one thing to read before using this:** ``touch`` is the only mark of
activity that exists. ``list``, ``status``, ``logs`` and ``gc`` deliberately do
not reset a session's idle clock, so an agent watching a long-running app must
call ``touch`` or the session will be reaped on schedule.
"""
from __future__ import annotations
import argparse
import sys
import time
from typing import Any, Callable
from hhstreamd.clock import format_duration, parse_duration
from hhstreamd.protocol import ExitCode
from . import format as fmt
from .client import DEFAULT_SOCKET, ENV_SOCKET, ClientError, call, reply_exit_code
__all__ = ["register", "build_parser", "dispatch", "main", "ExitCode"]
#: Printed under `session --help` and under `start --help`. It is here, in two
#: places, because it is the single fact an agent gets wrong.
ACTIVITY_NOTE = (
"ACTIVITY: 'touch' is the ONLY thing that resets a session's idle clock. "
"'list', 'status', 'logs' and 'gc' do NOT count — if polling kept sessions "
"alive, an abandoned session would never be reaped. An agent that leaves an "
"app running must call 'hhstream session touch <id>' before the idle timeout "
"expires."
)
_FOLLOW_INTERVAL_S = 0.5
# --------------------------------------------------------------------------
# registration
# --------------------------------------------------------------------------
def register(subparsers: Any, *, name: str = "session") -> argparse.ArgumentParser:
"""Add the ``session`` namespace to an existing argparse subparser set.
Returns the namespace's parser, so a host that wants to add to it can.
"""
parser = subparsers.add_parser(
name,
help="manage tracked application sessions (needs the hhstream session daemon)",
description="Start, inspect, keep alive and kill tracked application sessions.",
epilog=ACTIVITY_NOTE,
formatter_class=argparse.RawDescriptionHelpFormatter,
)
_build_namespace(parser)
return parser
def _socket_parent() -> argparse.ArgumentParser:
"""Shared ``--socket`` so it may be given before *or* after the verb.
``SUPPRESS`` matters: without it the verb-level default would overwrite a
value given at the namespace level.
"""
common = argparse.ArgumentParser(add_help=False)
common.add_argument(
"--socket",
default=argparse.SUPPRESS,
metavar="PATH",
help=f"control socket (default: ${ENV_SOCKET} or {DEFAULT_SOCKET})",
)
return common
def _build_namespace(parser: argparse.ArgumentParser) -> None:
common = _socket_parent()
parser.add_argument(
"--socket",
default=None,
metavar="PATH",
help=f"control socket (default: ${ENV_SOCKET} or {DEFAULT_SOCKET})",
)
# `hhstream session` with no verb lists, because that is what everyone
# types first.
parser.set_defaults(func=_cmd_list, hhstream_session_handler=_cmd_list, all=False, json=False)
verbs = parser.add_subparsers(dest="session_verb", metavar="VERB")
def add(name: str, handler: Callable[[argparse.Namespace], int], **kwargs: Any):
sub = verbs.add_parser(name, parents=[common], **kwargs)
sub.set_defaults(func=handler, hhstream_session_handler=handler)
sub.add_argument("--json", action="store_true", help="emit the protocol object on stdout")
return sub
# ---- start -------------------------------------------------------
start = add(
"start",
_cmd_start,
help="launch an allowlisted application as a tracked session",
description=(
"Launch an allowlisted application. Prints the bare session id on "
"stdout and human-readable detail on stderr, so $(hhstream session "
"start foo) is the id and nothing else.\n\n"
"Everything after a literal -- is passed to the application as an "
"argv list: 'session start chromium -- --new-window URL'."
),
epilog=ACTIVITY_NOTE,
formatter_class=argparse.RawDescriptionHelpFormatter,
)
start.add_argument("app", help="an app name from the daemon's allowlist, never a path")
start.add_argument(
"argv",
nargs="*",
metavar="-- ARGV...",
help="arguments for the application, after a literal --. Passed as an "
"argv list; there is never a shell.",
)
start.add_argument(
"--idle-timeout",
type=_duration,
metavar="DURATION",
help="reap the session after this long with no 'touch' (e.g. 30s, 5m, 3h)",
)
start.add_argument(
"--wait-ready",
type=_duration,
metavar="DURATION",
help="how long to wait for the app to signal readiness",
)
home = start.add_mutually_exclusive_group()
home.add_argument(
"--home", metavar="PATH", help="run the app with this HOME instead of the inherited one"
)
home.add_argument(
"--ephemeral-home",
action="store_true",
help="give the session a private HOME and delete it when the session ends "
"(opt-in; by default the app inherits the real home directory)",
)
start.add_argument(
"--request-id",
metavar="ID",
help="deduplication token: retrying a start with the same id returns the "
"original reply instead of launching a second application",
)
# ---- list --------------------------------------------------------
lst = add("list", _cmd_list, help="list sessions (does NOT count as activity)")
lst.add_argument("--all", action="store_true", help="include tombstones of ended sessions")
# ---- status ------------------------------------------------------
status = add("status", _cmd_status, help="show one session (does NOT count as activity)")
status.add_argument("id", help="session id")
status.add_argument(
"--events", type=int, default=20, metavar="N", help="recent audit events to include"
)
# ---- touch -------------------------------------------------------
add(
"touch",
_cmd_touch,
help="reset the idle clock — THE ONLY MARK OF ACTIVITY",
description=(
"Reset the session's idle clock. Injects nothing, changes nothing "
"else, and is the only verb that counts as activity: an agent "
"observing a long-running application has no other way to say "
"'still working'."
),
epilog=ACTIVITY_NOTE,
formatter_class=argparse.RawDescriptionHelpFormatter,
).add_argument("id", help="session id")
# ---- kill --------------------------------------------------------
kill = add("kill", _cmd_kill, help="terminate a session and everything it spawned")
kill.add_argument("id", help="session id")
kill.add_argument(
"--grace",
type=_duration,
metavar="DURATION",
help="how long to wait after the first signal before SIGKILL (default: the daemon's)",
)
kill.add_argument(
"--signal", default="TERM", metavar="SIG", help="first signal to send (default: TERM)"
)
kill.add_argument("--request-id", metavar="ID", help="deduplication token")
# ---- gc ----------------------------------------------------------
add(
"gc",
_cmd_gc,
help="prune tombstones and reconcile the registry with /proc "
"(does NOT count as activity)",
).add_argument("--request-id", metavar="ID", help="deduplication token")
# ---- adopt -------------------------------------------------------
adopt = add("adopt", _cmd_adopt, help="bring an already-running process under management")
adopt.add_argument("--pid", type=int, required=True, help="pid to adopt")
adopt.add_argument("--app", help="name to record (default: the executable's basename)")
adopt.add_argument("--idle-timeout", type=_duration, metavar="DURATION")
adopt.add_argument("--request-id", metavar="ID", help="deduplication token")
# ---- logs --------------------------------------------------------
logs = add(
"logs",
_cmd_logs,
help="the application's captured stdout/stderr (does NOT count as activity)",
)
logs.add_argument("id", help="session id")
logs.add_argument("--lines", type=int, default=100, metavar="N", help="tail this many lines")
logs.add_argument(
"--follow", "-f", action="store_true", help="keep printing new output until interrupted"
)
def _duration(text: str) -> float:
try:
return parse_duration(text)
except ValueError as exc:
raise argparse.ArgumentTypeError(str(exc)) from exc
# --------------------------------------------------------------------------
# handlers
# --------------------------------------------------------------------------
def _socket_of(args: argparse.Namespace) -> str | None:
return getattr(args, "socket", None)
def _request(args: argparse.Namespace, op: str, **fields: Any) -> dict[str, Any]:
return call(op, socket_path=_socket_of(args), **fields)
def _finish(args: argparse.Namespace, reply: dict[str, Any], render) -> int:
"""Print a reply and return its exit code.
Errors always go to stderr and always carry the machine field, whatever the
output mode, because that is what an agent branches on.
"""
code = reply_exit_code(reply)
if getattr(args, "json", False):
fmt.dump_json(reply, stream=sys.stdout if reply.get("ok") else sys.stderr)
elif reply.get("ok"):
text = render(reply)
if text:
print(text)
else:
print(fmt.render_error(reply), file=sys.stderr)
return int(code)
def _guard(handler: Callable[[argparse.Namespace], int]) -> Callable[[argparse.Namespace], int]:
def wrapper(args: argparse.Namespace) -> int:
try:
return handler(args)
except ClientError as exc:
if getattr(args, "json", False):
fmt.dump_json(
{"ok": False, "error": exc.error, "reason": exc.reason, "hint": exc.hint},
stream=sys.stderr,
)
else:
print(f"error: {exc.reason}", file=sys.stderr)
if exc.hint:
print(f"hint: {exc.hint}", file=sys.stderr)
return int(exc.exit_code)
except KeyboardInterrupt:
return int(ExitCode.USAGE)
wrapper.__name__ = handler.__name__
wrapper.__doc__ = handler.__doc__
return wrapper
@_guard
def _cmd_start(args: argparse.Namespace) -> int:
# argparse has already removed the separating `--`, so what is left is the
# application's own argv, as a list. It is never joined into a string.
argv = list(args.argv or [])
reply = _request(
args,
"start",
app=args.app,
argv=argv,
idle_timeout_s=args.idle_timeout,
wait_ready_s=args.wait_ready,
home=args.home,
ephemeral_home=args.ephemeral_home or None,
request_id=args.request_id,
)
if not reply.get("ok"):
# A failed start may still have created a session (it exited at once, or
# it is slow to become ready). Print the id so it can be inspected.
if reply.get("id") and not args.json:
print(reply["id"])
return _finish(args, reply, lambda r: "")
session = reply.get("session") or {}
if args.json:
fmt.dump_json(session)
return int(ExitCode.OK)
session_id = reply.get("id") or session.get("id") or ""
# stdout is the bare id and nothing else; detail goes to stderr.
print(session_id)
print(
f"started {session.get('app')} → {session_id} "
f"(pid {session.get('pid')}, pgid {session.get('pgid')}, sid {session.get('sid')})",
file=sys.stderr,
)
print(
f"state {session.get('state')} via {session.get('ready_signal')}; "
f"idle timeout {_fmt_timeout(session.get('idle_timeout_s'))}",
file=sys.stderr,
)
print(
f"keep it alive with: hhstream session touch {session_id} "
"(the only mark of activity; list/status/logs/gc do not count)",
file=sys.stderr,
)
return int(ExitCode.OK)
def _fmt_timeout(seconds: Any) -> str:
return "disabled" if not seconds else format_duration(seconds)
@_guard
def _cmd_list(args: argparse.Namespace) -> int:
reply = _request(args, "list", all=bool(getattr(args, "all", False)) or None)
return _finish(args, reply, lambda r: fmt.render_list(r.get("sessions") or []))
@_guard
def _cmd_status(args: argparse.Namespace) -> int:
reply = _request(args, "status", id=args.id, events=args.events)
return _finish(args, reply, lambda r: fmt.render_status(r.get("session") or {}))
@_guard
def _cmd_touch(args: argparse.Namespace) -> int:
reply = _request(args, "touch", id=args.id)
def render(r: dict[str, Any]) -> str:
session = r.get("session") or {}
return (
f"{session.get('id')}: idle clock reset; "
f"expires in {format_duration(session.get('expires_in_s'))}"
)
return _finish(args, reply, render)
@_guard
def _cmd_kill(args: argparse.Namespace) -> int:
reply = _request(
args,
"kill",
id=args.id,
grace_s=args.grace,
signal=args.signal,
request_id=args.request_id,
)
def render(r: dict[str, Any]) -> str:
"""Say what the daemon actually did, not merely that it finished.
The old rendering could only distinguish "signalled" from "not
signalled", and printed *the process was already gone* for both the
routine case and the case where the daemon refused to signal a live
application (BUG-02). A refusal is now an error reply and never reaches
here; what is left is reported with the signal and the process group it
went to, so "killed" cannot be read as more than it says.
"""
session = r.get("session") or {}
bits = [f"{session.get('id')}: {session.get('state')}"]
sent = session.get("signal_sent")
if sent:
bits.append(f"sent {sent} to process group {session.get('pgid')}")
if session.get("escalated_to_sigkill"):
bits.append("escalated to SIGKILL")
stragglers = session.get("stragglers") or 0
if stragglers:
bits.append(f"{stragglers} straggler(s) swept by session id")
if session.get("signalled") is False and not sent:
bits.append("nothing signalled (the process was already gone)")
survivors = session.get("survivors") or []
if survivors:
# The loudest thing this command can say: the kill did not work.
bits.append(
f"WARNING: {len(survivors)} process(es) SURVIVED "
f"({', '.join(str(p) for p in survivors)})"
)
return "; ".join(bits)
return _finish(args, reply, render)
@_guard
def _cmd_gc(args: argparse.Namespace) -> int:
reply = _request(args, "gc", request_id=args.request_id)
return _finish(args, reply, fmt.render_gc)
@_guard
def _cmd_adopt(args: argparse.Namespace) -> int:
reply = _request(
args,
"adopt",
pid=args.pid,
app=args.app,
idle_timeout_s=args.idle_timeout,
request_id=args.request_id,
)
if reply.get("ok") and not args.json:
print(reply.get("id", ""))
session = reply.get("session") or {}
print(
f"adopted pid {session.get('pid')} as {reply['id']}; "
"exit code will be unavailable (not a child of the daemon)",
file=sys.stderr,
)
return int(ExitCode.OK)
return _finish(args, reply, lambda r: "")
@_guard
def _cmd_logs(args: argparse.Namespace) -> int:
reply = _request(args, "logs", id=args.id, lines=args.lines)
if not reply.get("ok") or not args.follow:
return _finish(args, reply, lambda r: "\n".join(r.get("lines") or []))
# --follow polls from the returned offset. The daemon holds no streaming
# connection open, so a follower that dies costs the daemon nothing.
for line in reply.get("lines") or []:
print(line)
offset = reply.get("offset", 0)
try:
while True:
if reply.get("eof"):
break
time.sleep(_FOLLOW_INTERVAL_S)
reply = _request(args, "logs", id=args.id, offset=offset)
if not reply.get("ok"):
return _finish(args, reply, lambda r: "")
for line in reply.get("lines") or []:
print(line)
offset = reply.get("offset", offset)
except KeyboardInterrupt:
pass
return int(ExitCode.OK)
# --------------------------------------------------------------------------
# standalone use
# --------------------------------------------------------------------------
def build_parser(prog: str = "hhstream") -> argparse.ArgumentParser:
"""A host CLI stand-in, so the handlers can be driven without one."""
parser = argparse.ArgumentParser(prog=prog)
subparsers = parser.add_subparsers(dest="command", metavar="COMMAND")
register(subparsers)
return parser
def dispatch(args: argparse.Namespace) -> int:
"""Run the handler argparse selected. The host may call this or ``args.func``."""
handler = getattr(args, "func", None)
if handler is None:
return int(ExitCode.USAGE)
return int(handler(args))
def main(argv: list[str] | None = None) -> int:
parser = build_parser()
args = parser.parse_args(argv)
if getattr(args, "command", None) is None:
parser.print_help(sys.stderr)
return int(ExitCode.USAGE)
return dispatch(args)
if __name__ == "__main__": # pragma: no cover
raise SystemExit(main())
+9
View File
@@ -0,0 +1,9 @@
"""``python -m hhstream_session`` — drive the handlers without a host CLI.
The real integration is :func:`hhstream_session.register`; this exists so the
verbs can be exercised (and tested) standalone.
"""
from . import main
raise SystemExit(main())
+109
View File
@@ -0,0 +1,109 @@
"""The control-socket client (DESIGN.md §1.6, §10).
Blocking, stdlib sockets, one request per connection. **One connect attempt,
then exit 5** — a CLI that silently retries hides a down daemon, and the hint
tells the operator where to look.
"""
from __future__ import annotations
import os
import socket
from typing import Any
from hhstreamd import protocol
from hhstreamd.protocol import ExitCode, exit_code_for
DEFAULT_SOCKET = "/run/hhstream/control.sock"
ENV_SOCKET = "HHSTREAM_SOCKET"
#: How long to wait for the daemon to answer. The daemon's own per-request
#: deadline is shorter for every verb except `kill` and `start`, which are
#: allowed their grace and readiness budgets, so this is a backstop.
DEFAULT_TIMEOUT_S = 60.0
class ClientError(Exception):
"""A failure that never reached the daemon's dispatch."""
def __init__(self, error: str, reason: str, hint: str | None = None) -> None:
super().__init__(reason)
self.error = error
self.reason = reason
self.hint = hint
@property
def exit_code(self) -> ExitCode:
return exit_code_for(self.error)
def default_socket_path() -> str:
return os.environ.get(ENV_SOCKET, DEFAULT_SOCKET)
def call(
op: str,
*,
socket_path: str | None = None,
timeout: float = DEFAULT_TIMEOUT_S,
**fields: Any,
) -> dict[str, Any]:
"""Send one request, return the decoded reply.
Raises :class:`ClientError` with ``daemon_unreachable`` (exit 5) if the
socket cannot be reached, and ``timeout`` (exit 7) if the daemon accepts the
request but never answers.
"""
path = socket_path or default_socket_path()
message = protocol.request(op, **fields)
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
sock.settimeout(timeout)
try:
try:
sock.connect(path)
except (FileNotFoundError, ConnectionRefusedError, PermissionError, OSError) as exc:
raise ClientError(
"daemon_unreachable",
f"cannot reach the hhstream session daemon at {path}: {exc}",
hint="is hhstreamd running? (it is an ordinary process; check "
"however you supervise it)",
) from exc
try:
sock.sendall(protocol.encode(message))
buffer = bytearray()
while b"\n" not in buffer:
chunk = sock.recv(65536)
if not chunk:
break
buffer.extend(chunk)
if len(buffer) > protocol.MAX_LINE_BYTES:
raise ClientError("bad_request", "reply exceeds the frame limit")
except TimeoutError as exc:
raise ClientError(
"timeout", f"the daemon did not answer {op!r} within {timeout:g}s"
) from exc
except OSError as exc:
raise ClientError("daemon_unreachable", f"control socket error: {exc}") from exc
finally:
sock.close()
if not buffer.strip():
raise ClientError("daemon_unreachable", "the daemon closed the connection without replying")
try:
return protocol.decode(bytes(buffer))
except protocol.ProtocolError as exc:
raise ClientError(exc.error, f"malformed reply from the daemon: {exc.reason}") from exc
def reply_exit_code(reply: dict[str, Any]) -> ExitCode:
"""The frozen exit code for a decoded reply.
A reply that is not ``ok`` never yields 0, even if it carries no error name
the table knows: reporting success for an unexplained failure is the one
mistake an agent cannot recover from.
"""
if reply.get("ok"):
return ExitCode.OK
return exit_code_for(reply.get("error") or "bad_state")
+155
View File
@@ -0,0 +1,155 @@
"""Rendering for the human views.
``--json`` output is the protocol's own objects, unmodified: a flag that
reshapes the data would mean two contracts to keep in step. The tables below
are for people.
"""
from __future__ import annotations
import json
import sys
from typing import Any
from hhstreamd.clock import format_duration
DASH = "—"
def dump_json(obj: Any, stream=None) -> None:
print(json.dumps(obj, indent=2, sort_keys=True), file=stream or sys.stdout)
def _cell(value: Any) -> str:
return DASH if value is None else str(value)
def render_table(rows: list[list[str]], headers: list[str]) -> str:
widths = [len(h) for h in headers]
for row in rows:
for i, cell in enumerate(row):
widths[i] = max(widths[i], len(cell))
out = [" ".join(h.ljust(widths[i]) for i, h in enumerate(headers)).rstrip()]
for row in rows:
out.append(" ".join(cell.ljust(widths[i]) for i, cell in enumerate(row)).rstrip())
return "\n".join(out)
def render_list(sessions: list[dict[str, Any]]) -> str:
"""The default view.
``EXPIRES IN`` is on it rather than behind a flag because it is the number
an agent actually needs.
There is **no ``FG`` column** (DESIGN.md §1.4, amended): foreground-ness
needs compositor IPC, which this milestone does not have, so the column was
``?`` on every row of every listing — a header and six columns of width
paying for no information, and a reader who has to learn that ``?`` means
*unknown* rather than *no*. ``foreground`` stays in the record and in
``status``, where it is one line among many and can become real without
changing the shape of this table.
"""
if not sessions:
return "no sessions"
rows = []
for s in sessions:
live = s.get("live")
if live:
idle = format_duration(s.get("idle_s"))
expires = format_duration(s.get("expires_in_s"))
if s.get("expires_in_s") is None:
expires = "never"
else:
idle = DASH
left = s.get("tombstone_expires_in_s")
expires = f"(tombstone, {format_duration(left)} left)" if left is not None else DASH
rows.append(
[
_cell(s.get("id")),
_cell(s.get("app")),
_cell(s.get("state")),
_cell(s.get("pid")) if live else DASH,
idle,
expires,
]
)
return render_table(rows, ["ID", "APP", "STATE", "PID", "IDLE", "EXPIRES IN"])
def render_status(session: dict[str, Any]) -> str:
live = session.get("live")
lines = [
f"{session.get('id')} ({session.get('app')})",
f" state {session.get('state')}"
+ (f" [{session.get('last_error')}]" if session.get("last_error") else ""),
f" pid {_cell(session.get('pid'))} pgid {_cell(session.get('pgid'))}"
f" sid {_cell(session.get('sid'))}",
f" argv {' '.join(session.get('argv') or []) or DASH}",
f" home {_cell(session.get('home'))}",
f" created {_cell(session.get('created_at'))}",
f" last activity {_cell(session.get('last_activity_at'))}"
f" ({session.get('command_count', 0)} command(s) counted)",
f" idle {format_duration(session.get('idle_s'))}"
f" of {format_duration(session.get('idle_timeout_s'))}",
f" expires in {format_duration(session.get('expires_in_s'))}"
if session.get("expires_in_s") is not None
else " expires in never (idle timeout disabled)",
f" ready signal {_cell(session.get('ready_signal'))}",
f" adopted {session.get('adopted')}",
]
if not live:
lines.append(
f" exit code {_cell(session.get('exit_code'))}"
f" detected by {_cell(session.get('exit_detected_by'))}"
)
if session.get("exit_detected_by") in ("pidfd", "proc_poll", "proc_check"):
lines.append(" exit code unavailable (adopted process)")
compositor = session.get("compositor") or {}
lines.append(
f" window {_cell(compositor.get('window'))}"
f" workspace {_cell(compositor.get('workspace'))}"
f" foreground {_cell(compositor.get('foreground'))}"
)
if compositor.get("note"):
lines.append(f" {compositor['note']}")
events = session.get("events") or []
if events:
lines.append(" recent events")
for event in events[-20:]:
lines.append(f" {event.get('ts')} {event.get('event')}")
return "\n".join(lines)
def render_gc(result: dict[str, Any]) -> str:
lines = [
f"pruned {result.get('pruned', 0)} tombstone(s), "
f"reconciled {len(result.get('reconciled') or [])} session(s)",
f"{result.get('live', 0)} live, {result.get('tombstones', 0)} tombstone(s) remain",
]
for item in result.get("reconciled") or []:
lines.append(f" {item['id']}: {item['why']}")
unresolved = result.get("unresolved") or []
if unresolved:
# Not a routine exit: the daemon could not establish whether these pids
# are still their sessions', so it signalled nothing. Said loudly
# because the application may well still be running (BUG-02 item 5).
lines.append(
"UNRESOLVED — identity could not be established, nothing was signalled:"
)
for item in unresolved:
mismatched = ", ".join(item.get("mismatched") or []) or "no usable evidence"
lines.append(f" {item['id']}: {item['why']} ({mismatched})")
orphans = result.get("orphans") or []
if orphans:
lines.append("orphaned processes (escaped the process group AND the session id):")
for item in orphans:
lines.append(f" {item['id']} sid {item['sid']}: pids {item['pids']}")
return "\n".join(lines)
def render_error(reply: dict[str, Any]) -> str:
parts = [f"error: {reply.get('reason') or reply.get('error')}"]
if reply.get("hint"):
parts.append(f"hint: {reply['hint']}")
return "\n".join(parts)
+8
View File
@@ -0,0 +1,8 @@
"""hhstreamd — the hhstream session daemon (M0).
Deliberately import-light: the CLI handler package imports
:mod:`hhstreamd.protocol` for the wire format and the exit-code table, and
must not pay for the daemon's dependencies to do it.
"""
__version__ = "0.1.0"
+253
View File
@@ -0,0 +1,253 @@
"""``python -m hhstreamd`` — the session daemon.
An ordinary long-running process. It is deliberately **agnostic about how it
is supervised**: no unit file, no ``sd_notify``, no ``systemctl``. It creates
the directories it needs if they are not already there, so it runs the same
under systemd, under a shell, or under a test harness.
"""
from __future__ import annotations
import argparse
import asyncio
import contextlib
import logging
import os
import signal
import sys
from pathlib import Path
from . import __version__
from .audit import AuditLog
from .clock import Clock, parse_duration
from .compositor import CompositorError, CompositorTarget
from .launcher import Launcher
from .manager import DEFAULT_SETTLE_S, ManagerConfig, SessionManager
from .policy import Policy, PolicyError, load_policy
from .registry import Registry
from .server import ControlServer
log = logging.getLogger("hhstreamd")
DEFAULT_SOCKET = "/run/hhstream/control.sock"
DEFAULT_RUNTIME_DIR = "/run/hhstream"
DEFAULT_STATE_DIR = "/var/lib/hhstream"
DEFAULT_CONFIG = "/etc/hhstream/config.toml"
ENV_SOCKET = "HHSTREAM_SOCKET"
ENV_RUNTIME_DIR = "HHSTREAM_RUNTIME_DIR"
ENV_STATE_DIR = "HHSTREAM_STATE_DIR"
ENV_CONFIG = "HHSTREAM_CONFIG"
ENV_COMPOSITOR_RUNTIME_DIR = "HHSTREAM_COMPOSITOR_RUNTIME_DIR"
#: Printed at startup when no compositor runtime directory was given. It is a
#: warning and not an error because the daemon is useful without a compositor —
#: a console app, a test harness — but on a desktop machine this is BUG-01
#: waiting to happen, and the operator should see it before the first `start`.
NO_COMPOSITOR_WARNING = (
"no --compositor-runtime-dir (or $%s): launched applications will inherit "
"this daemon's environment (XDG_RUNTIME_DIR=%s, WAYLAND_DISPLAY=%s) and may "
"not be able to reach a compositor — a GUI app will then exit immediately "
"with 'Failed to connect to Wayland display'"
)
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
prog="hhstreamd",
description=(
"The hhstream session daemon. Owns managed application sessions, "
"their process groups and the idle clock."
),
epilog=(
"Every flag below may also be set from the environment: "
f"{ENV_SOCKET}, {ENV_RUNTIME_DIR}, {ENV_STATE_DIR}, {ENV_CONFIG}, "
f"{ENV_COMPOSITOR_RUNTIME_DIR}. "
"The flag wins. The daemon never speaks the Wayland protocol (it only "
"locates the compositor's socket so a launched app can) and never "
"integrates with systemd; supervise it however you like."
),
)
parser.add_argument("--version", action="version", version=f"hhstreamd {__version__}")
parser.add_argument(
"--socket",
default=os.environ.get(ENV_SOCKET, DEFAULT_SOCKET),
help="control socket path (default: %(default)s)",
)
parser.add_argument(
"--state-dir",
default=os.environ.get(ENV_STATE_DIR, DEFAULT_STATE_DIR),
help="session records, logs and the audit log (default: %(default)s)",
)
parser.add_argument(
"--runtime-dir",
default=os.environ.get(ENV_RUNTIME_DIR, DEFAULT_RUNTIME_DIR),
help="per-session XDG_RUNTIME_DIR parent (default: %(default)s)",
)
parser.add_argument(
"--compositor-runtime-dir",
default=os.environ.get(ENV_COMPOSITOR_RUNTIME_DIR),
metavar="PATH",
help="the COMPOSITOR's runtime directory — the one holding its wayland-N "
"socket. This is not --runtime-dir (that is where this daemon's own "
"control socket and per-session scratch directories live). Given, the "
"socket is discovered by name and exported to every launched "
"application as XDG_RUNTIME_DIR/WAYLAND_DISPLAY/XDG_SESSION_TYPE, and "
"the daemon refuses to start if the directory holds no usable socket. "
"Omitted, applications inherit this daemon's environment and may not "
"reach a compositor at all.",
)
parser.add_argument(
"--config",
default=os.environ.get(ENV_CONFIG, DEFAULT_CONFIG),
help="TOML app allowlist (default: %(default)s). A missing file means "
"an empty allowlist and every start refused.",
)
parser.add_argument(
"--settle",
type=parse_duration,
default=DEFAULT_SETTLE_S,
metavar="DURATION",
help="how long after spawn to wait before calling a session ready, so an "
"app that exits instantly is reported as exited rather than ready "
"(default: %(default)ss)",
)
parser.add_argument(
"--socket-mode",
type=lambda v: int(v, 8),
default=0o660,
metavar="OCTAL",
help="mode for the control socket (default: 0660)",
)
parser.add_argument(
"--log-level",
default=os.environ.get("HHSTREAM_LOG_LEVEL", "info"),
choices=["debug", "info", "warning", "error"],
)
parser.add_argument(
"--check",
action="store_true",
help="validate the configuration and exit without listening",
)
return parser
def resolve_compositor(args: argparse.Namespace) -> CompositorTarget | None:
"""Validate ``--compositor-runtime-dir``, or warn that there is none.
Raises :class:`CompositorError` so the caller can refuse to start: a daemon
that accepts work and then kills every application that arrives is worse
than one that refuses while the operator is still watching.
"""
if not args.compositor_runtime_dir:
log.warning(
NO_COMPOSITOR_WARNING,
ENV_COMPOSITOR_RUNTIME_DIR,
os.environ.get("XDG_RUNTIME_DIR", "<unset>"),
os.environ.get("WAYLAND_DISPLAY", "<unset>"),
)
return None
target = CompositorTarget.from_runtime_dir(args.compositor_runtime_dir)
log.info(
"compositor: %s (WAYLAND_DISPLAY=%s)",
target.session.runtime_dir,
target.session.wayland_display,
)
return target
async def serve(
args: argparse.Namespace, policy: Policy, compositor: CompositorTarget | None
) -> int:
clock = Clock()
# Absolute, because these end up in a session record and in a launched
# app's environment: a relative HOME or XDG_RUNTIME_DIR would be resolved
# against whatever directory the app happens to be started in.
state_dir = Path(args.state_dir).resolve()
runtime_dir = Path(args.runtime_dir).resolve()
config = ManagerConfig(
state_dir=state_dir,
runtime_dir=runtime_dir,
policy=policy,
settle_s=args.settle,
compositor=compositor,
)
manager = SessionManager(
config,
clock=clock,
registry=Registry(state_dir, clock),
launcher=Launcher(runtime_dir, state_dir, compositor=compositor),
audit=AuditLog(state_dir / "events.jsonl", clock),
)
server = ControlServer(
manager, Path(args.socket).absolute(), socket_mode=args.socket_mode
)
summary = await manager.startup()
log.info(
"adopted %(adopted)s, tombstoned %(tombstoned)s, quarantined %(quarantined)s, "
"pruned %(pruned)s",
summary,
)
log.info(
"allowlist: %s",
", ".join(sorted(policy.apps)) or f"(empty — {policy.source or 'no config'})",
)
await server.start()
stop = asyncio.Event()
loop = asyncio.get_running_loop()
for sig in (signal.SIGINT, signal.SIGTERM):
with contextlib.suppress(NotImplementedError):
loop.add_signal_handler(sig, stop.set)
try:
await stop.wait()
finally:
log.info("shutting down; sessions are left running for the next daemon to adopt")
await server.stop()
await manager.shutdown()
return 0
def main(argv: list[str] | None = None) -> int:
args = build_parser().parse_args(argv)
logging.basicConfig(
level=getattr(logging, args.log_level.upper()),
format="%(asctime)s %(levelname)-7s %(name)s: %(message)s",
stream=sys.stderr,
)
try:
policy = load_policy(args.config)
except (PolicyError, OSError, ValueError) as exc:
print(f"hhstreamd: bad config {args.config}: {exc}", file=sys.stderr)
return 2
try:
compositor = resolve_compositor(args)
except CompositorError as exc:
# Refuse, naming the directory: every application launched from here
# would die on startup, and "state exited" one command later is a much
# worse place to find that out.
print(f"hhstreamd: {exc}", file=sys.stderr)
return 2
if args.check:
print(f"config {args.config}: {len(policy.apps)} app(s) allowlisted")
for name, rule in sorted(policy.apps.items()):
print(f" {name} -> {rule.exec}")
if compositor is None:
print("compositor: none configured; apps inherit the daemon's environment")
else:
print(
f"compositor: {compositor.session.runtime_dir} "
f"(WAYLAND_DISPLAY={compositor.session.wayland_display})"
)
return 0
try:
return asyncio.run(serve(args, policy, compositor))
except OSError as exc:
print(f"hhstreamd: {exc}", file=sys.stderr)
return 1
if __name__ == "__main__":
raise SystemExit(main())
+116
View File
@@ -0,0 +1,116 @@
"""The append-only event log (DESIGN.md §2.3, §8 item 5).
One JSON object per line in ``<state>/events.jsonl``, size-rotated. Every
lifecycle event and every accepted command, with **both** timestamps: wall time
for a human reading it at 09:00 to find out why Blender died at 03:00, and
boottime so the entry can be lined up with the deadline arithmetic that
actually drove the daemon.
A write failure is logged and dropped. Refusing a command because a log line
could not be written is the wrong trade (DESIGN.md §7 #15).
"""
from __future__ import annotations
import json
import logging
import os
from pathlib import Path
from typing import Any
from .clock import Clock, iso
from .textio import tail_lines
log = logging.getLogger("hhstreamd.audit")
DEFAULT_MAX_BYTES = 16 * 1024 * 1024
DEFAULT_KEEP = 3
#: Arguments are bounded in the audit record so one pathological request cannot
#: dominate the log.
MAX_FIELD_CHARS = 2048
class AuditLog:
def __init__(
self,
path: Path | str,
clock: Clock | None = None,
*,
max_bytes: int = DEFAULT_MAX_BYTES,
keep: int = DEFAULT_KEEP,
) -> None:
self.path = Path(path)
self.clock = clock or Clock()
self.max_bytes = max_bytes
self.keep = keep
def emit(self, event: str, **fields: Any) -> None:
record = {
"ts": iso(self.clock.wall()),
"boottime": round(self.clock.boottime(), 3),
"event": event,
}
record.update({k: _bound(v) for k, v in fields.items() if v is not None})
try:
self._rotate_if_needed()
self.path.parent.mkdir(parents=True, exist_ok=True)
line = json.dumps(record, ensure_ascii=True, sort_keys=True) + "\n"
with open(self.path, "a", encoding="utf-8") as fh:
fh.write(line)
except OSError as exc:
log.error("audit write failed (%s): dropping event %s", exc, event)
def _rotate_if_needed(self) -> None:
try:
size = self.path.stat().st_size
except FileNotFoundError:
return
if size < self.max_bytes:
return
for index in range(self.keep - 1, 0, -1):
src = self.path.with_suffix(self.path.suffix + f".{index}")
dst = self.path.with_suffix(self.path.suffix + f".{index + 1}")
if src.exists():
os.replace(src, dst)
os.replace(self.path, self.path.with_suffix(self.path.suffix + ".1"))
#: How far back `tail` will look. Filtering by session means the window
#: may hold far fewer than `limit` matching events; a wider window costs
#: one read of a file that is already on the page cache.
TAIL_BYTES = 1 << 20
def tail(self, limit: int = 20, *, session: str | None = None) -> list[dict[str, Any]]:
"""The most recent events, newest last. Used by ``status``.
Bounded in bytes, not just in lines: the audit log rotates at 16 MiB and
reading all of it to show twenty entries would be absurd.
"""
if limit <= 0:
return []
lines, _, _ = tail_lines(self.path, limit=10_000, max_bytes=self.TAIL_BYTES)
out: list[dict[str, Any]] = []
for line in reversed(lines):
line = line.strip()
if not line:
continue
try:
record = json.loads(line)
except json.JSONDecodeError:
continue
if session is not None and record.get("session") != session:
continue
out.append(record)
if len(out) >= limit:
break
out.reverse()
return out
def _bound(value: Any) -> Any:
if isinstance(value, str) and len(value) > MAX_FIELD_CHARS:
return value[:MAX_FIELD_CHARS] + "…"
if isinstance(value, list):
return [_bound(v) for v in value[:64]]
return value
+101
View File
@@ -0,0 +1,101 @@
"""Clocks (DESIGN.md §6.1).
Every deadline in the daemon is computed on ``CLOCK_BOOTTIME``:
* wall time is wrong — an NTP step or a DST change would silently extend or
collapse an idle window;
* ``CLOCK_MONOTONIC`` is wrong — it stops across system suspend, so a host
suspended for four hours would wake believing a session had been idle for
seconds;
* ``CLOCK_BOOTTIME`` includes suspend, which is what an operator means by
"no commands for more than three hours".
A wall-clock ISO timestamp is recorded alongside, for humans and logs only.
Nothing computes with it.
The clock is an object so it can be injected: :class:`ManualClock` is what the
reaper's unit tests advance by three hours without sleeping.
"""
from __future__ import annotations
import datetime as _dt
import time
class Clock:
"""The real clock: ``CLOCK_BOOTTIME`` for arithmetic, wall time for display."""
def boottime(self) -> float:
return time.clock_gettime(time.CLOCK_BOOTTIME)
def wall(self) -> float:
return time.time()
def wall_iso(self) -> str:
return iso(self.wall())
class ManualClock(Clock):
"""A clock under the test's control. Never used by the shipped daemon."""
def __init__(self, boottime: float = 1000.0, wall: float | None = None) -> None:
self._boottime = float(boottime)
self._wall = float(wall) if wall is not None else 1_700_000_000.0
def boottime(self) -> float:
return self._boottime
def wall(self) -> float:
return self._wall
def advance(self, seconds: float) -> None:
"""Move both clocks forward. Suspend is indistinguishable from this."""
self._boottime += float(seconds)
self._wall += float(seconds)
def iso(epoch: float) -> str:
"""Format a wall-clock epoch as a UTC ISO-8601 timestamp, second precision."""
return (
_dt.datetime.fromtimestamp(epoch, _dt.UTC)
.replace(microsecond=0)
.isoformat()
.replace("+00:00", "Z")
)
def parse_duration(text: str) -> float:
"""Parse ``30s`` / ``5m`` / ``3h`` / ``2d`` / a bare number of seconds.
Raises :class:`ValueError` with a message fit for an argparse error.
"""
s = text.strip().lower()
if not s:
raise ValueError("empty duration")
units = {"s": 1.0, "m": 60.0, "h": 3600.0, "d": 86400.0}
multiplier = 1.0
if s[-1] in units:
multiplier = units[s[-1]]
s = s[:-1]
try:
value = float(s)
except ValueError:
raise ValueError(f"not a duration: {text!r} (expected e.g. 30s, 5m, 3h)") from None
if value < 0:
raise ValueError(f"negative duration: {text!r}")
return value * multiplier
def format_duration(seconds: float | None) -> str:
"""Render a duration as ``H:MM:SS`` for the human table. ``None`` is an em dash."""
if seconds is None:
return "—"
seconds = int(max(0.0, seconds))
hours, rest = divmod(seconds, 3600)
minutes, secs = divmod(rest, 60)
return f"{hours}:{minutes:02d}:{secs:02d}"
+240
View File
@@ -0,0 +1,240 @@
"""Finding the compositor's Wayland socket, and the environment that reaches it.
This module exists because of BUG-01: the daemon spawned children with its own
inherited environment, and on the target machine that environment names the
*login* session (``XDG_RUNTIME_DIR=/run/user/1000``, no ``WAYLAND_DISPLAY``),
where no compositor is listening. Every GUI application therefore died inside a
second with ``Failed to connect to Wayland display``, and the only symptom one
command later was ``state exited``.
Three facts shape what is here.
**The compositor's runtime directory is not the daemon's.** ``--runtime-dir``
is where the daemon's control socket and the per-session scratch directories
live; ``--compositor-runtime-dir`` is where the compositor's ``wayland-N``
lives, owned by a *different* user (DESIGN.md §3.1, user A). Conflating them is
the bug, so they are separate options and this module only ever reads the
latter.
**The socket's name is not knowable in advance.** It is ``wayland-1`` on the
target machine, not ``wayland-0``, because a crashed compositor had already
taken the first name; libwayland itself picks the first free one. So the socket
is *discovered*, never assumed, and a stale leftover must not win over the live
one — hence the connect probe in :func:`discover`.
**A daemon that cannot reach a compositor should refuse to start.** Accepting
work and then killing every application that arrives is worse than refusing at
the point where the operator is still looking at the terminal. So discovery
raises :class:`CompositorError` and the caller exits; it never degrades quietly.
"""
from __future__ import annotations
import errno
import logging
import os
import socket
import stat
from dataclasses import dataclass
from pathlib import Path
log = logging.getLogger("hhstreamd.compositor")
#: Wayland sockets are ``wayland-<n>``; libwayland also writes a ``wayland-<n>.lock``
#: beside each one, which is an ordinary file and never a candidate.
SOCKET_PREFIX = "wayland-"
#: How long to wait for ``connect()`` on a candidate socket. The compositor is
#: local and accepting, or it is not; this only bounds a pathological case.
PROBE_TIMEOUT_S = 0.5
class CompositorError(Exception):
"""The configured compositor runtime directory holds no usable socket."""
@dataclass(frozen=True, slots=True)
class CompositorSession:
"""Where the compositor is, and the environment that attaches to it."""
runtime_dir: Path
wayland_display: str
@property
def socket_path(self) -> Path:
return self.runtime_dir / self.wayland_display
def env(self) -> dict[str, str]:
"""The variables a child needs, laid *on top of* its inherited ones.
``XDG_RUNTIME_DIR`` is the compositor's, not the daemon's and not the
session's private scratch directory: a toolkit resolves
``WAYLAND_DISPLAY`` relative to it, and the daemon's own value names a
login session with nothing listening in it. ``WAYLAND_DISPLAY`` is the
bare socket name rather than an absolute path, which is what every
toolkit expects to see. ``XDG_SESSION_TYPE`` is what GTK, Qt and
Chromium read to choose their Wayland backend over X11.
"""
return {
"XDG_RUNTIME_DIR": str(self.runtime_dir),
"WAYLAND_DISPLAY": self.wayland_display,
"XDG_SESSION_TYPE": "wayland",
}
@dataclass(slots=True)
class _Candidate:
name: str
connectable: bool
refused: bool # nobody is listening: a leftover from a crashed compositor
detail: str
@property
def rank(self) -> tuple[int, int, str]:
# Live sockets first; among equals, the highest ``wayland-N`` wins,
# because a compositor takes the first free name and so the survivor of
# a crash-and-restart is the *later* one.
return (0 if self.connectable else 1, -_suffix_order(self.name), self.name)
def _suffix_order(name: str) -> int:
suffix = name[len(SOCKET_PREFIX) :]
return int(suffix) if suffix.isdigit() else -1
def _probe(path: Path) -> _Candidate:
"""Try to connect, to tell a live socket from a crashed compositor's leftover."""
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
try:
sock.settimeout(PROBE_TIMEOUT_S)
sock.connect(str(path))
except OSError as exc:
# ECONNREFUSED is definitive: the inode is there and no process is
# behind it. Anything else (EACCES, EPERM, a timeout) is a reason to
# warn, not to discard the candidate — it may still be the live socket,
# and refusing to start over an ambiguous probe is its own outage.
refused = exc.errno in (errno.ECONNREFUSED, errno.ENOENT)
return _Candidate(path.name, False, refused, f"{type(exc).__name__}: {exc}")
finally:
sock.close()
return _Candidate(path.name, True, False, "connectable")
def candidates(runtime_dir: Path) -> list[str]:
"""Names in ``runtime_dir`` that are Wayland sockets, unordered.
Only ``S_ISSOCK`` entries count, so the ``.lock`` file beside each socket
and any directory named like one are never picked.
"""
found = []
with os.scandir(runtime_dir) as entries:
for entry in sorted(entries, key=lambda e: e.name):
if not entry.name.startswith(SOCKET_PREFIX):
continue
try:
mode = entry.stat(follow_symlinks=True).st_mode
except OSError: # vanished, or a dangling symlink
continue
if stat.S_ISSOCK(mode):
found.append(entry.name)
return found
def discover(runtime_dir: Path | str) -> CompositorSession:
"""Resolve the compositor's session from its runtime directory.
Raises :class:`CompositorError`, naming the directory, if it cannot be read,
holds no Wayland socket, or holds only sockets that nothing is listening on.
"""
path = Path(runtime_dir).expanduser()
try:
resolved = path.resolve(strict=True)
except OSError as exc:
raise CompositorError(f"compositor runtime dir {path}: {exc.strerror or exc}") from exc
if not resolved.is_dir():
raise CompositorError(f"compositor runtime dir {resolved} is not a directory")
try:
names = candidates(resolved)
except OSError as exc:
raise CompositorError(
f"compositor runtime dir {resolved}: cannot be listed ({exc.strerror or exc})"
) from exc
if not names:
raise CompositorError(
f"compositor runtime dir {resolved} holds no Wayland socket "
f"({SOCKET_PREFIX}N); is the compositor running, and is this the "
"right directory? It is the compositor's XDG_RUNTIME_DIR, not the "
"daemon's --runtime-dir."
)
probed = sorted((_probe(resolved / name) for name in names), key=lambda c: c.rank)
best = probed[0]
if not best.connectable and all(c.refused for c in probed):
detail = ", ".join(f"{c.name} ({c.detail})" for c in probed)
raise CompositorError(
f"compositor runtime dir {resolved} holds no usable Wayland socket: "
f"{detail}. These look like leftovers from a compositor that is no "
"longer running."
)
if not best.connectable:
log.warning(
"compositor socket %s did not accept a probe connection (%s); using it anyway",
best.name,
best.detail,
)
if len(probed) > 1:
log.info(
"compositor runtime dir %s holds %d Wayland sockets (%s); chose %s",
resolved,
len(probed),
", ".join(f"{c.name} [{c.detail}]" for c in probed),
best.name,
)
return CompositorSession(runtime_dir=resolved, wayland_display=best.name)
class CompositorTarget:
"""The configured compositor, re-resolved at every spawn.
Re-resolving matters because the compositor can restart under a running
daemon and come back on a different socket name (that is exactly how the
target machine ended up on ``wayland-1``). If re-resolution fails, the last
known-good session is reused and the failure is logged: a transient listing
error must not turn into every subsequent application being launched with no
compositor at all.
"""
def __init__(self, session: CompositorSession) -> None:
self.session = session
@classmethod
def from_runtime_dir(cls, runtime_dir: Path | str) -> "CompositorTarget":
"""Validate at startup. Raises :class:`CompositorError` if unusable."""
return cls(discover(runtime_dir))
def resolve(self) -> CompositorSession:
try:
session = discover(self.session.runtime_dir)
except CompositorError as exc:
log.warning(
"re-resolving the compositor socket failed (%s); keeping %s",
exc,
self.session.wayland_display,
)
return self.session
if session != self.session:
log.info(
"compositor socket changed: %s -> %s",
self.session.wayland_display,
session.wayland_display,
)
self.session = session
return session
def env(self) -> dict[str, str]:
return self.resolve().env()
+304
View File
@@ -0,0 +1,304 @@
"""Spatial containers: one headless output and one workspace per session.
A session used to be a process group and nothing more, so every application
landed on whichever workspace the compositor happened to be showing. Two
sessions then shared a screen: a capture of one caught the other's windows, and
a pointer aimed at one could land in the other. A container fixes that by
giving the session a place of its own — a dedicated headless output, a numbered
workspace bound to that output as its default and marked persistent so it
survives the window closing, and a window rule that pins anything the session
spawns there.
**This module computes command lines and nothing else.** It does not run them,
does not talk to a socket, does not read a file, does not look at a clock and
does not import :mod:`hhstreamd.ipc`. That separation is deliberate: the
compositor is the part that cannot be tested without a compositor, so every
decision that *can* be made in a pure function is made here, where a unit test
can assert the exact argv list. The broker executes what comes back.
Three properties the callers depend on:
**Byte-identical output for identical arguments.** The ``create`` list is
emitted in a fixed order — create the output, bind the workspace, pin the
window, then set the mode — that does not depend on the inputs, and no set or
dict is ever iterated to build a command. The only thing a caller can vary is
what it passes in, in the order the signature names: nothing else here is a
source of order. So ``container_plan`` is safe to call twice and diff, which is
how ``container.reconcile`` decides whether a live container matches its plan.
**Argv lists, never strings.** These values reach a compositor that will act on
them, so there is no shell, no quoting and no interpolation of a caller's string
into a command *word*. The one place caller data lands inside a word is a rule
body (``workspace 1000, monitor:hh-x``), where a comma or a newline would end
one field and begin another — which is why the validation below is a control and
not a style preference.
**Validation raises, and names the value.** A session id that is not
``[a-z0-9-]``, a non-positive size, an empty app id: all :class:`ValueError`.
The broker mints its own ids and sizes, so a rejection here means a bug upstream
of us, and a message that quotes the offending value is what makes that bug
findable.
"""
from __future__ import annotations
import re
from dataclasses import dataclass
#: Every output this broker creates carries it. A start-up scan uses the prefix
#: to tell its own orphans — outputs left behind by a crash — from the outputs a
#: human or another tool made, which it must not remove.
OUTPUT_PREFIX = "hh-"
#: Session ids as the broker mints them. Deliberately narrower than "a safe
#: command word": lowercase, digits and the dash, so the name is also a legal
#: Hyprland output name and a regex literal with nothing to escape.
SESSION_ID_RE = re.compile(r"\A[a-z0-9-]+\Z")
#: An application's window class, which we do not mint — it is whatever the
#: toolkit reports, so it has to allow the shapes real classes take
#: (``org.gnome.Nautilus``, ``Alacritty``, ``firefox-esr``). It still excludes
#: every character that could end a field in a rule body or a line in the
#: request: comma, space, quote, newline, ``$``, ``^``.
APP_ID_RE = re.compile(r"\A[A-Za-z0-9._+-]+\Z")
class NoSlots(Exception):
"""The workspace band is full. The caller refuses the session; it never reuses."""
@dataclass(frozen=True, slots=True)
class ContainerPlan:
"""What a session's container is, and the two command lists that realise it.
``create`` and ``teardown`` are complete argv lists in the order they must
run. ``teardown`` is not the reverse of ``create``: removing the output is
enough, because the workspace and window rules are keywords that die with
the compositor's configuration and name an output that no longer exists.
"""
output: str
workspace: int
mode: tuple[int, int]
create: list[list[str]]
teardown: list[list[str]]
def output_name(session_id: str) -> str:
"""``hh-<session-id>`` — the name of the session's headless output."""
return OUTPUT_PREFIX + _checked_session_id(session_id)
def contains_output(name: str) -> bool:
"""True only for an output name this broker could have created.
Used by the start-up scan, which removes what it recognises. So it is
strict in the direction that matters: a bare ``hh-``, a trailing newline or
an upper-case suffix is *not* one of ours, and a scan that answered True
would delete someone else's output.
"""
if not name.startswith(OUTPUT_PREFIX):
return False
return SESSION_ID_RE.match(name[len(OUTPUT_PREFIX) :]) is not None
def app_id_for(session_id: str) -> str:
"""The bare ``hh-<session-id>`` the launcher sets as the app's class.
Same string as the output name, and that is the point: the launcher stamps
it on the application (``--class``, ``app_id``) and
:func:`container_plan` anchors it into the window rule, so the window finds
its own workspace with no lookup and no race against the window appearing.
"""
return OUTPUT_PREFIX + _checked_session_id(session_id)
def container_plan(
session_id: str,
workspace: int,
width: int,
height: int,
app_id: str,
) -> ContainerPlan:
"""The commands that create and tear down one session's container.
``app_id`` is passed in rather than derived from ``session_id`` because the
rule has to match whatever class the application actually reports: it is
:func:`app_id_for` for a well-behaved app the launcher could stamp, and the
app's own class for one that ignores the stamp.
"""
name = output_name(session_id)
slot = _checked_int(workspace, "workspace", minimum=1)
w = _checked_int(width, "width", minimum=1)
h = _checked_int(height, "height", minimum=1)
match = _checked_app_id(app_id)
return ContainerPlan(
output=name,
workspace=slot,
mode=(w, h),
create=[
# The output first: the workspace rule below names it, and a rule
# for an output that does not exist yet binds to nothing.
["hyprctl", "output", "create", "headless", name],
# `default:true` makes this the workspace the output shows, and
# `persistent:true` keeps it alive when the last window closes — a
# container that evaporated between two applications would send the
# next window back to the shared screen.
[
"hyprctl",
"keyword",
"workspace",
f"{slot}, monitor:{name}, default:true, persistent:true",
],
# Anchored (`^...$`) so a class that merely contains the app id
# cannot claim another session's workspace. `initial_class` rather
# than the live class: applications rename their window, and the
# placement has to be decided when it is mapped.
[
"hyprctl",
"keyword",
"windowrule",
f"match:initial_class ^{match}$, workspace {slot}, monitor {name}",
],
# Last, because the mode applies to an output that now exists and
# already has its workspace; setting it earlier would resize a
# screen that is about to be replaced.
["hyprctl", "keyword", "monitor", f"{name},{w}x{h},auto,1"],
],
teardown=[["hyprctl", "output", "remove", name]],
)
def quarantine_plan(quarantine_workspace: str) -> ContainerPlan | None:
"""The backstop rule: a window that matched no container goes here.
Without it, a window with an unexpected class — a dialog spawned by a
helper process, a toolkit that rewrites its class — lands on whatever
workspace is in front, which is the leak containers exist to prevent. The
rule matches every class and is therefore emitted *before* the per-session
rules, which are anchored and more specific.
``quarantine_workspace`` is a workspace *name* (``special:quarantine``), not
a slot out of the allocator's band, so the returned plan carries no output
and no mode: it is a rule, not a container. An empty or blank name means
the operator turned the backstop off, and the answer is ``None`` rather than
a plan with an empty command list — a caller that gets a plan should always
have something to run.
"""
target = quarantine_workspace.strip()
if not target:
return None
_checked_rule_value(target, "quarantine_workspace")
return ContainerPlan(
output="",
workspace=0,
mode=(0, 0),
create=[
[
"hyprctl",
"keyword",
"windowrule",
f"match:initial_class .*, workspace {target}",
]
],
# A keyword rule cannot be unset one at a time, and this one is not
# per-session anyway: it is installed once at start-up and goes away
# with the compositor.
teardown=[],
)
class WorkspaceAllocator:
"""Hands out workspace slots from a fixed band, lowest free first.
Lowest-free rather than monotonic so a long-lived broker's numbers stay
small and readable (``hyprctl workspaces`` is something a human reads), and
so the band is a real bound: ``count`` slots means at most ``count``
containers, with exhaustion surfacing as :class:`NoSlots` at allocation
rather than as a collision later.
"""
def __init__(self, base: int, count: int) -> None:
self._base = _checked_int(base, "base", minimum=1)
self._count = _checked_int(count, "count", minimum=1)
self._used: set[int] = set()
def allocate(self) -> int:
"""The lowest free slot in ``[base, base + count)``. Raises :class:`NoSlots`."""
for slot in range(self._base, self._base + self._count):
if slot not in self._used:
self._used.add(slot)
return slot
raise NoSlots(
f"workspace band {self._base}..{self._base + self._count - 1} is full "
f"({self._count} in use)"
)
def release(self, workspace: int) -> None:
"""Give a slot back. A slot we never handed out raises :class:`KeyError`.
Loudly, because the two ways to get here are a double release and a
caller releasing someone else's slot, and both mean the registry and
this allocator have drifted. Swallowing it would hand the same
workspace to two sessions a few minutes later, which is the leak that is
hard to trace back.
"""
if workspace not in self._used:
raise KeyError(
f"workspace {workspace} was not allocated by this allocator "
f"(in use: {sorted(self._used)})"
)
self._used.discard(workspace)
def in_use(self) -> list[int]:
"""Allocated slots, ascending. Sorted because callers log and diff it."""
return sorted(self._used)
def _checked_session_id(session_id: str) -> str:
if not isinstance(session_id, str) or not SESSION_ID_RE.match(session_id):
raise ValueError(
f"session id {session_id!r} is not [a-z0-9-]+; it names an output and "
"goes into a compositor command"
)
return session_id
def _checked_app_id(app_id: str) -> str:
if not isinstance(app_id, str) or not app_id:
raise ValueError(f"app id {app_id!r} is empty; a window rule needs a class to match")
if not APP_ID_RE.match(app_id):
raise ValueError(
f"app id {app_id!r} is not [A-Za-z0-9._+-]+; a window rule body is "
"comma-separated, so anything else could forge a rule field"
)
return app_id
def _checked_rule_value(value: str, label: str) -> str:
# Same reasoning as an app id, one rule field over: this ends up inside a
# `workspace <value>` field, where a comma starts the next field. The colon
# is allowed because `special:quarantine` needs it.
if not re.match(r"\A[A-Za-z0-9._:+-]+\Z", value):
raise ValueError(
f"{label} {value!r} is not [A-Za-z0-9._:+-]+; it is interpolated into "
"a comma-separated rule body"
)
return value
def _checked_int(value: int, label: str, *, minimum: int) -> int:
# bool is an int in Python, and `True` would silently become workspace 1.
if not isinstance(value, int) or isinstance(value, bool):
raise ValueError(f"{label} {value!r} is not an int")
if value < minimum:
raise ValueError(f"{label} {value!r} is not >= {minimum}")
return value
+237
View File
@@ -0,0 +1,237 @@
"""Spawning session applications (DESIGN.md §3.1, §3.3; brief boundaries).
Every app is spawned with ``start_new_session=True``, which gives it a fresh
session id *and* process group with ``sid == pgid == pid``. That pair is the
kill handle: there is no cgroup here, so the process group is what a teardown
signals and the session id is what the sweep searches for.
**Never a shell.** ``argv`` is a list, built by :mod:`hhstreamd.policy` from an
absolute path in the config plus caller arguments that matched an allowlist
pattern. There is no ``shell=True`` and no string interpolation into a command
line anywhere in this package.
**Why :class:`subprocess.Popen` and not ``asyncio.create_subprocess_exec``.**
asyncio's subprocess transport kills the child when it is closed — which
happens when the daemon's event loop shuts down, and again via the transport's
``__del__``. Using it here would mean every session application was SIGKILLed
the moment the daemon stopped, so no session would ever survive a daemon
restart and ``adopt`` would have nothing to adopt. That is the opposite of
what DESIGN.md §2.4 requires, and no unit file setting can fix it because the
kill comes from inside this process. A plain ``Popen`` has no such behaviour:
it never signals the child, and the daemon watches for exit through a pidfd
instead (see :mod:`hhstreamd.manager`). The fork/exec briefly blocks the event
loop, which is a few milliseconds and the right trade.
**The compositor environment (BUG-01).** A child does *not* get a usable
Wayland connection from the daemon's inherited environment: on the target machine
the daemon runs in the login session (``XDG_RUNTIME_DIR=/run/user/1000``, no
``WAYLAND_DISPLAY``) and the compositor listens somewhere else entirely. So when
``--compositor-runtime-dir`` names the compositor's runtime directory, the
discovered session (:mod:`hhstreamd.compositor`) is laid over the inherited
environment at spawn. Without that option the old inherited-environment
behaviour stands, and the daemon warns at startup that applications may not
reach a compositor.
The app runs as **the same unprivileged user as the daemon** and inherits the
real home directory. M0 invents no service account and forces no throwaway
``HOME``; ``--home`` pins a different one and ``--ephemeral-home`` opts in to a
per-session one that is deleted on teardown.
"""
from __future__ import annotations
import logging
import os
import shutil
import subprocess
from dataclasses import dataclass
from pathlib import Path
from .compositor import CompositorTarget
from .registry import Session
log = logging.getLogger("hhstreamd.launcher")
#: Environment variables the daemon's own plumbing must not leak into an app.
_STRIPPED = (
"HHSTREAM_SOCKET",
"HHSTREAM_STATE_DIR",
"HHSTREAM_RUNTIME_DIR",
"HHSTREAM_CONFIG",
"HHSTREAM_COMPOSITOR_RUNTIME_DIR",
)
@dataclass(slots=True)
class Spawned:
process: subprocess.Popen
pid: int
pgid: int
sid: int
home: str
#: The **daemon-owned** per-session runtime directory, which is what goes in
#: the record and what ``discard_runtime_dir`` may delete. It is not
#: necessarily the child's ``XDG_RUNTIME_DIR``: with a compositor
#: configured, that is the compositor's directory, owned by another user.
runtime_dir: str
log_path: str
class Launcher:
def __init__(
self,
runtime_dir: Path | str,
state_dir: Path | str,
*,
compositor: CompositorTarget | None = None,
) -> None:
self.runtime_dir = Path(runtime_dir)
self.state_dir = Path(state_dir)
#: The compositor to attach children to, or ``None`` to leave the
#: inherited environment alone (``--compositor-runtime-dir`` unset).
self.compositor = compositor
# ---- per-session directories -------------------------------------
def session_runtime_dir(self, session_id: str) -> Path:
return self.runtime_dir / "sessions" / session_id
def session_home(self, session_id: str) -> Path:
return self.state_dir / "homes" / session_id
def log_path(self, session_id: str) -> Path:
return self.state_dir / "logs" / f"{session_id}.log"
# ---- environment -------------------------------------------------
def build_env(self, session_id: str, home: Path) -> dict[str, str]:
"""The app's environment.
It inherits the daemon's, minus the daemon's own configuration, plus a
private ``XDG_RUNTIME_DIR`` for the app's own locks and sockets.
When a compositor is configured, its session is laid **on top** of that:
``XDG_RUNTIME_DIR`` becomes the compositor's (a toolkit resolves
``WAYLAND_DISPLAY`` against it), ``WAYLAND_DISPLAY`` the discovered
socket name, ``XDG_SESSION_TYPE`` ``wayland``. This is BUG-01: without
it a child inherits the daemon's login-session environment, finds no
compositor there, and dies within a second — and the only symptom is
``state exited`` one command later. The private directory is still
created and still recorded, because the daemon owns and cleans it up;
it is simply not what the app sees as ``XDG_RUNTIME_DIR`` any more.
With no compositor configured, ``WAYLAND_DISPLAY`` is passed through
untouched when the daemon has one: the daemon still never speaks the
Wayland protocol itself, but it must not *prevent* an app from doing so.
"""
env = {k: v for k, v in os.environ.items() if k not in _STRIPPED}
runtime = self.session_runtime_dir(session_id)
runtime.mkdir(parents=True, exist_ok=True, mode=0o700)
env["HOME"] = str(home)
env["XDG_RUNTIME_DIR"] = str(runtime)
env["HHSTREAM_SESSION_ID"] = session_id
if self.compositor is not None:
env.update(self.compositor.env())
return env
def resolve_home(self, session_id: str, home: str | None, ephemeral: bool) -> tuple[Path, bool]:
"""Decide the app's ``HOME``. Returns ``(path, is_ephemeral)``."""
if home is not None and ephemeral:
raise ValueError("--home and --ephemeral-home are mutually exclusive")
if ephemeral:
path = self.session_home(session_id)
path.mkdir(parents=True, exist_ok=True, mode=0o700)
return path, True
if home is not None:
return Path(home), False
inherited = os.environ.get("HOME") or str(Path.home())
return Path(inherited), False
def discard_home(self, session: Session) -> None:
"""Delete an ephemeral home on teardown. Best effort, never fatal."""
if not session.ephemeral_home or not session.home:
return
expected = self.session_home(session.id)
if Path(session.home) != expected: # never remove a directory we did not create
log.warning("refusing to delete %s: not the session's ephemeral home", session.home)
return
shutil.rmtree(session.home, ignore_errors=True)
def discard_runtime_dir(self, session: Session) -> None:
"""Remove the session's private ``XDG_RUNTIME_DIR`` once it is pruned.
Kept until the tombstone is pruned rather than removed at teardown, so
a post-mortem can still look at what the app left behind.
"""
if not session.runtime_dir:
return
expected = self.session_runtime_dir(session.id)
if Path(session.runtime_dir) != expected:
log.warning("refusing to delete %s: not the session's runtime dir", session.runtime_dir)
return
shutil.rmtree(session.runtime_dir, ignore_errors=True)
# ---- spawn -------------------------------------------------------
def spawn(
self,
session_id: str,
argv: list[str],
*,
home: str | None = None,
ephemeral_home: bool = False,
) -> Spawned:
"""Start the application. Raises ``OSError`` if it cannot be executed."""
if not argv:
raise ValueError("argv is empty")
if not os.path.isabs(argv[0]):
# Policy already guarantees this; asserting it here means a future
# caller cannot bypass the allowlist by handing in a bare name that
# would be resolved against PATH.
raise ValueError(f"argv[0] must be an absolute path, got {argv[0]!r}")
home_path, is_ephemeral = self.resolve_home(session_id, home, ephemeral_home)
env = self.build_env(session_id, home_path)
log_file = self.log_path(session_id)
log_file.parent.mkdir(parents=True, exist_ok=True, mode=0o750)
# stdout and stderr are captured to a file the daemon owns, because
# `session logs` is served from here rather than from the journal.
fh = open(log_file, "ab", buffering=0)
try:
process = subprocess.Popen(
argv,
stdin=subprocess.DEVNULL,
stdout=fh,
stderr=subprocess.STDOUT,
cwd=str(home_path) if home_path.is_dir() else "/",
env=env,
start_new_session=True, # sid == pgid == pid; the kill handle
close_fds=True,
)
finally:
fh.close()
pid = process.pid
# A process spawned with start_new_session has sid == pgid == pid by
# construction. Read them back anyway: if the app has already exited,
# /proc is gone and the recorded values must still be the right ones to
# signal, which they are.
pgid = sid = pid
return Spawned(
process=process,
pid=pid,
pgid=pgid,
sid=sid,
home=str(home_path),
# The daemon-owned per-session directory, which is what
# `discard_runtime_dir` is allowed to delete — *not* the compositor's
# XDG_RUNTIME_DIR, which belongs to another user entirely.
runtime_dir=str(self.session_runtime_dir(session_id)),
log_path=str(log_file),
)
+1000
View File
@@ -0,0 +1,1000 @@
"""The session manager: every verb's implementation, and the one kill path.
This module owns the state machine. :mod:`hhstreamd.server` is only framing
and dispatch; everything that decides anything lives here.
Two things are worth finding quickly:
* :meth:`SessionManager.teardown` is **the** teardown — ``killpg(SIGTERM)``,
grace, ``killpg(SIGKILL)``, then the ``/proc`` session-id sweep (DESIGN.md
§3.3). ``kill`` and the reaper both call it with a different ``reason``, so
there is one kill path and not two.
* :meth:`SessionManager.touch` is the **only** mark of activity in M0. There is
no input verb and no session-scoped frame verb, so an agent observing a
long-running app has no other way to say "still working". ``list``,
``status``, ``logs`` and ``gc`` deliberately do not stamp it: if monitoring
refreshed the clock, a supervisor polling every 60 s would keep every
abandoned session alive forever and the reaper would never fire.
"""
from __future__ import annotations
import asyncio
import contextlib
import logging
import os
import signal
import subprocess
import time
from dataclasses import dataclass
from pathlib import Path
from typing import Any
from . import procmon
from .audit import AuditLog
from .clock import Clock, iso
from .compositor import CompositorTarget
from .launcher import Launcher
from .policy import Policy, PolicyError, check_app_name
from .reaper import Reaper
from .registry import (
TERMINAL_ERROR,
Registry,
Session,
State,
)
from .textio import tail_lines
log = logging.getLogger("hhstreamd.manager")
#: How long after spawn the daemon waits before calling a session ready.
#: It exists for one real failure mode: an app that hands off to an existing
#: instance and exits immediately (DESIGN.md §7 #3). Without the settle,
#: `start` would return a healthy-looking id for a process that is already
#: gone. It is deliberately short.
DEFAULT_SETTLE_S = 0.1
#: How long a teardown waits for SIGKILL to take effect before it calls a pid a
#: survivor. SIGKILL is not synchronous: the task is woken and torn down by the
#: kernel, which takes microseconds normally but can block on uninterruptible
#: I/O. Short enough not to stall a verb, long enough that a healthy process is
#: never libelled as unkillable.
SURVIVOR_GRACE_S = 0.5
#: A single `logs` reply never carries more than this, so one request cannot
#: exceed the protocol's frame limit however much the app has written.
MAX_LOG_CHUNK_BYTES = 256 * 1024
SIGNALS_BY_NAME = {
"TERM": signal.SIGTERM,
"SIGTERM": signal.SIGTERM,
"KILL": signal.SIGKILL,
"SIGKILL": signal.SIGKILL,
"INT": signal.SIGINT,
"SIGINT": signal.SIGINT,
"HUP": signal.SIGHUP,
"SIGHUP": signal.SIGHUP,
"QUIT": signal.SIGQUIT,
"SIGQUIT": signal.SIGQUIT,
}
class ManagerError(Exception):
"""A verb that failed for a reason the protocol has a name for."""
def __init__(self, error: str, reason: str, hint: str | None = None, **fields: Any) -> None:
super().__init__(reason)
self.error = error
self.reason = reason
self.hint = hint
self.fields = fields
@dataclass(slots=True)
class ManagerConfig:
state_dir: Path
runtime_dir: Path
policy: Policy
settle_s: float = DEFAULT_SETTLE_S
#: The compositor launched applications attach to, or ``None`` for the
#: inherited environment (``--compositor-runtime-dir`` unset).
compositor: CompositorTarget | None = None
class SessionManager:
def __init__(
self,
config: ManagerConfig,
*,
clock: Clock | None = None,
registry: Registry | None = None,
launcher: Launcher | None = None,
audit: AuditLog | None = None,
) -> None:
self.config = config
self.clock = clock or Clock()
self.registry = registry or Registry(config.state_dir, self.clock)
self.launcher = launcher or Launcher(
config.runtime_dir, config.state_dir, compositor=config.compositor
)
self.audit = audit or AuditLog(Path(config.state_dir) / "events.jsonl", self.clock)
self.reaper = Reaper(
now=self.clock.boottime,
sessions=self.registry.live,
on_expire=self._reap,
)
# Per-session asyncio plumbing, never persisted.
self._exited: dict[str, asyncio.Event] = {}
self._watchers: dict[str, asyncio.Task] = {}
self._busy: set[str] = set()
# ------------------------------------------------------------------
# startup / shutdown
# ------------------------------------------------------------------
async def startup(self) -> dict[str, Any]:
"""Load the mirror and adopt what is still alive (DESIGN.md §2.4)."""
self.registry.ensure_dirs()
records, quarantined = self.registry.load_records()
summary = {
"adopted": 0,
"tombstoned": 0,
"quarantined": len(quarantined),
"pruned": 0,
# Records whose pid could be neither confirmed nor ruled out.
# Never silent: each one is logged at ERROR as it happens.
"unresolved": 0,
}
for record in records:
if not record.live:
self.registry.insert_loaded(record)
continue
self.registry.insert_loaded(record)
if self.registry.stale_boot(record):
# The record predates this boot. Its processes cannot exist:
# pids restart from zero, so /proc would lie if we asked it.
self.registry.terminate(
record, State.EXITED, exit_detected_by="stale_boot", last_error="stale_boot"
)
summary["tombstoned"] += 1
self.audit.emit("adopt_stale_boot", session=record.id, app=record.app)
continue
verdict = self._identity_of(record)
if not verdict.ok:
# A wrapper-launched app used to land here, because its argv had
# changed since launch; it is now adopted like any other
# (BUG-02 item 4). What reaches this branch is a process that
# really is gone, or a pid the daemon cannot claim — and in the
# latter case `_identity_of` has already said so at ERROR, and
# `gc` will report any process left sharing the session id.
self.registry.terminate(
record,
State.EXITED,
exit_detected_by="proc_check",
last_error=verdict.reason,
)
summary["tombstoned"] += 1
if verdict.unresolved:
summary["unresolved"] += 1
self.audit.emit(
"adopt_failed",
session=record.id,
app=record.app,
why=verdict.reason,
checked=list(verdict.checked),
mismatched=list(verdict.mismatched),
)
continue
record.adopted = True
record.exit_detected_by = None
self.registry.persist(record, force=True)
self._watch(record)
summary["adopted"] += 1
self.audit.emit("adopted", session=record.id, app=record.app, pid=record.pid)
summary["pruned"] = self._prune_tombstones()
self.audit.emit("daemon_started", pid=os.getpid(), **summary)
# Expired deadlines are reaped on the reaper's first pass, which runs
# before its first sleep.
self.reaper.start()
return summary
async def shutdown(self) -> None:
"""Stop watching, flush the mirror. **Sessions are left running.**
The daemon is supervised by someone else and may be restarted at any
time; killing every app on the way out would make a restart destructive.
``startup`` adopts them back.
"""
await self.reaper.stop()
for task in list(self._watchers.values()):
task.cancel()
for task in list(self._watchers.values()):
with contextlib.suppress(asyncio.CancelledError, Exception):
await task
self._watchers.clear()
self.registry.flush_all()
self.audit.emit("daemon_stopping", pid=os.getpid(), live=len(self.registry.live()))
# ------------------------------------------------------------------
# helpers
# ------------------------------------------------------------------
def now(self) -> float:
return self.clock.boottime()
def _require(self, session_id: Any) -> Session:
if not isinstance(session_id, str) or not session_id:
raise ManagerError("invalid_argument", "a session id is required")
session = self.registry.get(session_id)
if session is None:
raise ManagerError("no_such_session", f"no session {session_id!r}")
return session
def _require_live(self, session_id: Any) -> Session:
session = self._require(session_id)
if not session.live:
if session.last_error in procmon.UNRESOLVED:
# BUG-02 item 5. This session was not observed to exit — the
# daemon gave up on it because it could not prove the pid was
# still ours, and signalled nothing. Answering a later `kill`
# with the same "is exited" as a routine exit would hide exactly
# the fact the caller needs: an application may still be running.
raise ManagerError(
session.last_error,
f"session {session.id} was given up on: pid {session.pid} could not "
f"be identified as this session ({session.last_error}), so nothing "
f"was ever signalled",
hint="if the application is still running, `hhstream session gc` "
"lists it under orphaned processes; kill it by hand",
id=session.id,
state=session.state,
)
raise ManagerError(
TERMINAL_ERROR.get(session.state, "bad_state"),
f"session {session.id} is {session.state}",
id=session.id,
state=session.state,
)
return session
def _event(self, session_id: str) -> asyncio.Event:
return self._exited.setdefault(session_id, asyncio.Event())
def _identity_of(self, session: Session) -> procmon.Verdict:
"""The one identity gate. **Every** caller goes through here (BUG-02).
``kill``, the reaper, ``gc`` and restart adoption all have to answer the
same question — "is the pid in this record still the process we
recorded?" — and before BUG-02 they each asked it with their own call to
a check that was wrong in the same way. Routing them through one method
means the evidence, the logging and the drift reporting cannot drift
apart between callers.
"""
verdict = procmon.check(
session.pid,
starttime=session.pid_starttime,
sid=session.sid,
pgid=session.pgid,
cmdline_sha256=session.cmdline_sha256,
)
if verdict.ok and verdict.cmdline_drifted:
# Entirely normal: a wrapper script `exec`ed the real binary, or the
# app rewrote its own argv. Worth one line in the log and nothing
# more — it is not evidence of anything (BUG-02).
log.debug(
"%s (pid %s): argv changed since launch; identity confirmed by %s",
session.id,
session.pid,
"+".join(verdict.checked),
)
elif verdict.unresolved:
log.error(
"%s: refusing to treat pid %s as this session — %s (checked %s, mismatched %s)",
session.id,
session.pid,
verdict.reason,
"+".join(verdict.checked) or "nothing",
"+".join(verdict.mismatched) or "nothing",
)
return verdict
# ------------------------------------------------------------------
# exit watching
# ------------------------------------------------------------------
def _watch(self, session: Session, process: subprocess.Popen | None = None) -> None:
"""Watch one session for exit. One path for own children and adopted ones.
``os.pidfd_open`` (Linux >= 5.3) notifies on exit for a process that is
**not** our child, which is the adopted case, and works just as well for
one that is. The pidfd becomes readable when the process terminates, so
there is no polling and no latency.
**The documented fallback** for a kernel without ``pidfd_open``, or a
pidfd this event loop cannot poll, is a one-second ``/proc`` liveness
poll (``procmon.POLL_INTERVAL_S``). It costs one ``stat`` per session
per second and detects an exit up to a second late; nothing else
changes, because both paths converge on :meth:`_on_exit`.
**The exit status** comes from ``waitpid`` and is therefore available
only for the daemon's own children. For an adopted process the record
says ``exit_code: null`` with ``exit_detected_by`` naming the mechanism,
rather than inventing a zero.
"""
def reap() -> int | None:
if process is None:
return None
# The pidfd fired, so the child is a zombie and waitpid will not
# block. poll() is subprocess's waitpid(WNOHANG).
for _ in range(50):
code = process.poll()
if code is not None:
return code
time.sleep(0.002)
return None # pragma: no cover - the kernel would have to lie
async def watch_pidfd(fd: int) -> None:
loop = asyncio.get_running_loop()
done = asyncio.Event()
try:
loop.add_reader(fd, done.set)
except (OSError, NotImplementedError): # pragma: no cover
os.close(fd)
log.warning("pidfd is not pollable here; falling back to /proc polling")
await watch_poll()
return
try:
await done.wait()
finally:
with contextlib.suppress(OSError, ValueError):
loop.remove_reader(fd)
os.close(fd)
self._on_exit(session.id, exit_code=reap(), detected_by="pidfd")
async def watch_poll() -> None:
while True:
await asyncio.sleep(procmon.POLL_INTERVAL_S)
if not procmon.is_alive(session.pid):
self._on_exit(session.id, exit_code=reap(), detected_by="proc_poll")
return
fd = procmon.pidfd_open(session.pid)
if fd is None and not procmon.is_alive(session.pid):
# Already gone between spawn/adopt and here.
self._on_exit(session.id, exit_code=reap(), detected_by="proc_check")
return
coro = watch_pidfd(fd) if fd is not None else watch_poll()
self._watchers[session.id] = asyncio.create_task(coro, name=f"hhstreamd.watch:{session.id}")
def _on_exit(self, session_id: str, *, exit_code: int | None, detected_by: str) -> None:
self._event(session_id).set()
session = self.registry.get(session_id)
if session is None or not session.live:
return
self.registry.terminate(
session, State.EXITED, exit_code=exit_code, exit_detected_by=detected_by
)
self.audit.emit(
"session_exited",
session=session.id,
app=session.app,
pid=session.pid,
exit_code=exit_code,
detected_by=detected_by,
)
self.reaper.wake()
# ------------------------------------------------------------------
# verbs
# ------------------------------------------------------------------
async def start(
self,
app: str,
argv: list[str] | None = None,
*,
idle_timeout_s: float | None = None,
wait_ready_s: float | None = None,
home: str | None = None,
ephemeral_home: bool = False,
) -> dict[str, Any]:
policy = self.config.policy
policy.check_capacity(len(self.registry.live()))
rule, full_argv = policy.resolve(app, argv)
if idle_timeout_s is None and rule.idle_timeout_s is not None:
idle_timeout_s = rule.idle_timeout_s
timeout = policy.check_idle_timeout(idle_timeout_s)
wait_ready = policy.wait_ready_s if wait_ready_s is None else float(wait_ready_s)
session_id = self.registry.new_id(app)
try:
spawned = self.launcher.spawn(
session_id, full_argv, home=home, ephemeral_home=ephemeral_home
)
except (OSError, ValueError) as exc:
self.audit.emit("start_failed", app=app, argv=full_argv, reason=str(exc))
raise ManagerError(
"refused", f"could not execute {full_argv[0]!r}: {exc}"
) from exc
identity = procmon.identity(spawned.pid)
now = self.now()
session = Session(
id=session_id,
app=app,
argv=full_argv,
pid=spawned.pid,
pgid=identity.pgid if identity else spawned.pgid,
sid=identity.sid if identity else spawned.sid,
pid_starttime=identity.starttime if identity else 0,
cmdline_sha256=identity.cmdline_sha256 if identity else "",
home=spawned.home,
runtime_dir=spawned.runtime_dir,
log_path=spawned.log_path,
created_at=iso(self.clock.wall()),
created_boottime=now,
last_activity_at=iso(self.clock.wall()),
last_activity_boottime=now, # `start` stamps activity implicitly
idle_timeout_s=timeout,
state=State.STARTING,
ephemeral_home=ephemeral_home,
command_count=1,
)
self.registry.add(session)
self._exited.pop(session_id, None)
self._watch(session, spawned.process)
self.audit.emit(
"session_started",
session=session.id,
app=app,
pid=session.pid,
pgid=session.pgid,
sid=session.sid,
argv=full_argv,
idle_timeout_s=timeout,
)
ready = await self._settle(session, wait_ready)
self.reaper.wake()
record = session.public(self.now())
if session.state == State.EXITED:
# The app exited during the settle: surface it rather than handing
# back a healthy-looking id (DESIGN.md §7 #3).
raise ManagerError(
"session_exited",
f"{app} exited immediately (exit code {session.exit_code})",
hint="the app may have handed off to an existing instance; "
"try --ephemeral-home to force a fresh one",
id=session.id,
session=record,
)
if not ready:
# DESIGN.md §7 #4: the session is left alive and the id is printed;
# killing an app that took 20 s to start is worse than a slow
# success. Unreachable in M0, where readiness is "still running".
raise ManagerError(
"readiness_timeout",
f"{app} did not signal readiness within {wait_ready:g}s",
id=session.id,
session=record,
)
return record
async def _settle(self, session: Session, wait_ready: float) -> bool:
"""Wait briefly for the process to prove it is still there.
M0 has no compositor, so the design's ``wayland-fd`` readiness probe
degrades to **absent**: the only honest signal available here is that
the process is alive and is still the process we spawned. That is
recorded as ``ready_signal: "spawned"`` so nothing downstream mistakes
it for evidence that a window was mapped.
"""
settle = min(self.config.settle_s, wait_ready) if wait_ready > 0 else 0.0
if settle > 0:
with contextlib.suppress(TimeoutError):
await asyncio.wait_for(self._event(session.id).wait(), timeout=settle)
if not session.live:
return False
session.state = State.READY
session.ready_signal = "spawned"
self.registry.persist(session, force=True)
return True
def list(self, *, include_tombstones: bool = False) -> dict[str, Any]:
"""Read-only. **Does not** count as activity."""
now = self.now()
sessions = self.registry.all() if include_tombstones else self.registry.live()
return {
"sessions": [s.public(now) for s in sessions],
"live": len(self.registry.live()),
"tombstones": len(self.registry.tombstones()),
"boottime": now,
}
def status(self, session_id: str, *, events: int = 20) -> dict[str, Any]:
"""Read-only. **Does not** count as activity."""
session = self._require(session_id)
record = session.public(self.now())
record["events"] = self.audit.tail(events, session=session.id)
record["compositor"] = {
# Enrichment the design calls optional, absent by construction in
# M0: the daemon never speaks the Wayland protocol and never talks
# to Hyprland's IPC. Locating the compositor's socket so a child
# can attach to it (see hhstreamd.compositor) tells us nothing
# about windows, so none of this becomes knowable.
"ipc": "absent",
"window": None,
"workspace": None,
"foreground": None,
"note": "M0 does not talk to the compositor; these are never guessed",
}
return record
def touch(self, session_id: str) -> dict[str, Any]:
"""**The only mark of activity in M0.** Resets the idle clock, injects nothing."""
session = self._require_live(session_id)
self.registry.touch(session, force_flush=True)
self.audit.emit(
"touch",
session=session.id,
app=session.app,
expires_in_s=round(session.expires_in(self.now()) or 0.0, 3),
)
self.reaper.wake()
return session.public(self.now())
async def kill(
self,
session_id: str,
*,
grace_s: float | None = None,
sig: str | int = signal.SIGTERM,
) -> dict[str, Any]:
session = self._require_live(session_id)
signum = _signal_number(sig)
grace = self.config.policy.grace_s if grace_s is None else float(grace_s)
if grace < 0:
raise ManagerError("invalid_argument", "grace may not be negative")
result = await self.teardown(session, reason="killed", grace_s=grace, sig=signum)
record = {**session.public(self.now()), **result}
if result["refused"]:
# BUG-02 item 5: the caller asked for a kill that did not happen, so
# they get an error and exit 4, not a success line that reads like
# the process was already gone.
raise ManagerError(
result["refused"],
f"refused to signal session {session.id}: pid {session.pid} can no longer "
f"be identified as this session "
f"({', '.join(result['identity_mismatched']) or 'no usable evidence'} "
f"of {', '.join(result['identity_checked']) or 'nothing'})",
hint="nothing was signalled; if an application is still running, "
"find it with `hhstream session gc` and kill it by hand",
id=session.id,
session=record,
)
return record
async def _survivors(self, session: Session) -> list[int]:
"""Pids of the session still alive after the teardown has finished.
The leader plus anything still carrying the session id. This is the
honest answer to "did the kill work", and it is what stops ``kill`` from
reporting success over a process that is still on the screen (BUG-02).
"""
# Real time, not the injected clock: this waits on the kernel, not on
# session bookkeeping, and a test that freezes the clock must not turn
# it into an infinite loop.
deadline = time.monotonic() + SURVIVOR_GRACE_S
while True:
alive = set(procmon.session_members(session.sid, exclude={os.getpid()}))
if procmon.is_alive(session.pid):
alive.add(session.pid)
if not alive or time.monotonic() >= deadline:
return sorted(alive)
await asyncio.sleep(0.02)
async def _reap(self, session_id: str) -> None:
session = self.registry.get(session_id)
if session is None or not session.live:
return
result = await self.teardown(session, reason="reaped", grace_s=self.config.policy.grace_s)
if result["refused"] or result["survivors"]:
# There is no caller to raise at here, so the loudness has to be in
# the log and the audit trail. The session is tombstoned either
# way, which is what keeps the reaper from spinning on it.
log.error(
"reaping %s did not complete: refused=%s survivors=%s",
session_id,
result["refused"],
result["survivors"],
)
async def teardown(
self,
session: Session,
*,
reason: str,
grace_s: float,
sig: int = signal.SIGTERM,
) -> dict[str, Any]:
"""The one kill path (DESIGN.md §3.3, §6.3).
``killpg(TERM)`` → grace → ``killpg(KILL)`` → sweep ``/proc`` for the
session id. The sweep is the substitute for a cgroup kill: a child that
called ``setsid()`` once has left the process group but kept the session
id, so it is caught here. A child that called ``setsid()`` twice escapes
even this; ``gc`` reports it.
**It reports what it actually did** (BUG-02 item 1): the signal that was
sent and to which process group, whether it escalated, how many
stragglers the sweep took, and — the one that matters — whether anything
is *still alive* afterwards. Before BUG-02 the only thing a caller got
when the gate refused was ``signalled: false``, which the CLI rendered as
"the process was already gone"; an application that cannot be killed
while the tool reports success is the failure mode this exists to
prevent. A refusal is now carried back in ``refused`` and turned into an
error by :meth:`kill`.
"""
if session.id in self._busy:
raise ManagerError("busy", f"session {session.id} is already being torn down")
self._busy.add(session.id)
state = State.REAPED if reason == "reaped" else State.KILLED
idle_s = session.idle_for(self.now())
self.audit.emit(
f"{reason}_started",
session=session.id,
app=session.app,
pid=session.pid,
pgid=session.pgid,
signal=signal.Signals(sig).name,
grace_s=grace_s,
idle_s=None if idle_s is None else round(idle_s, 3),
)
try:
exited = self._event(session.id)
# Only signal a process group we can still prove is ours.
verdict = self._identity_of(session)
escalated = False
sent: str | None = None
refused: str | None = None
if verdict.ok:
# The wrapper case lands here now: `argv` changed when the
# wrapper `exec`ed the real binary, and that is not evidence of
# anything, so the application is signalled like any other.
procmon.killpg(session.pgid, sig)
sent = signal.Signals(sig).name
if grace_s > 0:
with contextlib.suppress(TimeoutError):
await asyncio.wait_for(exited.wait(), timeout=grace_s)
if procmon.is_alive(session.pid) or procmon.session_members(
session.sid, exclude={os.getpid()}
):
escalated = True
procmon.killpg(session.pgid, signal.SIGKILL)
# `sent` stays the signal the caller asked for; the
# escalation is reported on its own, so the two are not
# confused for one another.
# Give the kernel a moment to deliver SIGKILL before the
# sweep decides what is left.
with contextlib.suppress(TimeoutError):
await asyncio.wait_for(exited.wait(), timeout=1.0)
elif verdict.unresolved:
# Nothing is signalled, and the caller is *told* so: the pid
# cannot be claimed as this session's, so a kill aimed at it
# would be aimed at a stranger. `kill` raises on this.
refused = verdict.reason
session.last_error = verdict.reason
if refused is not None:
# The recorded session id may now belong to something else
# entirely, so the sweep would be aiming at strangers. Skip it.
stragglers: list[int] = []
else:
stragglers = procmon.sweep_session(session.sid, exclude={os.getpid()})
# What is left standing. The sweep has just sent SIGKILL, so give
# the kernel a bounded moment to act on it before calling anything a
# survivor: a pid that is still here after this is a real failure to
# kill, and the caller has to hear about it rather than read
# "killed" and assume. On a refusal nothing was signalled and the
# pids involved may not be ours at all, so there is nothing to call
# a survivor: `refused` is the thing the caller has to hear.
survivors = [] if refused else await self._survivors(session)
if refused is None:
state_now = state
else:
# Not killed — the daemon never signalled anything. Recording
# KILLED here would be the same lie in the registry that the CLI
# was telling on stdout.
state_now = State.EXITED
self.registry.terminate(
session,
state_now,
exit_detected_by=session.exit_detected_by
or ("identity_refused" if refused else "killpg"),
last_error=session.last_error,
)
self.launcher.discard_home(session)
self.audit.emit(
f"{reason}_completed",
session=session.id,
app=session.app,
escalated_to_sigkill=escalated,
stragglers=len(stragglers),
straggler_pids=stragglers,
signalled=verdict.ok,
signal_sent=sent,
pgid=session.pgid,
refused=refused,
survivors=survivors,
identity=verdict.reason or "verified",
cmdline_drifted=verdict.cmdline_drifted,
idle_s=None if idle_s is None else round(idle_s, 3),
)
if survivors:
log.error(
"%s: %d process(es) survived the teardown: %s",
session.id,
len(survivors),
survivors,
)
self.reaper.wake()
return {
"reason": reason,
"signalled": verdict.ok,
#: The signal actually delivered, or ``None`` if none was.
"signal_sent": sent,
"pgid": session.pgid,
"escalated_to_sigkill": escalated,
"stragglers": len(stragglers),
"straggler_pids": stragglers,
#: Still alive after everything. Non-empty means the teardown
#: did not do what it was asked to do.
"survivors": survivors,
#: ``"verified"``, or the reason the gate refused.
"identity": verdict.reason or "verified",
"identity_checked": list(verdict.checked),
"identity_mismatched": list(verdict.mismatched),
#: The recorded argv no longer matches the live one. Normal for
#: an app behind a wrapper script; never a reason to refuse.
"cmdline_drifted": verdict.cmdline_drifted,
#: Set when the daemon refused to signal at all (BUG-02 item 5).
"refused": refused,
}
finally:
self._busy.discard(session.id)
async def adopt(
self,
pid: int,
*,
app: str | None = None,
idle_timeout_s: float | None = None,
) -> dict[str, Any]:
"""Bring an already-running process under management.
The same machinery the daemon uses to re-adopt its own sessions after a
restart, exposed as a verb — and the bridge from an unmanaged launch
into management.
"""
try:
pid = int(pid)
except (TypeError, ValueError):
raise ManagerError("invalid_argument", f"not a pid: {pid!r}") from None
if pid <= 1:
raise ManagerError("refused", f"refusing to adopt pid {pid}")
if pid == os.getpid():
raise ManagerError("refused", "refusing to adopt the daemon itself")
for existing in self.registry.live():
if existing.pid == pid:
raise ManagerError(
"already_managed",
f"pid {pid} is already session {existing.id}",
id=existing.id,
)
identity = procmon.identity(pid)
if identity is None:
raise ManagerError("no_such_process", f"no process with pid {pid}")
argv = procmon.cmdline_argv(pid) or []
if app is None:
app = Path(argv[0]).name if argv else f"pid{pid}"
try:
check_app_name(app)
except PolicyError as exc:
raise ManagerError(exc.error, exc.reason, exc.hint) from exc
self.config.policy.check_capacity(len(self.registry.live()))
timeout = self.config.policy.check_idle_timeout(idle_timeout_s)
now = self.now()
session_id = self.registry.new_id(app)
session = Session(
id=session_id,
app=app,
argv=argv,
pid=pid,
pgid=identity.pgid,
sid=identity.sid,
pid_starttime=identity.starttime,
cmdline_sha256=identity.cmdline_sha256,
home=os.environ.get("HOME", "/"),
runtime_dir=str(self.launcher.session_runtime_dir(session_id)),
log_path=None, # the daemon did not spawn it, so it captured no output
created_at=iso(self.clock.wall()),
created_boottime=now,
last_activity_at=iso(self.clock.wall()),
last_activity_boottime=now,
idle_timeout_s=timeout,
state=State.READY,
ready_signal="adopted",
adopted=True,
command_count=1,
)
self.registry.add(session)
self._exited.pop(session_id, None)
self._watch(session)
self.audit.emit(
"session_adopted", session=session.id, app=app, pid=pid, idle_timeout_s=timeout
)
self.reaper.wake()
return session.public(self.now())
def gc(self) -> dict[str, Any]:
"""Prune tombstones and reconcile the table with ``/proc``.
Read-only with respect to the idle clock: **``gc`` does not count as
activity.**
"""
now = self.now()
reconciled: list[dict[str, Any]] = []
#: Reconciled sessions whose pid could not be resolved either way.
#: Reported separately so a caller can tell "it exited" (routine)
#: from "the daemon has lost track of a running application".
unresolved: list[dict[str, Any]] = []
for session in self.registry.live():
verdict = self._identity_of(session)
if verdict.ok:
# Including the wrapper case: a session whose argv changed after
# launch is healthy, and `gc` used to tombstone it while its
# application kept running (BUG-02 item 4).
continue
why = verdict.reason
self.registry.terminate(
session,
State.EXITED,
exit_detected_by="gc_proc_scan",
last_error=None if why == procmon.GONE else why,
)
entry = {"id": session.id, "why": why}
if verdict.unresolved:
entry["mismatched"] = list(verdict.mismatched)
unresolved.append(entry)
reconciled.append(entry)
self.audit.emit(
"gc_reconciled",
session=session.id,
why=why,
mismatched=list(verdict.mismatched),
)
orphans: list[dict[str, Any]] = []
for session in self.registry.tombstones():
survivors = procmon.session_members(session.sid, exclude={os.getpid()})
if survivors:
# Processes still sharing a dead session's session id: the
# double-`setsid()` leak the design admits to (§7 #7).
orphans.append({"id": session.id, "sid": session.sid, "pids": survivors})
pruned = self._prune_tombstones()
self.reaper.wake()
self.audit.emit(
"gc",
pruned=pruned,
reconciled=len(reconciled),
unresolved=len(unresolved),
orphans=len(orphans),
)
return {
"pruned": pruned,
"reconciled": reconciled,
"unresolved": unresolved,
"orphans": orphans,
"live": len(self.registry.live()),
"tombstones": len(self.registry.tombstones()),
"boottime": now,
}
def _prune_tombstones(self) -> int:
now = self.now()
pruned = 0
for session in self.registry.tombstones():
expires = session.tombstone_expires()
if expires is None or expires > now:
continue
self.launcher.discard_home(session)
self.launcher.discard_runtime_dir(session)
self.registry.drop(session.id)
self._exited.pop(session.id, None)
self._watchers.pop(session.id, None)
pruned += 1
return pruned
def logs(self, session_id: str, *, lines: int = 100, offset: int | None = None) -> dict[str, Any]:
"""The daemon's capture of the app's stdout/stderr.
Served from ``<state>/logs/<id>.log`` rather than the journal, because
apps are ordinary child processes here. Read-only: **does not** count
as activity. ``offset`` supports ``--follow`` without the daemon
holding a streaming connection open.
"""
session = self._require(session_id)
if not session.log_path:
return {
"id": session.id,
"path": None,
"lines": [],
"offset": 0,
"eof": True,
"note": "adopted sessions have no captured output",
}
path = Path(session.log_path)
try:
size = path.stat().st_size
except OSError:
return {"id": session.id, "path": str(path), "lines": [], "offset": 0, "eof": True}
if offset is None:
tail, truncated, size = tail_lines(path, max(0, int(lines)))
return {
"id": session.id,
"path": str(path),
"lines": tail,
"offset": size,
"truncated": truncated,
# `eof` means "no more output will ever arrive", which is true
# exactly when the session has ended. A follower polls until it.
"eof": not session.live,
}
offset = max(0, min(int(offset), size))
try:
with open(path, "rb") as fh:
fh.seek(offset)
chunk = fh.read(MAX_LOG_CHUNK_BYTES)
new_offset = fh.tell()
except OSError as exc:
raise ManagerError("refused", f"could not read logs: {exc}") from exc
return {
"id": session.id,
"path": str(path),
"lines": chunk.decode("utf-8", "replace").splitlines(),
"offset": new_offset,
"truncated": new_offset < size,
"eof": not session.live and new_offset >= size,
}
def _signal_number(sig: str | int) -> int:
if isinstance(sig, int):
try:
return signal.Signals(sig).value
except ValueError:
raise ManagerError("invalid_argument", f"unknown signal: {sig!r}") from None
name = str(sig).upper()
if name in SIGNALS_BY_NAME:
return SIGNALS_BY_NAME[name]
raise ManagerError(
"invalid_argument",
f"unknown signal: {sig!r}",
hint=f"one of {', '.join(sorted(set(SIGNALS_BY_NAME)))}",
)
+237
View File
@@ -0,0 +1,237 @@
"""The app allowlist and the input bounds on ``start`` (DESIGN.md §8 item 1).
This is the main control, and the only part of §8 that is in M0:
* ``start`` resolves an app *name* against the config file to an **absolute
path**. An arbitrary binary is refused with exit 6. Without this,
``hhstream session start /bin/sh`` makes every other control decorative.
* ``argv`` is a **list**. There is never a shell: no ``shell=True``, no string
interpolation into a command line, anywhere in this package.
* Caller-supplied arguments are matched against ``args_allow`` patterns. An
app with no ``args_allow`` accepts no caller arguments at all.
Deliberately **not** in M0 (brief: "nothing else from §8"): the ``SO_PEERCRED``
uid/gid authorisation, per-session ``HOME`` by default, ``nice``,
``oom_score_adj``, rlimits, the RSS watchdog and the keybinding denylist.
``max_sessions`` is kept, because exit code 6 in the frozen table names it.
"""
from __future__ import annotations
import fnmatch
import logging
import os
import tomllib
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
log = logging.getLogger("hhstreamd.policy")
DEFAULT_IDLE_TIMEOUT_S = 3 * 3600.0
DEFAULT_MAX_IDLE_TIMEOUT_S = 24 * 3600.0
DEFAULT_MAX_SESSIONS = 4
DEFAULT_GRACE_S = 10.0
DEFAULT_WAIT_READY_S = 15.0
# Input bounds that apply to `start`. They exist so a malformed or hostile
# request is refused cheaply, before anything is spawned.
MAX_APP_NAME = 64
MAX_ARGV_ITEMS = 64
MAX_ARG_BYTES = 4096
MAX_ARGV_BYTES = 64 * 1024
APP_NAME_CHARS = set("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789._-+")
class PolicyError(Exception):
"""A request refused by policy. ``error`` maps to an exit code."""
def __init__(self, error: str, reason: str, hint: str | None = None) -> None:
super().__init__(reason)
self.error = error
self.reason = reason
self.hint = hint
@dataclass(frozen=True, slots=True)
class AppRule:
name: str
exec: str
args: list[str] = field(default_factory=list)
args_allow: list[str] = field(default_factory=list)
idle_timeout_s: float | None = None
def check_args(self, argv: list[str]) -> None:
for arg in argv:
if not any(fnmatch.fnmatchcase(arg, pattern) for pattern in self.args_allow):
raise PolicyError(
"argument_refused",
f"argument {arg!r} is not permitted for app {self.name!r}",
hint="args_allow in the hhstream config controls this",
)
@dataclass(frozen=True, slots=True)
class Policy:
apps: dict[str, AppRule] = field(default_factory=dict)
max_sessions: int = DEFAULT_MAX_SESSIONS
idle_timeout_s: float = DEFAULT_IDLE_TIMEOUT_S
max_idle_timeout_s: float = DEFAULT_MAX_IDLE_TIMEOUT_S
grace_s: float = DEFAULT_GRACE_S
wait_ready_s: float = DEFAULT_WAIT_READY_S
source: str | None = None
# ---- resolution --------------------------------------------------
def resolve(self, app: str, argv: list[str] | None) -> tuple[AppRule, list[str]]:
"""Resolve an app name and caller argv to the full argv to execute.
Returns ``(rule, argv)`` where ``argv[0]`` is the rule's absolute exec
path. Raises :class:`PolicyError` for anything not allowlisted.
"""
check_app_name(app)
argv = list(argv or [])
check_argv(argv)
rule = self.apps.get(app)
if rule is None:
known = ", ".join(sorted(self.apps)) or "(the allowlist is empty)"
raise PolicyError(
"not_allowlisted",
f"app {app!r} is not in the allowlist",
hint=f"allowlisted apps: {known}; see {self.source or 'the hhstream config'}",
)
rule.check_args(argv)
return rule, [rule.exec, *rule.args, *argv]
def check_idle_timeout(self, seconds: float | None) -> float:
if seconds is None:
return self.idle_timeout_s
try:
value = float(seconds)
except (TypeError, ValueError):
raise PolicyError("invalid_argument", f"idle timeout is not a number: {seconds!r}") from None
if value < 0:
raise PolicyError("invalid_argument", "idle timeout may not be negative")
if value > self.max_idle_timeout_s:
raise PolicyError(
"refused",
f"idle timeout {value:g}s exceeds the configured maximum "
f"{self.max_idle_timeout_s:g}s",
)
return value
def check_capacity(self, live_count: int) -> None:
if self.max_sessions and live_count >= self.max_sessions:
raise PolicyError(
"session_cap_reached",
f"{live_count} live sessions, max_sessions is {self.max_sessions}",
hint="kill or gc a session, or raise max_sessions in the config",
)
def check_app_name(app: Any) -> None:
if not isinstance(app, str) or not app:
raise PolicyError("invalid_argument", "app name must be a non-empty string")
if len(app) > MAX_APP_NAME:
raise PolicyError("invalid_argument", f"app name longer than {MAX_APP_NAME} characters")
bad = set(app) - APP_NAME_CHARS
if bad:
raise PolicyError(
"invalid_argument",
f"app name contains disallowed characters: {''.join(sorted(bad))!r}",
)
def check_argv(argv: Any) -> None:
"""Bounds on the caller-supplied argv. A list of strings, and not a large one."""
if not isinstance(argv, list):
raise PolicyError("invalid_argument", "argv must be a list of strings")
if len(argv) > MAX_ARGV_ITEMS:
raise PolicyError("invalid_argument", f"argv has more than {MAX_ARGV_ITEMS} items")
total = 0
for arg in argv:
if not isinstance(arg, str):
raise PolicyError("invalid_argument", f"argv item is not a string: {arg!r}")
if "\0" in arg:
raise PolicyError("invalid_argument", "argv item contains a NUL byte")
encoded = len(arg.encode("utf-8"))
if encoded > MAX_ARG_BYTES:
raise PolicyError("invalid_argument", f"argv item longer than {MAX_ARG_BYTES} bytes")
total += encoded
if total > MAX_ARGV_BYTES:
raise PolicyError("invalid_argument", f"argv longer than {MAX_ARGV_BYTES} bytes in total")
# --------------------------------------------------------------------------
# Config loading
# --------------------------------------------------------------------------
def load_policy(path: Path | str | None) -> Policy:
"""Read the allowlist from a TOML file.
A missing file is not an error: it yields an empty allowlist, so the daemon
starts and every ``start`` is refused with exit 6 and a hint naming the file
it looked for. Refusing to start would make a misconfigured deployment look
like a crash.
"""
if path is None:
return Policy(source=None)
path = Path(path)
if not path.exists():
log.warning("no config at %s: the app allowlist is empty, every start will be refused", path)
return Policy(source=str(path))
with open(path, "rb") as fh:
data = tomllib.load(fh)
return policy_from_dict(data, source=str(path))
def policy_from_dict(data: dict[str, Any], *, source: str | None = None) -> Policy:
defaults = data.get("defaults", {})
if not isinstance(defaults, dict):
raise PolicyError("invalid_argument", "[defaults] must be a table")
apps: dict[str, AppRule] = {}
raw_apps = data.get("apps", {})
if not isinstance(raw_apps, dict):
raise PolicyError("invalid_argument", "[apps] must be a table")
for name, spec in raw_apps.items():
apps[name] = _app_rule(name, spec)
return Policy(
apps=apps,
max_sessions=int(defaults.get("max_sessions", DEFAULT_MAX_SESSIONS)),
idle_timeout_s=float(defaults.get("idle_timeout_s", DEFAULT_IDLE_TIMEOUT_S)),
max_idle_timeout_s=float(defaults.get("max_idle_timeout_s", DEFAULT_MAX_IDLE_TIMEOUT_S)),
grace_s=float(defaults.get("grace_s", DEFAULT_GRACE_S)),
wait_ready_s=float(defaults.get("wait_ready_s", DEFAULT_WAIT_READY_S)),
source=source,
)
def _app_rule(name: str, spec: Any) -> AppRule:
if not isinstance(spec, dict):
raise PolicyError("invalid_argument", f"[apps.{name}] must be a table")
check_app_name(name)
exec_path = spec.get("exec")
if not isinstance(exec_path, str) or not exec_path:
raise PolicyError("invalid_argument", f"[apps.{name}] needs an 'exec' string")
if not os.path.isabs(exec_path):
raise PolicyError(
"invalid_argument",
f"[apps.{name}].exec must be an absolute path, got {exec_path!r}",
)
args = spec.get("args", [])
args_allow = spec.get("args_allow", [])
for key, value in (("args", args), ("args_allow", args_allow)):
if not isinstance(value, list) or not all(isinstance(v, str) for v in value):
raise PolicyError("invalid_argument", f"[apps.{name}].{key} must be a list of strings")
idle = spec.get("idle_timeout_s")
return AppRule(
name=name,
exec=exec_path,
args=list(args),
args_allow=list(args_allow),
idle_timeout_s=None if idle is None else float(idle),
)
+406
View File
@@ -0,0 +1,406 @@
"""``/proc`` inspection: liveness, identity, and the session-id sweep.
This is the substitute for the cgroup membership the design lost when the
daemon stopped being a systemd *user* service (DESIGN.md §2.4, §3.3):
* **Liveness** is ``/proc/<pid>`` existing.
* **Identity** is the evidence that makes it safe to signal a pid the daemon
did not spawn — see :func:`check` and the note below.
* **The sweep** finds processes whose session id matches a recorded session,
which catches a child that called ``setsid()`` once and so escaped the
process group. A child that calls ``setsid()`` twice escapes even this; that
is the honest cost of having no cgroup, and ``gc`` reports it.
**Why the command line is not identity evidence (BUG-02).** The original
triple-check hashed ``/proc/<pid>/cmdline`` and refused to signal when the hash
moved. ``/proc/<pid>/cmdline`` is not a property of the *process*; it is a
window onto the current image's argument vector, and it is replaced wholesale by
every ``execve()``. On the target platform ``/run/current-system/sw/bin/<app>``
is almost always a wrapper script that ``exec``s the real binary, which arrives
with an argv of its own; Chromium then rewrites argv in place for its zygote and
renderer processes on top of that. So a perfectly healthy session was being
classified as a recycled pid, and ``kill``, ``gc`` and restart adoption all
quietly did the wrong thing. The hash is still recorded and still reported —
as *drift*, which is useful in a post-mortem — but it never gates a signal.
**What is left, and why it is sound.**
* ``starttime`` (``stat`` field 22) is written once by the kernel when the task
is created and is never writable afterwards, by the process or by anyone
else. ``execve()`` replaces the address space, not the task, so it survives
an arbitrarily long chain of wrappers. Two processes holding the same pid are
necessarily created at different times, so a difference here is proof of
reuse. The one residual window is the clock-tick granularity of the field
(``USER_HZ``, 10 ms on this kernel): a pid could in principle be recycled
within the same tick, which would require the kernel to wrap the whole pid
space — ``/proc/sys/kernel/pid_max``, four million by default — in under 10 ms.
``boot_id`` (see :mod:`hhstreamd.registry`) closes the across-reboot case
separately, so this is the entire remaining exposure and it is accepted
knowingly. **This field is mandatory**: without it there is no identity, and
the caller is refused rather than silently obliged.
* ``sid`` and ``pgid`` are inherited across ``execve()`` and survive a wrapper
just as well, but they are only *immutable* for a session/group leader: a
leader's ``setsid()`` and ``setpgid()`` both fail with ``EPERM``, whereas an
ordinary process may lawfully leave at any moment. Every session the daemon
spawns is a leader by construction (``start_new_session=True`` gives
``sid == pgid == pid``), so the check is applied exactly when the recorded
value equals the recorded pid and skipped otherwise — an *adopted* mid-session
process that later calls ``setsid()`` must not be mistaken for a stranger.
They corroborate ``starttime``; they are never trusted alone, because a
recycled pid that calls ``setsid()`` acquires ``sid == pgid == pid`` too.
"""
from __future__ import annotations
import errno
import hashlib
import os
import signal
from dataclasses import dataclass
PROC = "/proc"
@dataclass(frozen=True, slots=True)
class ProcStat:
"""The fields of ``/proc/<pid>/stat`` this daemon cares about."""
pid: int
comm: str
state: str
ppid: int
pgid: int
sid: int
starttime: int
def read_stat(pid: int, *, proc: str = PROC) -> ProcStat | None:
"""Parse ``/proc/<pid>/stat``. ``None`` if the process is gone.
``comm`` may contain spaces and parentheses, so the fields after it are
located from the *last* ``)`` rather than by splitting the whole line.
"""
try:
with open(f"{proc}/{pid}/stat", "rb") as fh:
raw = fh.read()
except (FileNotFoundError, ProcessLookupError, NotADirectoryError):
return None
except OSError as exc: # ESRCH racing with exit, EACCES on a hardened /proc
if exc.errno in (errno.ESRCH, errno.EACCES, errno.EPERM):
return None
raise
open_paren = raw.find(b"(")
close = raw.rfind(b")")
if open_paren == -1 or close < open_paren:
return None
comm = raw[open_paren + 1 : close].decode("utf-8", "replace")
rest = raw[close + 2 :].split()
# `rest` begins at field 3 (state), so field N lives at index N - 3.
if len(rest) < 20:
return None
try:
return ProcStat(
pid=int(raw[:open_paren].strip()),
comm=comm,
state=rest[0].decode(),
ppid=int(rest[1]),
pgid=int(rest[2]),
sid=int(rest[3]),
starttime=int(rest[19]),
)
except (ValueError, IndexError):
return None
def read_cmdline(pid: int, *, proc: str = PROC) -> bytes | None:
"""Raw NUL-separated ``/proc/<pid>/cmdline``. ``None`` if the process is gone."""
try:
with open(f"{proc}/{pid}/cmdline", "rb") as fh:
return fh.read()
except (FileNotFoundError, ProcessLookupError, NotADirectoryError):
return None
except OSError as exc:
if exc.errno in (errno.ESRCH, errno.EACCES, errno.EPERM):
return None
raise
def cmdline_argv(pid: int, *, proc: str = PROC) -> list[str] | None:
"""``/proc/<pid>/cmdline`` split into an argv list."""
raw = read_cmdline(pid, proc=proc)
if raw is None:
return None
parts = raw.split(b"\0")
if parts and parts[-1] == b"":
parts.pop()
return [p.decode("utf-8", "replace") for p in parts]
def hash_cmdline(raw: bytes) -> str:
"""The ``cmdline_sha256`` of DESIGN.md §2.1."""
return hashlib.sha256(raw).hexdigest()
def cmdline_hash(pid: int, *, proc: str = PROC) -> str | None:
raw = read_cmdline(pid, proc=proc)
return None if raw is None else hash_cmdline(raw)
#: `/proc/<pid>/stat` states for a process that has already terminated. A
#: zombie still has a `/proc` entry — it holds its pid until its real parent
#: reaps it — but it will never run again and cannot be signalled into doing
#: anything. Treating it as alive would make the daemon escalate to SIGKILL
#: against a corpse and report sessions as running after they ended, which is
#: the common case for an *adopted* process whose real parent is slow to wait().
DEAD_STATES = frozenset("ZX")
def is_alive(pid: int, *, proc: str = PROC) -> bool:
"""True if the process exists and has not already terminated."""
stat = read_stat(pid, proc=proc)
return stat is not None and stat.state not in DEAD_STATES
@dataclass(frozen=True, slots=True)
class Identity:
"""The evidence recorded at spawn and re-checked after a daemon restart."""
pid: int
pgid: int
sid: int
starttime: int
cmdline_sha256: str
def identity(pid: int, *, proc: str = PROC) -> Identity | None:
"""Capture the anti-pid-reuse evidence for a live process."""
stat = read_stat(pid, proc=proc)
if stat is None:
return None
raw = read_cmdline(pid, proc=proc)
if raw is None:
return None
return Identity(
pid=pid,
pgid=stat.pgid,
sid=stat.sid,
starttime=stat.starttime,
cmdline_sha256=hash_cmdline(raw),
)
#: Reasons :func:`check` can refuse. ``gone`` is routine — the process simply
#: exited. The other two are not: ``pid_reuse_detected`` means a *different*
#: process now holds the pid, and ``identity_unknown`` means the record carries
#: no usable evidence either way. A caller may never signal on any of them, and
#: must never report the latter two as a quiet success (BUG-02).
GONE = "gone"
PID_REUSE = "pid_reuse_detected"
IDENTITY_UNKNOWN = "identity_unknown"
#: Refusals the caller has to be *loud* about: both mean the daemon could not
#: establish that the pid is still the session's, so an operation the caller
#: asked for did not happen.
UNRESOLVED = frozenset({PID_REUSE, IDENTITY_UNKNOWN})
@dataclass(frozen=True, slots=True)
class Verdict:
"""The outcome of an identity check, with the evidence that produced it.
``checked`` and ``mismatched`` are what makes this falsifiable in a log or a
test: they name the fields that actually gated the decision, so weakening
the gate cannot be done silently.
"""
ok: bool
reason: str | None
#: Fields that were in a position to refuse, in the order they were tried.
checked: tuple[str, ...] = ()
#: The subset of ``checked`` that did not match.
mismatched: tuple[str, ...] = ()
#: The recorded ``cmdline_sha256`` no longer matches the live one. This is
#: *reported*, never acted on: see the module docstring.
cmdline_drifted: bool = False
@property
def unresolved(self) -> bool:
"""True when the identity could be neither confirmed nor ruled out."""
return self.reason in UNRESOLVED
def check(
pid: int,
*,
starttime: int | None,
sid: int | None = None,
pgid: int | None = None,
cmdline_sha256: str | None = None,
proc: str = PROC,
) -> Verdict:
"""Decide whether ``pid`` is still the process whose identity was recorded.
The gate is ``starttime``, corroborated by ``sid`` and ``pgid`` when the
recorded process was its own session/group leader — which every session the
daemon spawns is. ``cmdline_sha256`` is accepted and compared, but a
difference only sets :attr:`Verdict.cmdline_drifted`; it never refuses,
because an ``exec``ing wrapper script changes it as a matter of routine
(BUG-02). The reasoning for each field is in the module docstring.
"""
stat = read_stat(pid, proc=proc)
if stat is None or stat.state in DEAD_STATES:
return Verdict(False, GONE)
drifted = False
if cmdline_sha256:
live_hash = cmdline_hash(pid, proc=proc)
drifted = live_hash is not None and live_hash != cmdline_sha256
if not starttime or starttime <= 0:
# No evidence at all. Refusing is the only safe answer, and the caller
# is required to say so rather than skip whatever it was asked to do.
return Verdict(False, IDENTITY_UNKNOWN, cmdline_drifted=drifted)
checked = ["starttime"]
mismatched = [] if stat.starttime == starttime else ["starttime"]
# `sid`/`pgid` are immutable only for a leader; see the module docstring.
if sid is not None and sid == pid:
checked.append("sid")
if stat.sid != sid:
mismatched.append("sid")
if pgid is not None and pgid == pid:
checked.append("pgid")
if stat.pgid != pgid:
mismatched.append("pgid")
if mismatched:
return Verdict(
False, PID_REUSE, tuple(checked), tuple(mismatched), cmdline_drifted=drifted
)
return Verdict(True, None, tuple(checked), (), cmdline_drifted=drifted)
def verify(
pid: int,
*,
starttime: int | None,
sid: int | None = None,
pgid: int | None = None,
cmdline_sha256: str | None = None,
proc: str = PROC,
) -> tuple[bool, str | None]:
""":func:`check` reduced to ``(ok, reason)`` for callers that want no detail."""
verdict = check(
pid,
starttime=starttime,
sid=sid,
pgid=pgid,
cmdline_sha256=cmdline_sha256,
proc=proc,
)
return verdict.ok, verdict.reason
def iter_pids(*, proc: str = PROC) -> list[int]:
try:
entries = os.listdir(proc)
except OSError:
return []
return sorted(int(e) for e in entries if e.isdigit())
def session_members(sid: int, *, proc: str = PROC, exclude: set[int] | None = None) -> list[int]:
"""Every live pid whose ``/proc/<pid>/stat`` session id equals ``sid``.
Refuses ``sid <= 1``: sweeping session 0 or 1 would select init and most of
the machine.
"""
if sid is None or sid <= 1:
return []
exclude = exclude or set()
found = []
for pid in iter_pids(proc=proc):
if pid in exclude:
continue
stat = read_stat(pid, proc=proc)
if stat is not None and stat.sid == sid and stat.state not in DEAD_STATES:
found.append(pid)
return found
def kill_pid(pid: int, sig: int = signal.SIGKILL) -> bool:
"""Signal one pid, tolerating the race where it has already exited."""
try:
os.kill(pid, sig)
return True
except ProcessLookupError:
return False
except PermissionError:
return False
def killpg(pgid: int, sig: int) -> bool:
"""Signal a whole process group, refusing the dangerous group ids.
``pgid <= 1`` would mean "every process we may signal" or init's group;
neither is ever a session's group, because every session is spawned with
``start_new_session=True`` and so has ``pgid == pid``.
"""
if pgid is None or pgid <= 1:
return False
try:
os.killpg(pgid, sig)
return True
except ProcessLookupError:
return False
except PermissionError:
return False
def sweep_session(
sid: int, *, proc: str = PROC, exclude: set[int] | None = None, sig: int = signal.SIGKILL
) -> list[int]:
"""Kill every straggler sharing the session id. Returns the pids signalled."""
victims = session_members(sid, proc=proc, exclude=exclude)
killed = [pid for pid in victims if kill_pid(pid, sig)]
return killed
# --------------------------------------------------------------------------
# Exit notification
# --------------------------------------------------------------------------
#: ``os.pidfd_open`` needs Linux >= 5.3 and gives exit notification for
#: processes that are *not* our children — which is exactly the adopted case.
#: Where it is unavailable the daemon falls back to a 1 s ``/proc`` liveness
#: poll, which costs one ``stat`` per adopted session per second and detects an
#: exit up to a second late. Own children never need either path: asyncio's
#: child watcher gives both notification and the exit status.
HAVE_PIDFD = hasattr(os, "pidfd_open")
#: How often the fallback polls, in seconds.
POLL_INTERVAL_S = 1.0
def pidfd_open(pid: int) -> int | None:
"""Open a pidfd, or ``None`` if unsupported or the process is already gone."""
if not HAVE_PIDFD:
return None
try:
return os.pidfd_open(pid)
except (ProcessLookupError, PermissionError, OSError):
return None
+186
View File
@@ -0,0 +1,186 @@
"""Wire format and the exit-code table (DESIGN.md §1.6, §3.4).
Newline-delimited JSON over a unix ``SOCK_STREAM`` socket: one request object,
one response object, connection closed. This module is shared by the daemon
and by the CLI handlers, so the framing and the exit-code mapping exist exactly
once. It imports nothing outside the standard library and nothing from the
rest of ``hhstreamd``.
"""
from __future__ import annotations
import enum
import json
from typing import Any
PROTOCOL_VERSION = 1
#: A single request or response line may not exceed this many bytes, including
#: the terminating newline. A peer that sends more is answered with
#: ``message_too_large`` and disconnected rather than being allowed to grow the
#: daemon's read buffer without bound.
MAX_LINE_BYTES = 1 << 20 # 1 MiB
class ProtocolError(Exception):
"""A frame could not be read or decoded.
``error`` is the machine-readable name reported to the peer.
"""
def __init__(self, error: str, reason: str) -> None:
super().__init__(reason)
self.error = error
self.reason = reason
# --------------------------------------------------------------------------
# Exit codes (DESIGN.md §1.6). Frozen: agents branch on these.
# --------------------------------------------------------------------------
class ExitCode(enum.IntEnum):
OK = 0
USAGE = 2
NO_SUCH_SESSION = 3
BAD_STATE = 4
DAEMON_UNREACHABLE = 5
POLICY = 6
TIMEOUT = 7
BUSY = 8
#: Machine-readable error names, grouped by the exit code they map to. Every
#: ``{"ok": false, "error": ...}`` the daemon can emit appears here; anything
#: unknown falls back to :data:`ExitCode.BAD_STATE` (see :func:`exit_code_for`).
EXIT_CODE_FOR_ERROR: dict[str, ExitCode] = {
# 2 — usage
"bad_request": ExitCode.USAGE,
"unknown_op": ExitCode.USAGE,
"unsupported_version": ExitCode.USAGE,
"message_too_large": ExitCode.USAGE,
"invalid_argument": ExitCode.USAGE,
# 3 — no such session
"no_such_session": ExitCode.NO_SUCH_SESSION,
"no_such_process": ExitCode.NO_SUCH_SESSION,
# 4 — not in a usable state
"session_exited": ExitCode.BAD_STATE,
"session_killed": ExitCode.BAD_STATE,
"session_reaped": ExitCode.BAD_STATE,
"session_starting": ExitCode.BAD_STATE,
"compositor_lost": ExitCode.BAD_STATE,
"bad_state": ExitCode.BAD_STATE,
"pid_reuse_detected": ExitCode.BAD_STATE,
# The session's identity could not be established either way, so the
# daemon refused rather than signalling blind (BUG-02 item 5).
"identity_unknown": ExitCode.BAD_STATE,
# 5 — daemon unreachable (produced by the client, never by the daemon)
"daemon_unreachable": ExitCode.DAEMON_UNREACHABLE,
# 6 — refused by policy
"not_allowlisted": ExitCode.POLICY,
"argument_refused": ExitCode.POLICY,
"session_cap_reached": ExitCode.POLICY,
"refused": ExitCode.POLICY,
# 7 — timed out
"timeout": ExitCode.TIMEOUT,
"readiness_timeout": ExitCode.TIMEOUT,
# 8 — busy
"busy": ExitCode.BUSY,
"already_managed": ExitCode.BUSY,
}
def exit_code_for(error: str | None) -> ExitCode:
"""Map a machine-readable error name to its frozen exit code.
An unrecognised name maps to :data:`ExitCode.BAD_STATE`: a caller that does
not know what went wrong should still be told the session is not usable,
which is strictly safer than reporting success.
"""
if not error:
return ExitCode.OK
return EXIT_CODE_FOR_ERROR.get(error, ExitCode.BAD_STATE)
# --------------------------------------------------------------------------
# Message construction
# --------------------------------------------------------------------------
def request(op: str, **fields: Any) -> dict[str, Any]:
"""Build a request object with the protocol version attached."""
msg: dict[str, Any] = {"v": PROTOCOL_VERSION, "op": op}
msg.update({k: v for k, v in fields.items() if v is not None})
return msg
def ok(**fields: Any) -> dict[str, Any]:
return {"ok": True, **fields}
def error(err: str, reason: str, *, hint: str | None = None, **fields: Any) -> dict[str, Any]:
"""Build an error reply.
``error`` is the machine field agents branch on, ``reason`` the human one
(DESIGN.md §1.6).
"""
msg: dict[str, Any] = {"ok": False, "error": err, "reason": reason}
if hint is not None:
msg["hint"] = hint
msg.update(fields)
return msg
# --------------------------------------------------------------------------
# Framing
# --------------------------------------------------------------------------
def encode(message: dict[str, Any]) -> bytes:
"""Serialise one message to a single newline-terminated line.
``ensure_ascii`` keeps the line free of raw newlines inside strings by
construction, so the framing cannot be broken by a session id, an argv
element or an error message.
"""
line = json.dumps(message, ensure_ascii=True, separators=(",", ":")).encode("ascii")
if len(line) + 1 > MAX_LINE_BYTES:
raise ProtocolError("message_too_large", f"message is {len(line)} bytes")
return line + b"\n"
def decode(line: bytes) -> dict[str, Any]:
"""Parse one frame (with or without its trailing newline) into a dict."""
if len(line) > MAX_LINE_BYTES:
raise ProtocolError("message_too_large", f"frame is {len(line)} bytes")
text = line.rstrip(b"\n")
if not text.strip():
raise ProtocolError("bad_request", "empty frame")
try:
obj = json.loads(text)
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise ProtocolError("bad_request", f"malformed JSON: {exc}") from exc
if not isinstance(obj, dict):
raise ProtocolError("bad_request", f"expected a JSON object, got {type(obj).__name__}")
return obj
def decode_request(line: bytes) -> dict[str, Any]:
"""Parse and validate a request frame: an object with a known ``v`` and ``op``."""
obj = decode(line)
version = obj.get("v", PROTOCOL_VERSION)
if version != PROTOCOL_VERSION:
raise ProtocolError(
"unsupported_version",
f"protocol version {version!r} is not supported (daemon speaks {PROTOCOL_VERSION})",
)
op = obj.get("op")
if not isinstance(op, str) or not op:
raise ProtocolError("bad_request", "request has no 'op' field")
return obj
+187
View File
@@ -0,0 +1,187 @@
"""The idle reaper (DESIGN.md §6).
Deadline-scheduled, not polled: the reaper computes
``min(last_activity_boottime + idle_timeout_s)`` across live sessions and
sleeps until that instant or until a state change wakes it. Zero wakeups while
idle; no one-second tick burning a core on an otherwise quiet server.
**The clock is injected at construction** — ``now()`` returns a
``CLOCK_BOOTTIME`` reading in production and whatever a test says otherwise.
The selection arithmetic below is therefore pure and covered by unit tests that
never sleep and never spawn anything.
"""
from __future__ import annotations
import asyncio
import logging
from typing import Awaitable, Callable, Iterable, Protocol
log = logging.getLogger("hhstreamd.reaper")
class Expirable(Protocol):
"""What the reaper needs of a session. ``Session`` satisfies it."""
id: str
@property
def live(self) -> bool: ...
def deadline(self) -> float | None: ...
def deadlines(sessions: Iterable[Expirable]) -> dict[str, float]:
"""Every live session's deadline, keyed by id.
Sessions in a terminal state and sessions with the idle timeout disabled
(``idle_timeout_s == 0``) have no deadline and do not appear.
"""
out: dict[str, float] = {}
for session in sessions:
deadline = session.deadline()
if deadline is not None:
out[session.id] = deadline
return out
def next_deadline(sessions: Iterable[Expirable]) -> float | None:
"""The earliest deadline, or ``None`` when nothing can expire.
``None`` is the empty-set case and means "sleep until woken".
"""
found = deadlines(sessions)
return min(found.values()) if found else None
def select_expired(sessions: Iterable[Expirable], now: float) -> list[str]:
"""Ids whose deadline has arrived, oldest deadline first.
The comparison is ``deadline <= now`` so a session adopted after a daemon
restart with an already-expired deadline is selected on the first pass
rather than waiting another full timeout (DESIGN.md §2.4 item 6).
"""
expired = [(deadline, sid) for sid, deadline in deadlines(sessions).items() if deadline <= now]
expired.sort()
return [sid for _, sid in expired]
def sleep_for(sessions: Iterable[Expirable], now: float) -> float | None:
"""How long to sleep before the next deadline.
``None`` means "indefinitely"; ``0.0`` means something is already expired.
"""
deadline = next_deadline(sessions)
if deadline is None:
return None
return max(0.0, deadline - now)
class Reaper:
"""The deadline-scheduled task.
``sessions`` is a callable returning the current live set — the registry is
authoritative, and the reaper never holds its own copy of it.
``on_expire(session_id)`` performs the teardown; it is exactly ``kill`` with
a different reason, so there is one kill path, not two (DESIGN.md §6.3).
"""
def __init__(
self,
now: Callable[[], float],
sessions: Callable[[], Iterable[Expirable]],
on_expire: Callable[[str], Awaitable[None]],
*,
max_sleep: float = 3600.0,
retry_delay: float = 1.0,
) -> None:
self._now = now
self._sessions = sessions
self._on_expire = on_expire
self._max_sleep = max_sleep
self._retry_delay = retry_delay
self._wake = asyncio.Event()
self._task: asyncio.Task | None = None
self.reaped: list[str] = []
# ---- pure helpers, exposed for tests and for `status` -------------
def next_deadline(self) -> float | None:
return next_deadline(self._sessions())
def expired(self) -> list[str]:
return select_expired(self._sessions(), self._now())
def sleep_for(self) -> float | None:
return sleep_for(self._sessions(), self._now())
# ---- lifecycle ---------------------------------------------------
def wake(self) -> None:
"""Re-evaluate the schedule now. Called on every state change."""
self._wake.set()
def start(self) -> asyncio.Task:
if self._task is None:
self._task = asyncio.create_task(self.run(), name="hhstreamd.reaper")
return self._task
async def stop(self) -> None:
if self._task is not None:
self._task.cancel()
try:
await self._task
except asyncio.CancelledError:
pass
self._task = None
async def tick(self) -> list[str]:
"""Reap everything currently expired. Returns the ids reaped."""
reaped: list[str] = []
for session_id in self.expired():
try:
await self._on_expire(session_id)
except Exception: # pragma: no cover - teardown must not kill the task
log.exception("reaping %s failed", session_id)
else:
reaped.append(session_id)
self.reaped.extend(reaped)
return reaped
async def run(self) -> None:
while True:
# Cleared *before* the tick, so a state change that lands while we
# are tearing a session down still forces a re-evaluation instead
# of being swallowed.
self._wake.clear()
reaped = await self.tick()
delay = self.sleep_for()
# Nothing can expire: sleep until something wakes us. Zero
# wakeups on a quiet server, which is the point of scheduling on a
# deadline rather than ticking.
#
# With a deadline, the wait is additionally capped at `max_sleep`.
# That is not belt-and-braces: deadlines are on CLOCK_BOOTTIME but
# asyncio's timers run on CLOCK_MONOTONIC, which *stops* across
# system suspend. A host suspended for four hours would otherwise
# wake and keep waiting. Re-evaluating at most an hour of
# monotonic time later bounds that error to an hour, per daemon,
# not per session.
timeout: float | None = None if delay is None else min(delay, self._max_sleep)
if timeout == 0:
if reaped:
continue
# Still expired but nothing was reaped: the teardown failed.
# Back off rather than spinning on it.
log.warning("session(s) past their deadline could not be reaped; retrying")
timeout = self._retry_delay
try:
await asyncio.wait_for(self._wake.wait(), timeout=timeout)
except TimeoutError:
pass
+394
View File
@@ -0,0 +1,394 @@
"""The session record and its persistence (DESIGN.md §2.1, §2.3, §2.4).
Authoritative state is in daemon memory; the files under ``<state>/sessions/``
are a mirror, written ``tmp`` + :func:`os.replace` so a reader never sees a
half-written record. **A mirror write failure never fails a command** — it is
logged, and memory stays truthful.
"""
from __future__ import annotations
import base64
import json
import logging
import os
import secrets
import time
from dataclasses import asdict, dataclass, replace
from pathlib import Path
from typing import Any, Iterator
from .clock import Clock, iso
log = logging.getLogger("hhstreamd.registry")
SCHEMA = 2
#: Tombstones survive this long so a ``status`` after an unexplained failure
#: still answers the question. ``gc`` prunes them.
DEFAULT_TOMBSTONE_TTL_S = 3600.0
#: ``last_activity`` is hot, so the mirror is flushed at most this often for a
#: touch-only change — losing 15 s of an idle clock across a crash is
#: irrelevant against a three-hour timeout. State transitions always flush.
ACTIVITY_FLUSH_INTERVAL_S = 15.0
class State:
STARTING = "starting"
READY = "ready"
EXITED = "exited"
KILLED = "killed"
REAPED = "reaped"
COMPOSITOR_LOST = "compositor_lost"
#: States a session can never leave. Reaching one writes a tombstone.
TERMINAL = frozenset({State.EXITED, State.KILLED, State.REAPED, State.COMPOSITOR_LOST})
#: The error name a verb should be answered with when it arrives at a session
#: in a terminal state (all map to exit 4).
TERMINAL_ERROR = {
State.EXITED: "session_exited",
State.KILLED: "session_killed",
State.REAPED: "session_reaped",
State.COMPOSITOR_LOST: "compositor_lost",
}
def boot_id() -> str:
"""The kernel's boot id, or ``""`` if it cannot be read.
Sessions cannot outlive a reboot — the compositor does not — so a record
whose ``boot_id`` differs from the running kernel's describes processes
that are certainly gone, whatever ``/proc`` says about the pid today. This
is cheaper and far safer than hoping ``CLOCK_BOOTTIME`` values from a
previous boot happen not to collide.
"""
try:
return Path("/proc/sys/kernel/random/boot_id").read_text().strip()
except OSError: # pragma: no cover - /proc always has it on Linux
return ""
@dataclass(slots=True)
class Session:
"""DESIGN.md §2.1, minus the fields M0 cannot honestly fill.
``window``, ``workspace`` and ``foreground`` exist in the record and are
always ``None`` in M0: the compositor is not reachable from here, and the
design is explicit that nothing may be guessed.
"""
id: str
app: str
argv: list[str]
pid: int
pgid: int
sid: int
pid_starttime: int
cmdline_sha256: str
home: str
runtime_dir: str
created_at: str
created_boottime: float
last_activity_at: str
last_activity_boottime: float
idle_timeout_s: float
state: str = State.STARTING
adopted: bool = False
ready_signal: str | None = None
workspace: int | None = None
window: Any = None
foreground: bool | None = None
command_count: int = 0
exit_code: int | None = None
exit_detected_by: str | None = None
last_error: str | None = None
log_path: str | None = None
ephemeral_home: bool = False
tombstoned_boottime: float | None = None
tombstone_ttl_s: float = DEFAULT_TOMBSTONE_TTL_S
boot_id: str = ""
schema: int = SCHEMA
# ---- derived -----------------------------------------------------
@property
def live(self) -> bool:
return self.state not in TERMINAL
def deadline(self) -> float | None:
"""The boottime instant at which this session becomes idle-expired.
``None`` for a session that can never expire: one in a terminal state,
or one whose idle timeout is zero (disabled).
"""
if not self.live or not self.idle_timeout_s:
return None
return self.last_activity_boottime + self.idle_timeout_s
def expires_in(self, now: float) -> float | None:
deadline = self.deadline()
return None if deadline is None else deadline - now
def idle_for(self, now: float) -> float | None:
return None if not self.live else now - self.last_activity_boottime
def tombstone_expires(self) -> float | None:
if self.tombstoned_boottime is None:
return None
return self.tombstoned_boottime + self.tombstone_ttl_s
def to_dict(self) -> dict[str, Any]:
return asdict(self)
@classmethod
def from_dict(cls, data: dict[str, Any]) -> "Session":
known = {f for f in cls.__dataclass_fields__}
unknown = set(data) - known
if unknown:
# Forward compatibility: a newer daemon's extra fields are dropped
# rather than crashing the adopt path.
log.warning("dropping unknown record fields %s", sorted(unknown))
return cls(**{k: v for k, v in data.items() if k in known})
def public(self, now: float) -> dict[str, Any]:
"""The record as the protocol reports it, with the derived numbers."""
data = self.to_dict()
data["idle_s"] = self.idle_for(now)
data["expires_in_s"] = self.expires_in(now)
data["live"] = self.live
data["tombstone_expires_in_s"] = (
None if self.tombstone_expires() is None else self.tombstone_expires() - now
)
return data
def mint_id(app: str, taken: set[str], *, rng: Any = secrets) -> str:
"""``<app-slug>-<6 base32 chars>`` (DESIGN.md §2.2).
Handles are never reused: an id present in the live *or* tombstone set is
refused and another drawn. Nothing parses the slug; it is a readability
affordance.
"""
slug = "".join(c if c.isalnum() else "-" for c in app.lower()).strip("-") or "app"
slug = slug[:24]
for _ in range(1000):
raw = rng.token_bytes(5)
suffix = base64.b32encode(raw).decode("ascii").rstrip("=").lower()[:6]
candidate = f"{slug}-{suffix}"
if candidate not in taken:
return candidate
raise RuntimeError("could not mint an unused session id") # pragma: no cover
def atomic_write_json(path: Path, data: dict[str, Any]) -> None:
"""Write ``data`` to ``path`` atomically: temp file in the same directory,
fsync, then :func:`os.replace`.
Same directory matters — ``os.replace`` is only atomic within a filesystem.
"""
path.parent.mkdir(parents=True, exist_ok=True)
tmp = path.parent / f".{path.name}.{os.getpid()}.{secrets.token_hex(4)}.tmp"
try:
with open(tmp, "w", encoding="utf-8") as fh:
json.dump(data, fh, ensure_ascii=True, sort_keys=True)
fh.write("\n")
fh.flush()
os.fsync(fh.fileno())
os.replace(tmp, path)
except BaseException:
try:
tmp.unlink()
except OSError:
pass
raise
class Registry:
"""The in-memory session table and its on-disk mirror."""
def __init__(self, state_dir: Path | str, clock: Clock | None = None) -> None:
self.state_dir = Path(state_dir)
self.sessions_dir = self.state_dir / "sessions"
self.quarantine_dir = self.sessions_dir / "bad"
self.clock = clock or Clock()
self._sessions: dict[str, Session] = {}
self._last_flush: dict[str, float] = {}
self._boot_id = boot_id()
# ---- directories -------------------------------------------------
def ensure_dirs(self) -> None:
"""Create the state directories if the supervisor has not.
In production these come from the unit's ``StateDirectory=``; M0 ships
as an ordinary process, so the daemon creates what is missing rather
than depending on how it is supervised.
"""
self.sessions_dir.mkdir(parents=True, exist_ok=True, mode=0o750)
# ---- table -------------------------------------------------------
def __contains__(self, session_id: object) -> bool:
return session_id in self._sessions
def __len__(self) -> int:
return len(self._sessions)
def __iter__(self) -> Iterator[Session]:
return iter(list(self._sessions.values()))
def get(self, session_id: str) -> Session | None:
return self._sessions.get(session_id)
def all(self) -> list[Session]:
return sorted(self._sessions.values(), key=lambda s: s.created_boottime)
def live(self) -> list[Session]:
return [s for s in self.all() if s.live]
def tombstones(self) -> list[Session]:
return [s for s in self.all() if not s.live]
def taken_ids(self) -> set[str]:
"""Live *and* tombstoned ids, so handles are never reused."""
return set(self._sessions)
def new_id(self, app: str) -> str:
return mint_id(app, self.taken_ids())
def add(self, session: Session) -> Session:
if session.id in self._sessions:
raise KeyError(f"session id already in use: {session.id}")
if not session.boot_id:
session.boot_id = self._boot_id
self._sessions[session.id] = session
self.persist(session, force=True)
return session
def record_path(self, session_id: str) -> Path:
return self.sessions_dir / f"{session_id}.json"
def persist(self, session: Session, *, force: bool = False) -> None:
"""Mirror one record. Never raises: a mirror failure is not a command failure."""
now = self.clock.boottime()
if not force and now - self._last_flush.get(session.id, 0.0) < ACTIVITY_FLUSH_INTERVAL_S:
return
try:
self.ensure_dirs()
atomic_write_json(self.record_path(session.id), session.to_dict())
self._last_flush[session.id] = now
except OSError as exc:
log.error("mirror write failed for %s: %s (memory is authoritative)", session.id, exc)
def flush_all(self) -> None:
for session in self.all():
self.persist(session, force=True)
def drop(self, session_id: str) -> None:
"""Forget a record entirely and remove its mirror (used by ``gc``)."""
self._sessions.pop(session_id, None)
self._last_flush.pop(session_id, None)
try:
self.record_path(session_id).unlink()
except FileNotFoundError:
pass
except OSError as exc:
log.error("could not remove mirror for %s: %s", session_id, exc)
# ---- transitions -------------------------------------------------
def touch(self, session: Session, *, force_flush: bool = False) -> None:
"""Stamp activity. Callers stamp *after* a command is accepted."""
session.last_activity_boottime = self.clock.boottime()
session.last_activity_at = iso(self.clock.wall())
session.command_count += 1
self.persist(session, force=force_flush)
def terminate(
self,
session: Session,
state: str,
*,
exit_code: int | None = None,
exit_detected_by: str | None = None,
last_error: str | None = None,
) -> None:
"""Move a session to a terminal state and write its tombstone."""
session.state = state
if exit_code is not None:
session.exit_code = exit_code
if exit_detected_by is not None:
session.exit_detected_by = exit_detected_by
if last_error is not None:
session.last_error = last_error
session.tombstoned_boottime = self.clock.boottime()
self.persist(session, force=True)
# ---- startup adoption -------------------------------------------
def load_records(self) -> tuple[list[Session], list[tuple[Path, str]]]:
"""Read the mirror. Returns ``(records, quarantined)``.
A record that will not parse is moved to ``sessions/bad/`` and logged
loudly; its processes are left running and reported by ``gc`` rather
than killed on the basis of unreadable state (DESIGN.md §7 #13).
"""
records: list[Session] = []
quarantined: list[tuple[Path, str]] = []
if not self.sessions_dir.is_dir():
return records, quarantined
for path in sorted(self.sessions_dir.glob("*.json")):
try:
data = json.loads(path.read_text(encoding="utf-8"))
if not isinstance(data, dict):
raise ValueError("record is not an object")
session = Session.from_dict(data)
except (OSError, ValueError, TypeError) as exc:
reason = f"{type(exc).__name__}: {exc}"
log.error("quarantining unreadable record %s: %s", path, reason)
quarantined.append((path, reason))
self._quarantine(path)
continue
records.append(session)
return records, quarantined
def _quarantine(self, path: Path) -> None:
try:
self.quarantine_dir.mkdir(parents=True, exist_ok=True, mode=0o750)
os.replace(path, self.quarantine_dir / f"{path.name}.{int(time.time())}")
except OSError as exc: # pragma: no cover - best effort
log.error("could not quarantine %s: %s", path, exc)
def insert_loaded(self, session: Session) -> Session:
"""Put a record read from disk back into the table without re-persisting."""
self._sessions[session.id] = session
return session
def stale_boot(self, session: Session) -> bool:
"""True if the record was written before the current boot.
Its processes are gone whatever ``/proc`` says, because pids are
recycled from zero at boot.
"""
return bool(session.boot_id) and bool(self._boot_id) and session.boot_id != self._boot_id
def clone(session: Session, **changes: Any) -> Session:
return replace(session, **changes)
+335
View File
@@ -0,0 +1,335 @@
"""The control socket (DESIGN.md §3.4).
An asyncio unix ``SOCK_STREAM`` server speaking newline-delimited JSON: one
request object, one response object, connection closed. Chosen over HTTP
(framing and a server for no benefit on a local socket), D-Bus (no stdlib
client) and a bespoke binary protocol (unreadable at 3am). It is debuggable
with ``socat - UNIX-CONNECT:/run/hhstream/control.sock`` and costs nothing.
This module is framing and dispatch only. Every decision lives in
:mod:`hhstreamd.manager`.
"""
from __future__ import annotations
import asyncio
import collections
import contextlib
import logging
import os
import socket
import struct
from pathlib import Path
from typing import Any, Awaitable, Callable
from . import protocol
from .manager import ManagerError, SessionManager
from .policy import PolicyError
log = logging.getLogger("hhstreamd.server")
#: A request that has not produced a reply in this long is abandoned and the
#: caller told so, rather than holding the connection open indefinitely.
DEFAULT_REQUEST_TIMEOUT_S = 5.0
#: A `kill` may legitimately take `grace` seconds, so it gets its own budget.
KILL_TIMEOUT_MARGIN_S = 5.0
#: Replies to mutating verbs are remembered so a CLI that died before reading
#: the reply can retry without launching a second application.
DEDUP_ENTRIES = 64
#: Verbs whose replies are cached by `request_id`. The read verbs are absent
#: deliberately: they are idempotent, so a retry costs nothing and a cached
#: `list` would be a stale one.
MUTATING_VERBS = frozenset({"start", "touch", "kill", "adopt", "gc"})
class ControlServer:
def __init__(
self,
manager: SessionManager,
socket_path: Path | str,
*,
request_timeout_s: float = DEFAULT_REQUEST_TIMEOUT_S,
socket_mode: int = 0o660,
) -> None:
self.manager = manager
self.socket_path = Path(socket_path)
self.request_timeout_s = request_timeout_s
self.socket_mode = socket_mode
self._server: asyncio.AbstractServer | None = None
self._dedup: collections.OrderedDict[str, dict[str, Any]] = collections.OrderedDict()
self._ops: dict[str, Callable[[dict[str, Any], dict[str, Any]], Awaitable[dict[str, Any]]]] = {
"ping": self._op_ping,
"start": self._op_start,
"list": self._op_list,
"status": self._op_status,
"touch": self._op_touch,
"kill": self._op_kill,
"gc": self._op_gc,
"adopt": self._op_adopt,
"logs": self._op_logs,
}
# ------------------------------------------------------------------
# lifecycle
# ------------------------------------------------------------------
async def start(self) -> None:
self._prepare_socket_path()
self._server = await asyncio.start_unix_server(
self._handle,
path=str(self.socket_path),
# The stream limit is the frame limit: a peer cannot grow the
# daemon's read buffer past one message.
limit=protocol.MAX_LINE_BYTES,
)
with contextlib.suppress(OSError):
os.chmod(self.socket_path, self.socket_mode)
log.info("listening on %s", self.socket_path)
async def stop(self) -> None:
if self._server is not None:
self._server.close()
with contextlib.suppress(Exception):
await self._server.wait_closed()
self._server = None
with contextlib.suppress(FileNotFoundError, OSError):
self.socket_path.unlink()
def _prepare_socket_path(self) -> None:
"""Create the parent directory and clear a stale socket.
A socket file left by a crashed daemon is unlinked only after proving
nobody is listening on it: connecting and being refused is the proof.
Unlinking a *live* socket would silently steal another daemon's clients.
"""
self.socket_path.parent.mkdir(parents=True, exist_ok=True, mode=0o750)
if not self.socket_path.exists():
return
probe = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
try:
probe.settimeout(0.5)
probe.connect(str(self.socket_path))
except (ConnectionRefusedError, FileNotFoundError):
log.warning("removing stale socket %s", self.socket_path)
with contextlib.suppress(OSError):
self.socket_path.unlink()
except OSError as exc:
raise OSError(
f"{self.socket_path} exists and could not be probed: {exc}"
) from exc
else:
raise OSError(f"another daemon is already listening on {self.socket_path}")
finally:
probe.close()
# ------------------------------------------------------------------
# connection handling
# ------------------------------------------------------------------
async def _handle(self, reader: asyncio.StreamReader, writer: asyncio.StreamWriter) -> None:
caller = _peer_credentials(writer)
try:
try:
line = await reader.readuntil(b"\n")
except asyncio.IncompleteReadError as exc:
if not exc.partial:
return # a peer that connected and went away; not an error
line = exc.partial
except asyncio.LimitOverrunError:
await self._send(
writer,
protocol.error("message_too_large", "request exceeds the frame limit"),
)
return
if len(line) > protocol.MAX_LINE_BYTES:
await self._send(
writer,
protocol.error("message_too_large", "request exceeds the frame limit"),
)
return
response = await self._dispatch(line, caller)
await self._send(writer, response)
except (ConnectionResetError, BrokenPipeError):
pass
except Exception:
log.exception("unhandled error serving a request")
with contextlib.suppress(Exception):
await self._send(
writer, protocol.error("refused", "internal error; see the daemon log")
)
finally:
with contextlib.suppress(Exception):
writer.close()
await writer.wait_closed()
async def _send(self, writer: asyncio.StreamWriter, message: dict[str, Any]) -> None:
writer.write(protocol.encode(message))
await writer.drain()
async def _dispatch(self, line: bytes, caller: dict[str, Any]) -> dict[str, Any]:
started = self.manager.clock.boottime()
try:
request = protocol.decode_request(line)
except protocol.ProtocolError as exc:
self.manager.audit.emit(
"request_rejected", error=exc.error, reason=exc.reason, caller=caller
)
return protocol.error(exc.error, exc.reason)
op = request["op"]
request_id = request.get("request_id")
handler = self._ops.get(op)
if handler is None:
return protocol.error(
"unknown_op", f"unknown op {op!r}", hint=f"known: {', '.join(sorted(self._ops))}"
)
if isinstance(request_id, str) and op in MUTATING_VERBS and request_id in self._dedup:
cached = dict(self._dedup[request_id])
cached["deduplicated"] = True
return cached
timeout = self.request_timeout_s
if op == "kill":
# A kill is allowed to take its grace period plus the SIGKILL step.
grace = request.get("grace_s")
grace = self.manager.config.policy.grace_s if grace is None else float(grace or 0)
timeout = max(timeout, grace + KILL_TIMEOUT_MARGIN_S)
elif op == "start":
wait_ready = request.get("wait_ready_s")
wait_ready = (
self.manager.config.policy.wait_ready_s if wait_ready is None else float(wait_ready or 0)
)
timeout = max(timeout, wait_ready + 1.0)
try:
response = await asyncio.wait_for(handler(request, caller), timeout=timeout)
except TimeoutError:
response = protocol.error("timeout", f"{op} did not complete within {timeout:g}s")
except ManagerError as exc:
response = protocol.error(exc.error, exc.reason, hint=exc.hint, **exc.fields)
except PolicyError as exc:
response = protocol.error(exc.error, exc.reason, hint=exc.hint)
except (TypeError, ValueError) as exc:
response = protocol.error("invalid_argument", str(exc))
except Exception as exc:
log.exception("op %s failed", op)
response = protocol.error("refused", f"internal error: {type(exc).__name__}")
elapsed_ms = round((self.manager.clock.boottime() - started) * 1000, 3)
response.setdefault("elapsed_ms", elapsed_ms)
if isinstance(request_id, str):
response["request_id"] = request_id
if op in MUTATING_VERBS:
self._remember(request_id, response)
self.manager.audit.emit(
"request",
verb=op,
session=request.get("id") or response.get("id"),
caller=caller,
ok=bool(response.get("ok")),
error=response.get("error"),
duration_ms=elapsed_ms,
request_id=request_id if isinstance(request_id, str) else None,
)
return response
def _remember(self, request_id: str, response: dict[str, Any]) -> None:
self._dedup[request_id] = dict(response)
self._dedup.move_to_end(request_id)
while len(self._dedup) > DEDUP_ENTRIES:
self._dedup.popitem(last=False)
# ------------------------------------------------------------------
# ops
# ------------------------------------------------------------------
async def _op_ping(self, request: dict[str, Any], caller: dict[str, Any]) -> dict[str, Any]:
from . import __version__
return protocol.ok(
version=__version__,
pid=os.getpid(),
boottime=self.manager.now(),
live=len(self.manager.registry.live()),
)
async def _op_start(self, request: dict[str, Any], caller: dict[str, Any]) -> dict[str, Any]:
record = await self.manager.start(
request.get("app"),
request.get("argv"),
idle_timeout_s=request.get("idle_timeout_s"),
wait_ready_s=request.get("wait_ready_s"),
home=request.get("home"),
ephemeral_home=bool(request.get("ephemeral_home", False)),
)
return protocol.ok(id=record["id"], state=record["state"], session=record)
async def _op_list(self, request: dict[str, Any], caller: dict[str, Any]) -> dict[str, Any]:
return protocol.ok(**self.manager.list(include_tombstones=bool(request.get("all", False))))
async def _op_status(self, request: dict[str, Any], caller: dict[str, Any]) -> dict[str, Any]:
record = self.manager.status(request.get("id"), events=int(request.get("events", 20)))
return protocol.ok(id=record["id"], state=record["state"], session=record)
async def _op_touch(self, request: dict[str, Any], caller: dict[str, Any]) -> dict[str, Any]:
record = self.manager.touch(request.get("id"))
return protocol.ok(id=record["id"], state=record["state"], session=record)
async def _op_kill(self, request: dict[str, Any], caller: dict[str, Any]) -> dict[str, Any]:
record = await self.manager.kill(
request.get("id"),
grace_s=request.get("grace_s"),
sig=request.get("signal", "TERM"),
)
return protocol.ok(id=record["id"], state=record["state"], session=record)
async def _op_gc(self, request: dict[str, Any], caller: dict[str, Any]) -> dict[str, Any]:
return protocol.ok(**self.manager.gc())
async def _op_adopt(self, request: dict[str, Any], caller: dict[str, Any]) -> dict[str, Any]:
record = await self.manager.adopt(
request.get("pid"),
app=request.get("app"),
idle_timeout_s=request.get("idle_timeout_s"),
)
return protocol.ok(id=record["id"], state=record["state"], session=record)
async def _op_logs(self, request: dict[str, Any], caller: dict[str, Any]) -> dict[str, Any]:
return protocol.ok(
**self.manager.logs(
request.get("id"),
lines=int(request.get("lines", 100)),
offset=request.get("offset"),
)
)
def _peer_credentials(writer: asyncio.StreamWriter) -> dict[str, Any]:
"""``SO_PEERCRED`` of the connecting process, for the audit log.
M0 records who asked but does **not** authorise on it: the uid/gid
allowlist is §8 item 2, which the brief holds out of this milestone. The
socket's own mode and its directory are the access control here.
"""
sock = writer.get_extra_info("socket")
if sock is None:
return {}
try:
raw = sock.getsockopt(socket.SOL_SOCKET, socket.SO_PEERCRED, struct.calcsize("3i"))
pid, uid, gid = struct.unpack("3i", raw)
except OSError:
return {}
caller: dict[str, Any] = {"pid": pid, "uid": uid, "gid": gid}
try:
with open(f"/proc/{pid}/comm", encoding="utf-8") as fh:
caller["comm"] = fh.read().strip()
except OSError:
pass
return caller
+51
View File
@@ -0,0 +1,51 @@
"""Reading the tail of a file that may be arbitrarily large.
Both the audit log and an application's captured output are append-only files
with no upper bound in M0, and both are read by verbs that must stay cheap.
Reading either whole — ``readlines()`` on a gigabyte of Chromium stderr — would
turn a ``status`` into an out-of-memory event on a machine that already has no
resource caps (DESIGN.md §3.3). So every tail read is bounded in bytes, and
says when it truncated.
"""
from __future__ import annotations
import os
from pathlib import Path
#: The most any single tail read will pull into memory.
DEFAULT_TAIL_BYTES = 1 << 20 # 1 MiB
def tail_lines(
path: Path | str, limit: int, *, max_bytes: int = DEFAULT_TAIL_BYTES
) -> tuple[list[str], bool, int]:
"""The last ``limit`` lines of ``path``.
Returns ``(lines, truncated, size)``. ``truncated`` is true when the window
did not reach the start of the file, so a caller can say so rather than
implying it showed everything.
"""
if limit <= 0:
try:
return [], False, os.stat(path).st_size
except OSError:
return [], False, 0
try:
with open(path, "rb") as fh:
size = fh.seek(0, os.SEEK_END)
window = min(size, max_bytes)
fh.seek(size - window)
data = fh.read(window)
except OSError:
return [], False, 0
partial = window < size
if partial:
# The window almost certainly starts mid-line; drop that fragment
# rather than reporting a half line as if it were whole.
newline = data.find(b"\n")
data = data[newline + 1 :] if newline != -1 else b""
lines = data.decode("utf-8", "replace").splitlines()
truncated = partial or len(lines) > limit
return lines[-limit:], truncated, size
+106
View File
@@ -0,0 +1,106 @@
"""The workspace allocator: lowest free, reuse after release, and loud bugs."""
from __future__ import annotations
import pytest
from hhstreamd.containers import NoSlots, WorkspaceAllocator
def test_allocates_the_band_from_its_base_upwards() -> None:
alloc = WorkspaceAllocator(1000, 4)
assert [alloc.allocate() for _ in range(4)] == [1000, 1001, 1002, 1003]
assert alloc.in_use() == [1000, 1001, 1002, 1003]
def test_a_released_slot_is_reused_before_a_higher_free_one() -> None:
alloc = WorkspaceAllocator(1000, 4)
for _ in range(3):
alloc.allocate()
alloc.release(1001)
assert alloc.in_use() == [1000, 1002]
assert alloc.allocate() == 1001
assert alloc.allocate() == 1003
assert alloc.in_use() == [1000, 1001, 1002, 1003]
def test_a_full_band_raises_no_slots_and_stays_unchanged() -> None:
alloc = WorkspaceAllocator(1000, 2)
alloc.allocate()
alloc.allocate()
with pytest.raises(NoSlots) as exc:
alloc.allocate()
assert "1000..1001" in str(exc.value)
assert alloc.in_use() == [1000, 1001]
# Exhaustion is not terminal: a release makes the slot available again.
alloc.release(1000)
assert alloc.allocate() == 1000
def test_releasing_a_slot_we_never_handed_out_is_a_key_error() -> None:
alloc = WorkspaceAllocator(1000, 2)
alloc.allocate()
# Inside the band but free, and outside the band: both are caller bugs, and
# a silent no-op would let two sessions share a workspace later.
with pytest.raises(KeyError) as free_slot:
alloc.release(1001)
assert "1001" in str(free_slot.value)
with pytest.raises(KeyError):
alloc.release(7)
assert alloc.in_use() == [1000]
def test_a_double_release_raises() -> None:
alloc = WorkspaceAllocator(1000, 2)
slot = alloc.allocate()
alloc.release(slot)
with pytest.raises(KeyError):
alloc.release(slot)
assert alloc.in_use() == []
def test_in_use_is_ascending_whatever_the_allocation_order() -> None:
alloc = WorkspaceAllocator(1000, 3)
a, b, c = alloc.allocate(), alloc.allocate(), alloc.allocate()
alloc.release(b)
alloc.release(a)
alloc.allocate()
alloc.allocate()
assert alloc.in_use() == [1000, 1001, 1002]
assert c == 1002
def test_a_band_must_be_a_real_band() -> None:
with pytest.raises(ValueError) as count:
WorkspaceAllocator(1000, 0)
assert "count" in str(count.value)
with pytest.raises(ValueError) as base:
WorkspaceAllocator(0, 4)
assert "base" in str(base.value)
with pytest.raises(ValueError) as kind:
WorkspaceAllocator(1000, 2.5) # type: ignore[arg-type]
assert "count" in str(kind.value)
assert "is not an int" in str(kind.value)
def test_two_allocators_do_not_share_state() -> None:
# The set is per-instance; a class attribute here would hand the same
# workspace to two brokers' containers.
first = WorkspaceAllocator(1000, 2)
second = WorkspaceAllocator(1000, 2)
assert first.allocate() == 1000
assert second.allocate() == 1000
assert first.in_use() == second.in_use() == [1000]
+223
View File
@@ -0,0 +1,223 @@
"""Container plans: the exact argv lists, and the validation that guards them.
The argv lists are asserted whole rather than probed for substrings. These
commands are the only thing this module produces, nobody here runs them, and a
test that accepted "contains the output name" would pass on a command that
silently changed shape under a refactor.
"""
from __future__ import annotations
import pytest
from hhstreamd.containers import (
ContainerPlan,
app_id_for,
contains_output,
container_plan,
output_name,
quarantine_plan,
)
def test_create_is_the_four_commands_in_order() -> None:
plan = container_plan("foot-aaaaaa", 1000, 1280, 720, "hh-foot-aaaaaa")
assert plan.create == [
["hyprctl", "output", "create", "headless", "hh-foot-aaaaaa"],
[
"hyprctl",
"keyword",
"workspace",
"1000, monitor:hh-foot-aaaaaa, default:true, persistent:true",
],
[
"hyprctl",
"keyword",
"windowrule",
"match:initial_class ^hh-foot-aaaaaa$, workspace 1000, monitor hh-foot-aaaaaa",
],
["hyprctl", "keyword", "monitor", "hh-foot-aaaaaa,1280x720,auto,1"],
]
def test_teardown_removes_the_output_and_nothing_else() -> None:
plan = container_plan("foot-aaaaaa", 1000, 1280, 720, "hh-foot-aaaaaa")
assert plan.teardown == [["hyprctl", "output", "remove", "hh-foot-aaaaaa"]]
def test_plan_describes_what_it_built() -> None:
plan = container_plan("x1", 1007, 1920, 1080, "org.gnome.Nautilus")
assert isinstance(plan, ContainerPlan)
assert (plan.output, plan.workspace, plan.mode) == ("hh-x1", 1007, (1920, 1080))
def test_a_foreign_class_is_matched_as_given() -> None:
# The rule has to match what the toolkit reports, which is not always the
# id the launcher stamped.
plan = container_plan("x1", 1001, 800, 600, "org.gnome.Nautilus")
assert plan.create[2] == [
"hyprctl",
"keyword",
"windowrule",
"match:initial_class ^org.gnome.Nautilus$, workspace 1001, monitor hh-x1",
]
def test_identical_arguments_give_byte_identical_commands() -> None:
first = container_plan("foot-aaaaaa", 1000, 1280, 720, "hh-foot-aaaaaa")
second = container_plan("foot-aaaaaa", 1000, 1280, 720, "hh-foot-aaaaaa")
assert first == second
assert first.create == second.create
assert first.teardown == second.teardown
def test_output_name_and_app_id_are_the_same_string() -> None:
assert output_name("foot-aaaaaa") == "hh-foot-aaaaaa"
assert app_id_for("foot-aaaaaa") == "hh-foot-aaaaaa"
@pytest.mark.parametrize(
"name",
[
"hh-foot-aaaaaa",
"hh-x",
"hh-1000",
],
)
def test_contains_output_claims_our_own_names(name: str) -> None:
assert contains_output(name) is True
@pytest.mark.parametrize(
"name",
[
"HEADLESS-1", # the compositor's own
"eDP-1",
"hh",
"hh-", # no session id: not something the broker ever created
"hh-Foot", # upper case is outside the minted alphabet
"hh-foot\n", # a trailing newline is not a name, it is two lines
"hh-foot aaaa",
" hh-foot",
"xhh-foot",
"",
],
)
def test_contains_output_refuses_everything_else(name: str) -> None:
# A scan removes what this says is ours, so a false positive deletes an
# output the broker did not create.
assert contains_output(name) is False
@pytest.mark.parametrize(
"bad",
["", "Foot", "foot_aaaa", "foot aaaa", "hh-foot;rm", "foot\n", "../x", "é"],
)
def test_a_session_id_outside_the_alphabet_is_refused(bad: str) -> None:
with pytest.raises(ValueError) as exc:
output_name(bad)
assert repr(bad) in str(exc.value)
assert "[a-z0-9-]" in str(exc.value)
with pytest.raises(ValueError) as app_exc:
app_id_for(bad)
assert repr(bad) in str(app_exc.value)
with pytest.raises(ValueError) as plan_exc:
container_plan(bad, 1000, 1280, 720, "hh-x")
assert repr(bad) in str(plan_exc.value)
@pytest.mark.parametrize("width,height", [(0, 720), (1280, 0), (-1, 720), (1280, -720)])
def test_a_non_positive_mode_is_refused(width: int, height: int) -> None:
with pytest.raises(ValueError) as exc:
container_plan("x1", 1000, width, height, "hh-x1")
message = str(exc.value)
offender = width if width < 1 else height
assert repr(offender) in message
assert ("width" in message) or ("height" in message)
assert ">= 1" in message
def test_a_fractional_mode_is_refused() -> None:
# 1280.5 would reach the compositor as "1280.5x720"; a mode is pixels.
with pytest.raises(ValueError) as exc:
container_plan("x1", 1000, 1280.5, 720, "hh-x1") # type: ignore[arg-type]
assert "width" in str(exc.value)
assert "is not an int" in str(exc.value)
def test_a_boolean_workspace_is_refused() -> None:
# True is an int in Python and would quietly become workspace 1.
with pytest.raises(ValueError) as exc:
container_plan("x1", True, 1280, 720, "hh-x1") # type: ignore[arg-type]
assert "workspace" in str(exc.value)
assert "is not an int" in str(exc.value)
@pytest.mark.parametrize("workspace", [0, -1])
def test_a_non_positive_workspace_is_refused(workspace: int) -> None:
with pytest.raises(ValueError) as exc:
container_plan("x1", workspace, 1280, 720, "hh-x1")
assert "workspace" in str(exc.value)
assert repr(workspace) in str(exc.value)
def test_an_empty_app_id_is_refused() -> None:
with pytest.raises(ValueError) as exc:
container_plan("x1", 1000, 1280, 720, "")
assert "app id" in str(exc.value)
assert "''" in str(exc.value)
@pytest.mark.parametrize(
"bad",
[
"hh-x1, workspace 1001", # a forged second rule field
"hh-x1$, monitor eDP-1",
"hh x1",
"hh-x1\nkeyword monitor eDP-1,disable",
"^hh-x1$",
"hh-x1'",
],
)
def test_an_app_id_that_could_forge_a_rule_field_is_refused(bad: str) -> None:
with pytest.raises(ValueError) as exc:
container_plan("x1", 1000, 1280, 720, bad)
assert repr(bad) in str(exc.value)
assert "[A-Za-z0-9._+-]" in str(exc.value)
def test_quarantine_plan_is_the_catch_all_keyword() -> None:
plan = quarantine_plan("special:quarantine")
assert plan is not None
assert plan.create == [
[
"hyprctl",
"keyword",
"windowrule",
"match:initial_class .*, workspace special:quarantine",
]
]
# A rule, not a container: nothing to remove, no output, no mode.
assert plan.teardown == []
assert (plan.output, plan.workspace, plan.mode) == ("", 0, (0, 0))
@pytest.mark.parametrize("disabled", ["", " ", "\n"])
def test_quarantine_plan_is_none_when_disabled(disabled: str) -> None:
assert quarantine_plan(disabled) is None
@pytest.mark.parametrize("bad", ["special:quarantine, monitor eDP-1", "a b", "q\n1", "q'"])
def test_a_quarantine_workspace_that_could_forge_a_field_is_refused(bad: str) -> None:
with pytest.raises(ValueError) as exc:
quarantine_plan(bad)
assert "quarantine_workspace" in str(exc.value)
assert repr(bad.strip()) in str(exc.value)
+80
View File
@@ -0,0 +1,80 @@
"""Shared fixtures.
Two things are worth knowing:
* **Nothing here sleeps for real time** except where the brief requires it: the
one integration test that proves a real three-second idle reap. Everything
else either injects a clock or polls with a deadline.
* **Unix socket paths are short by force.** ``AF_UNIX`` caps a path at 108
bytes and ``pytest``'s ``tmp_path`` can exceed that on a deep ``TMPDIR``, so
sockets live in a shallow ``mkdtemp`` of their own rather than under
``tmp_path``.
"""
from __future__ import annotations
import os
import shutil
import socket
import sys
import tempfile
import time
from pathlib import Path
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
#: Linux's sun_path is 108 bytes including the NUL.
SUN_PATH_MAX = 107
@pytest.fixture
def short_tmpdir() -> Path:
"""A temp directory shallow enough to hold a unix socket path."""
path = Path(tempfile.mkdtemp(prefix="hh"))
try:
yield path
finally:
shutil.rmtree(path, ignore_errors=True)
def socket_path(directory: Path, name: str = "c.sock") -> str:
path = str(directory / name)
if len(path.encode()) > SUN_PATH_MAX:
pytest.skip(f"socket path too long for AF_UNIX ({len(path)} bytes): {path}")
return path
def wait_until(predicate, timeout: float, interval: float = 0.02, what: str = "condition"):
"""Poll ``predicate`` until it is truthy. Returns its value, or fails.
This is how the integration tests wait: a deadline and a tight poll, never
a fixed sleep sized by guesswork.
"""
deadline = time.monotonic() + timeout
while True:
value = predicate()
if value:
return value
if time.monotonic() > deadline:
pytest.fail(f"timed out after {timeout}s waiting for {what}")
time.sleep(interval)
def pid_alive(pid: int) -> bool:
return os.path.isdir(f"/proc/{pid}")
def socket_is_live(path: str) -> bool:
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
try:
sock.settimeout(0.5)
sock.connect(path)
return True
except OSError:
return False
finally:
sock.close()
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,306 @@
"""The integration contract: ``register(subparsers)`` into someone else's CLI.
The registrant is assumed to know nothing about the daemon beyond the socket
path, so these tests drive the handlers through a host parser they build
themselves and stub out the transport.
"""
from __future__ import annotations
import argparse
import io
import json
from contextlib import redirect_stderr, redirect_stdout
import pytest
import hhstream_session
from hhstream_session import register
from hhstream_session.client import ClientError
from hhstreamd.protocol import ExitCode
@pytest.fixture
def host_cli():
"""A stand-in for the pre-existing ``hhstream`` CLI."""
parser = argparse.ArgumentParser(prog="hhstream")
sub = parser.add_subparsers(dest="command")
existing = sub.add_parser("frame", help="an existing subcommand")
existing.set_defaults(func=lambda args: 0)
register(sub)
return parser
@pytest.fixture
def sent(monkeypatch):
"""Capture what the handlers would send, and reply with what the test wants."""
class Transport(list):
"""The captured requests, plus the canned replies to hand back."""
replies = [
{
"ok": True,
"id": "stub-aaaaaa",
"session": {"id": "stub-aaaaaa", "state": "ready"},
"sessions": [],
"lines": [],
}
]
calls = Transport()
def fake_call(op, *, socket_path=None, timeout=None, **fields):
# Mirror protocol.request(): unset fields are not put on the wire, so
# the tests below can assert on what is actually sent.
wire = {k: v for k, v in fields.items() if v is not None}
calls.append({"op": op, "socket_path": socket_path, **wire})
return calls.replies[min(len(calls) - 1, len(calls.replies) - 1)]
monkeypatch.setattr(hhstream_session, "call", fake_call)
return calls
def run(parser: argparse.ArgumentParser, argv: list[str]) -> tuple[int, str, str]:
args = parser.parse_args(argv)
out, err = io.StringIO(), io.StringIO()
with redirect_stdout(out), redirect_stderr(err):
code = hhstream_session.dispatch(args)
return code, out.getvalue(), err.getvalue()
# ---------------------------------------------------------------------------
# registration
# ---------------------------------------------------------------------------
def test_registration_adds_exactly_one_subcommand_and_leaves_the_host_alone(host_cli):
args = host_cli.parse_args(["frame"])
assert args.func(args) == 0 # the pre-existing subcommand still works
args = host_cli.parse_args(["session", "list"])
assert callable(args.func)
def test_the_handler_is_exposed_under_both_the_common_name_and_a_namespaced_one(host_cli):
args = host_cli.parse_args(["session", "status", "foo-aaaaaa"])
assert args.func is args.hhstream_session_handler
def test_every_v1_verb_is_registered(host_cli):
for verb in ["start", "list", "status", "touch", "kill", "gc", "adopt", "logs"]:
argv = ["session", verb]
if verb in ("status", "touch", "kill", "logs"):
argv.append("foo-aaaaaa")
if verb == "start":
argv.append("foot")
if verb == "adopt":
argv += ["--pid", "1234"]
args = host_cli.parse_args(argv)
assert callable(args.func), verb
def test_session_with_no_verb_lists(host_cli, sent):
code, _, _ = run(host_cli, ["session"])
assert code == ExitCode.OK
assert sent[0]["op"] == "list"
def test_the_socket_may_be_given_before_or_after_the_verb(host_cli, sent):
run(host_cli, ["session", "--socket", "/a.sock", "list"])
run(host_cli, ["session", "list", "--socket", "/b.sock"])
assert sent[0]["socket_path"] == "/a.sock"
assert sent[1]["socket_path"] == "/b.sock"
def test_the_socket_falls_back_to_the_environment(host_cli, sent, monkeypatch):
"""The registrant's only piece of daemon knowledge."""
monkeypatch.setenv("HHSTREAM_SOCKET", "/env.sock")
run(host_cli, ["session", "list"])
# The handler passes None through; the client resolves the default.
assert sent[0]["socket_path"] is None
from hhstream_session.client import default_socket_path
assert default_socket_path() == "/env.sock"
# ---------------------------------------------------------------------------
# request shapes
# ---------------------------------------------------------------------------
def test_start_sends_the_app_and_an_argv_list_never_a_command_string(host_cli, sent):
run(host_cli, ["session", "start", "chromium", "--", "--new-window", "https://example.com"])
assert sent[0]["op"] == "start"
assert sent[0]["app"] == "chromium"
assert sent[0]["argv"] == ["--new-window", "https://example.com"]
def test_this_commands_own_flags_still_parse_after_the_app_name(host_cli, sent):
"""The separator divides the two argvs; it does not divide the parsers."""
run(host_cli, ["session", "start", "chromium", "--idle-timeout", "5m", "--", "--new-window"])
assert sent[0]["idle_timeout_s"] == 300.0
assert sent[0]["argv"] == ["--new-window"]
def test_an_application_flag_after_the_separator_is_never_read_as_ours(host_cli, sent):
run(host_cli, ["session", "start", "chromium", "--", "--idle-timeout", "5m"])
assert "idle_timeout_s" not in sent[0]
assert sent[0]["argv"] == ["--idle-timeout", "5m"]
def test_durations_are_parsed_on_the_way_in(host_cli, sent):
run(host_cli, ["session", "start", "foot", "--idle-timeout", "3h"])
assert sent[0]["idle_timeout_s"] == 10800.0
run(host_cli, ["session", "kill", "foo-aaaaaa", "--grace", "30s"])
assert sent[1]["grace_s"] == 30.0
def test_a_bad_duration_is_rejected_by_argparse(host_cli):
with pytest.raises(SystemExit):
host_cli.parse_args(["session", "start", "foot", "--idle-timeout", "soon"])
def test_home_and_ephemeral_home_are_mutually_exclusive(host_cli):
with pytest.raises(SystemExit):
host_cli.parse_args(["session", "start", "foot", "--home", "/x", "--ephemeral-home"])
def test_ephemeral_home_is_opt_in_and_absent_by_default(host_cli, sent):
run(host_cli, ["session", "start", "foot"])
assert "ephemeral_home" not in sent[0]
run(host_cli, ["session", "start", "foot", "--ephemeral-home"])
assert sent[1]["ephemeral_home"] is True
def test_list_all_is_only_sent_when_asked(host_cli, sent):
run(host_cli, ["session", "list"])
assert "all" not in sent[0]
run(host_cli, ["session", "list", "--all"])
assert sent[1]["all"] is True
def test_request_ids_are_forwarded_on_mutating_verbs(host_cli, sent):
run(host_cli, ["session", "start", "foot", "--request-id", "abc"])
assert sent[0]["request_id"] == "abc"
# ---------------------------------------------------------------------------
# output and exit codes
# ---------------------------------------------------------------------------
def test_start_prints_the_bare_id_on_stdout_and_detail_on_stderr(host_cli, sent):
sent.replies[:] = [
{
"ok": True,
"id": "foot-k3f9qx",
"session": {
"id": "foot-k3f9qx",
"app": "foot",
"pid": 48812,
"pgid": 48812,
"sid": 48812,
"state": "ready",
"ready_signal": "spawned",
"idle_timeout_s": 10800,
},
}
]
code, out, err = run(host_cli, ["session", "start", "foot"])
assert code == ExitCode.OK
assert out == "foot-k3f9qx\n" # exactly the id, so $(...) works
assert "pid 48812" in err
assert "touch foot-k3f9qx" in err # the keepalive is named where it is needed
def test_start_json_puts_one_object_on_stdout_and_nothing_on_stderr(host_cli, sent):
sent.replies[:] = [{"ok": True, "id": "foot-k3f9qx", "session": {"id": "foot-k3f9qx"}}]
code, out, err = run(host_cli, ["session", "start", "--json", "foot"])
assert code == ExitCode.OK
assert json.loads(out) == {"id": "foot-k3f9qx"}
assert err == ""
def test_json_is_available_on_every_read_verb(host_cli, sent):
sent.replies[:] = [{"ok": True, "sessions": []}]
code, out, _ = run(host_cli, ["session", "list", "--json"])
assert json.loads(out)["ok"] is True
sent.replies[:] = [{"ok": True, "session": {"id": "a"}, "id": "a"}]
code, out, _ = run(host_cli, ["session", "status", "--json", "a"])
assert json.loads(out)["session"] == {"id": "a"}
sent.replies[:] = [{"ok": True, "lines": ["hello"], "offset": 6, "eof": True}]
code, out, _ = run(host_cli, ["session", "logs", "--json", "a"])
assert json.loads(out)["lines"] == ["hello"]
def test_errors_go_to_stderr_with_the_machine_field_in_json_mode(host_cli, sent):
sent.replies[:] = [
{"ok": False, "error": "not_allowlisted", "reason": "app 'sh' is not in the allowlist"}
]
code, out, err = run(host_cli, ["session", "start", "--json", "sh"])
assert code == ExitCode.POLICY
assert out == ""
assert json.loads(err)["error"] == "not_allowlisted"
def test_errors_are_human_readable_without_json(host_cli, sent):
sent.replies[:] = [
{"ok": False, "error": "no_such_session", "reason": "no session 'x'", "hint": "try list"}
]
code, out, err = run(host_cli, ["session", "status", "x"])
assert code == ExitCode.NO_SUCH_SESSION
assert "no session 'x'" in err and "try list" in err
def test_a_transport_failure_becomes_exit_five_not_a_traceback(host_cli, monkeypatch):
def boom(*args, **kwargs):
raise ClientError("daemon_unreachable", "nothing is listening", hint="start it")
monkeypatch.setattr(hhstream_session, "call", boom)
code, _, err = run(host_cli, ["session", "list"])
assert code == ExitCode.DAEMON_UNREACHABLE
assert "nothing is listening" in err and "start it" in err
# ---------------------------------------------------------------------------
# the activity contract, as documented to the user
# ---------------------------------------------------------------------------
def test_the_help_text_says_touch_is_the_only_mark_of_activity(host_cli):
out = io.StringIO()
with redirect_stdout(out):
with pytest.raises(SystemExit):
host_cli.parse_args(["session", "--help"])
text = out.getvalue()
assert "touch" in text
assert "ONLY" in text
for verb in ["list", "status", "logs", "gc"]:
assert verb in text
@pytest.mark.parametrize("verb", ["list", "status", "logs", "gc"])
def test_each_read_verbs_help_says_it_does_not_count(host_cli, verb):
out = io.StringIO()
with redirect_stdout(out):
with pytest.raises(SystemExit):
host_cli.parse_args(["session", "--help"])
line = [l for l in out.getvalue().splitlines() if l.strip().startswith(verb)]
assert line, verb
assert "does NOT count" in " ".join(out.getvalue().split())
def test_touchs_help_calls_itself_the_only_mark_of_activity(host_cli):
out = io.StringIO()
with redirect_stdout(out):
with pytest.raises(SystemExit):
host_cli.parse_args(["session", "touch", "--help"])
assert "only verb that counts as activity" in " ".join(out.getvalue().split())
+414
View File
@@ -0,0 +1,414 @@
"""BUG-01: a launched application must be able to reach the compositor.
Every test here fails against the pre-fix daemon, which spawned children with
its own inherited environment — on the target machine the *login* session, with
no compositor listening in it — so every GUI application died within a second of
``start`` and the only symptom was ``state exited`` one command later.
A note on the fixtures: a Wayland socket that is **bound but not listening**
gives ``ECONNREFUSED`` on connect, which is exactly what a socket left behind by
a crashed compositor looks like. A bound *and listening* socket is what a live
one looks like. That is the whole difference between the two helpers below, and
it is why the tests need no compositor.
"""
from __future__ import annotations
import json
import logging
import os
import socket
import stat
import sys
from pathlib import Path
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from conftest import SUN_PATH_MAX
from hhstreamd import compositor
from hhstreamd.__main__ import (
ENV_COMPOSITOR_RUNTIME_DIR,
build_parser,
main,
resolve_compositor,
)
from hhstreamd.compositor import CompositorError, CompositorSession, CompositorTarget, discover
from hhstreamd.launcher import Launcher
def _bind(directory: Path, name: str, *, listening: bool) -> socket.socket:
path = directory / name
if len(str(path).encode()) > SUN_PATH_MAX:
pytest.skip(f"socket path too long for AF_UNIX: {path}")
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
sock.bind(str(path))
if listening:
sock.listen(4)
return sock
@pytest.fixture
def sockets():
"""Open sockets closed at teardown, so no test leaks a listener."""
opened: list[socket.socket] = []
def make(directory: Path, name: str, *, listening: bool = True) -> Path:
sock = _bind(directory, name, listening=listening)
opened.append(sock)
return directory / name
try:
yield make
finally:
for sock in opened:
sock.close()
# ---------------------------------------------------------------------------
# discovery: the socket's name is found, never assumed
# ---------------------------------------------------------------------------
def test_discovers_wayland_1_not_wayland_0(short_tmpdir, sockets):
"""The target machine's compositor is on ``wayland-1``. Nothing may assume ``-0``."""
sockets(short_tmpdir, "wayland-1")
session = discover(short_tmpdir)
assert session.wayland_display == "wayland-1"
assert session.runtime_dir == short_tmpdir.resolve()
assert session.socket_path == short_tmpdir.resolve() / "wayland-1"
def test_ignores_lock_files_and_other_non_sockets(short_tmpdir, sockets):
"""``wayland-1.lock`` is a regular file libwayland writes beside the socket."""
(short_tmpdir / "wayland-1.lock").write_text("")
(short_tmpdir / "wayland-0").write_text("not a socket") # a leftover regular file
(short_tmpdir / "wayland-9").mkdir() # and a directory named like one
sockets(short_tmpdir, "wayland-1")
assert discover(short_tmpdir).wayland_display == "wayland-1"
def test_candidates_lists_only_sockets(short_tmpdir, sockets):
sockets(short_tmpdir, "wayland-0")
(short_tmpdir / "wayland-0.lock").write_text("")
(short_tmpdir / "bus").write_text("")
assert compositor.candidates(short_tmpdir.resolve()) == ["wayland-0"]
def test_a_live_socket_beats_a_crashed_compositors_leftover(short_tmpdir, sockets):
"""Both are sockets; only one has anything behind it."""
sockets(short_tmpdir, "wayland-1", listening=False) # refuses connections
sockets(short_tmpdir, "wayland-0", listening=True)
assert discover(short_tmpdir).wayland_display == "wayland-0"
def test_among_live_sockets_the_later_name_wins(short_tmpdir, sockets):
"""A compositor takes the first free name, so the survivor of a restart is later."""
sockets(short_tmpdir, "wayland-0")
sockets(short_tmpdir, "wayland-1")
assert discover(short_tmpdir).wayland_display == "wayland-1"
def test_non_numeric_suffix_is_usable_but_ranks_last(short_tmpdir, sockets):
sockets(short_tmpdir, "wayland-hyprland")
assert discover(short_tmpdir).wayland_display == "wayland-hyprland"
# ---------------------------------------------------------------------------
# fail loudly, early, and correctly
# ---------------------------------------------------------------------------
def test_empty_runtime_dir_is_refused_and_named(short_tmpdir):
with pytest.raises(CompositorError) as exc:
discover(short_tmpdir)
message = str(exc.value)
assert str(short_tmpdir.resolve()) in message
assert "no Wayland socket" in message
# It must not be mistaken for the daemon's own --runtime-dir.
assert "--runtime-dir" in message
def test_missing_runtime_dir_is_refused_and_named(short_tmpdir):
missing = short_tmpdir / "nope"
with pytest.raises(CompositorError) as exc:
discover(missing)
assert str(missing) in str(exc.value)
def test_a_file_in_place_of_the_runtime_dir_is_refused(short_tmpdir):
path = short_tmpdir / "file"
path.write_text("")
with pytest.raises(CompositorError) as exc:
discover(path)
assert str(path) in str(exc.value)
def test_only_stale_sockets_is_refused(short_tmpdir, sockets):
"""Accepting work here would kill every application that arrived."""
sockets(short_tmpdir, "wayland-0", listening=False)
sockets(short_tmpdir, "wayland-1", listening=False)
with pytest.raises(CompositorError) as exc:
discover(short_tmpdir)
message = str(exc.value)
assert str(short_tmpdir.resolve()) in message
assert "no usable Wayland socket" in message
assert "wayland-0" in message and "wayland-1" in message
def test_an_unconnectable_but_not_refused_socket_is_used_with_a_warning(
short_tmpdir, sockets, caplog, monkeypatch
):
"""An ambiguous probe (EACCES, a timeout) is a warning, not an outage."""
sockets(short_tmpdir, "wayland-1", listening=False)
def refuse_with_eacces(path):
return compositor._Candidate(Path(path).name, False, False, "PermissionError: denied")
monkeypatch.setattr(compositor, "_probe", refuse_with_eacces)
with caplog.at_level(logging.WARNING, logger="hhstreamd.compositor"):
assert discover(short_tmpdir).wayland_display == "wayland-1"
assert "did not accept a probe connection" in caplog.text
# ---------------------------------------------------------------------------
# the environment a child is given
# ---------------------------------------------------------------------------
def test_session_env_is_the_three_variables():
session = CompositorSession(runtime_dir=Path("/run/user/1001"), wayland_display="wayland-1")
assert session.env() == {
"XDG_RUNTIME_DIR": "/run/user/1001",
"WAYLAND_DISPLAY": "wayland-1",
"XDG_SESSION_TYPE": "wayland",
}
def test_child_env_is_built_from_the_compositor_runtime_dir(short_tmpdir, sockets, tmp_path):
sockets(short_tmpdir, "wayland-1")
launcher = Launcher(
tmp_path / "runtime",
tmp_path / "state",
compositor=CompositorTarget.from_runtime_dir(short_tmpdir),
)
env = launcher.build_env("chromium-uynn6w", tmp_path / "home")
assert env["XDG_RUNTIME_DIR"] == str(short_tmpdir.resolve())
assert env["WAYLAND_DISPLAY"] == "wayland-1"
assert env["XDG_SESSION_TYPE"] == "wayland"
# Not the daemon's own runtime dir, and not the session's private one.
assert env["XDG_RUNTIME_DIR"] != str(launcher.session_runtime_dir("chromium-uynn6w"))
# The private directory is still created: the daemon owns and cleans it up.
assert launcher.session_runtime_dir("chromium-uynn6w").is_dir()
def test_child_env_keeps_the_inherited_environment_underneath(
short_tmpdir, sockets, tmp_path, monkeypatch
):
sockets(short_tmpdir, "wayland-1")
monkeypatch.setenv("PATH", "/usr/bin")
monkeypatch.setenv("DBUS_SESSION_BUS_ADDRESS", "unix:path=/run/user/1000/bus")
launcher = Launcher(
tmp_path / "runtime",
tmp_path / "state",
compositor=CompositorTarget.from_runtime_dir(short_tmpdir),
)
env = launcher.build_env("s", tmp_path / "home")
assert env["PATH"] == "/usr/bin"
assert env["DBUS_SESSION_BUS_ADDRESS"] == "unix:path=/run/user/1000/bus"
assert env["HOME"] == str(tmp_path / "home")
assert env["HHSTREAM_SESSION_ID"] == "s"
def test_the_daemons_own_configuration_never_leaks_to_a_child(tmp_path, monkeypatch):
monkeypatch.setenv(ENV_COMPOSITOR_RUNTIME_DIR, "/run/user/1001")
env = Launcher(tmp_path / "runtime", tmp_path / "state").build_env("s", tmp_path)
assert ENV_COMPOSITOR_RUNTIME_DIR not in env
def test_without_a_compositor_the_inherited_environment_still_stands(tmp_path, monkeypatch):
"""Point 4 of the brief: the old behaviour is kept, not broken."""
monkeypatch.setenv("WAYLAND_DISPLAY", "wayland-7")
monkeypatch.delenv("XDG_SESSION_TYPE", raising=False)
launcher = Launcher(tmp_path / "runtime", tmp_path / "state")
env = launcher.build_env("s", tmp_path / "home")
assert env["XDG_RUNTIME_DIR"] == str(launcher.session_runtime_dir("s"))
assert env["WAYLAND_DISPLAY"] == "wayland-7" # passed through, not invented
assert "XDG_SESSION_TYPE" not in env
def test_the_recorded_runtime_dir_stays_the_one_the_daemon_may_delete(
short_tmpdir, sockets, tmp_path
):
"""``discard_runtime_dir`` must never be pointed at another user's directory."""
sockets(short_tmpdir, "wayland-1")
launcher = Launcher(
tmp_path / "runtime",
tmp_path / "state",
compositor=CompositorTarget.from_runtime_dir(short_tmpdir),
)
spawned = launcher.spawn("s", [sys.executable, "-c", "pass"])
spawned.process.wait(timeout=10)
assert spawned.runtime_dir == str(launcher.session_runtime_dir("s"))
assert spawned.runtime_dir != str(short_tmpdir.resolve())
def test_a_real_child_sees_the_compositor_environment(short_tmpdir, sockets, tmp_path):
"""The end of the chain: what the process actually gets after exec."""
sockets(short_tmpdir, "wayland-1")
launcher = Launcher(
tmp_path / "runtime",
tmp_path / "state",
compositor=CompositorTarget.from_runtime_dir(short_tmpdir),
)
code = (
"import json, os, sys; "
"sys.stdout.write(json.dumps({k: os.environ.get(k) for k in "
"('XDG_RUNTIME_DIR', 'WAYLAND_DISPLAY', 'XDG_SESSION_TYPE')}))"
)
spawned = launcher.spawn("s", [sys.executable, "-c", code])
assert spawned.process.wait(timeout=30) == 0
seen = json.loads(Path(spawned.log_path).read_text())
assert seen == {
"XDG_RUNTIME_DIR": str(short_tmpdir.resolve()),
"WAYLAND_DISPLAY": "wayland-1",
"XDG_SESSION_TYPE": "wayland",
}
# And the socket it was pointed at is really there.
assert stat.S_ISSOCK(os.stat(short_tmpdir / seen["WAYLAND_DISPLAY"]).st_mode)
# ---------------------------------------------------------------------------
# re-resolution: the compositor can restart under a running daemon
# ---------------------------------------------------------------------------
def test_target_follows_a_compositor_that_came_back_on_another_socket(short_tmpdir, sockets):
first = sockets(short_tmpdir, "wayland-0")
target = CompositorTarget.from_runtime_dir(short_tmpdir)
assert target.session.wayland_display == "wayland-0"
sockets(short_tmpdir, "wayland-1") # restarted compositor, new name
first.unlink() # the old socket file went with the old compositor
assert target.env()["WAYLAND_DISPLAY"] == "wayland-1"
assert target.session.wayland_display == "wayland-1"
def test_target_keeps_the_last_known_good_when_re_resolution_fails(short_tmpdir, sockets, caplog):
sockets(short_tmpdir, "wayland-1")
target = CompositorTarget.from_runtime_dir(short_tmpdir)
(short_tmpdir / "wayland-1").unlink() # compositor gone, mid-flight
with caplog.at_level(logging.WARNING, logger="hhstreamd.compositor"):
env = target.env()
assert env["WAYLAND_DISPLAY"] == "wayland-1"
assert "re-resolving the compositor socket failed" in caplog.text
# ---------------------------------------------------------------------------
# the daemon's startup contract
# ---------------------------------------------------------------------------
def _args(argv: list[str]):
return build_parser().parse_args(argv)
def test_the_option_and_the_environment_variable_are_both_accepted(monkeypatch):
monkeypatch.setenv(ENV_COMPOSITOR_RUNTIME_DIR, "/from/env")
assert _args([]).compositor_runtime_dir == "/from/env"
assert _args(["--compositor-runtime-dir", "/from/flag"]).compositor_runtime_dir == "/from/flag"
def test_no_option_means_no_compositor_and_a_startup_warning(monkeypatch, caplog):
monkeypatch.delenv(ENV_COMPOSITOR_RUNTIME_DIR, raising=False)
with caplog.at_level(logging.WARNING, logger="hhstreamd"):
assert resolve_compositor(_args([])) is None
assert "may not be able to reach a compositor" in caplog.text
assert "--compositor-runtime-dir" in caplog.text
def test_resolve_compositor_returns_a_target_when_the_socket_is_there(short_tmpdir, sockets):
sockets(short_tmpdir, "wayland-1")
target = resolve_compositor(_args(["--compositor-runtime-dir", str(short_tmpdir)]))
assert target is not None
assert target.session.wayland_display == "wayland-1"
def test_daemon_refuses_to_start_on_an_unusable_compositor_dir(short_tmpdir, capsys, monkeypatch):
"""Exit before listening, with the directory in the message."""
monkeypatch.delenv(ENV_COMPOSITOR_RUNTIME_DIR, raising=False)
code = main(
[
"--check",
"--socket",
str(short_tmpdir / "c.sock"),
"--state-dir",
str(short_tmpdir / "state"),
"--runtime-dir",
str(short_tmpdir / "runtime"),
"--config",
str(short_tmpdir / "missing.toml"),
"--compositor-runtime-dir",
str(short_tmpdir),
]
)
assert code == 2
err = capsys.readouterr().err
assert str(short_tmpdir.resolve()) in err
assert "no Wayland socket" in err
def test_check_reports_the_discovered_compositor(short_tmpdir, sockets, capsys, monkeypatch):
monkeypatch.delenv(ENV_COMPOSITOR_RUNTIME_DIR, raising=False)
runtime = short_tmpdir / "comp"
runtime.mkdir()
sockets(runtime, "wayland-1")
code = main(
[
"--check",
"--state-dir",
str(short_tmpdir / "state"),
"--runtime-dir",
str(short_tmpdir / "runtime"),
"--config",
str(short_tmpdir / "missing.toml"),
"--compositor-runtime-dir",
str(runtime),
]
)
assert code == 0
out = capsys.readouterr().out
assert f"compositor: {runtime.resolve()}" in out
assert "WAYLAND_DISPLAY=wayland-1" in out
def test_check_says_so_when_no_compositor_is_configured(short_tmpdir, capsys, monkeypatch):
monkeypatch.delenv(ENV_COMPOSITOR_RUNTIME_DIR, raising=False)
code = main(
[
"--check",
"--state-dir",
str(short_tmpdir / "state"),
"--runtime-dir",
str(short_tmpdir / "runtime"),
"--config",
str(short_tmpdir / "missing.toml"),
]
)
assert code == 0
assert "compositor: none configured" in capsys.readouterr().out
+104
View File
@@ -0,0 +1,104 @@
"""The exit-code table of DESIGN.md §1.6.
These are frozen: agents branch on them. The test states the table in full so
a change to it is a change to this file.
"""
from __future__ import annotations
import pytest
from hhstream_session.client import ClientError, reply_exit_code
from hhstreamd.protocol import EXIT_CODE_FOR_ERROR, ExitCode, exit_code_for
def test_the_frozen_numbers():
assert ExitCode.OK == 0
assert ExitCode.USAGE == 2
assert ExitCode.NO_SUCH_SESSION == 3
assert ExitCode.BAD_STATE == 4
assert ExitCode.DAEMON_UNREACHABLE == 5
assert ExitCode.POLICY == 6
assert ExitCode.TIMEOUT == 7
assert ExitCode.BUSY == 8
# 1 is deliberately absent: it is the shell's "something went wrong", and
# reusing it would make an unhandled crash indistinguishable from a
# documented outcome.
assert 1 not in set(ExitCode)
@pytest.mark.parametrize(
"error,code",
[
# 2 — usage
("bad_request", 2),
("unknown_op", 2),
("unsupported_version", 2),
("message_too_large", 2),
("invalid_argument", 2),
# 3 — no such session
("no_such_session", 3),
("no_such_process", 3),
# 4 — not in a usable state
("session_exited", 4),
("session_killed", 4),
("session_reaped", 4),
("session_starting", 4),
("compositor_lost", 4),
("bad_state", 4),
("pid_reuse_detected", 4),
("identity_unknown", 4),
# 5 — daemon unreachable
("daemon_unreachable", 5),
# 6 — refused by policy
("not_allowlisted", 6),
("argument_refused", 6),
("session_cap_reached", 6),
("refused", 6),
# 7 — timed out
("timeout", 7),
("readiness_timeout", 7),
# 8 — busy
("busy", 8),
("already_managed", 8),
],
)
def test_every_error_name_maps_to_its_documented_code(error, code):
assert exit_code_for(error) == code
def test_the_table_has_no_entries_this_test_does_not_know_about():
"""Adding an error name must mean adding it to the test above."""
documented = {
"bad_request", "unknown_op", "unsupported_version", "message_too_large",
"invalid_argument", "no_such_session", "no_such_process", "session_exited",
"session_killed", "session_reaped", "session_starting", "compositor_lost",
"bad_state", "pid_reuse_detected", "identity_unknown",
"daemon_unreachable", "not_allowlisted",
"argument_refused", "session_cap_reached", "refused", "timeout",
"readiness_timeout", "busy", "already_managed",
}
assert set(EXIT_CODE_FOR_ERROR) == documented
def test_success_is_zero():
assert exit_code_for(None) == 0
assert exit_code_for("") == 0
def test_an_unknown_error_degrades_to_not_usable_rather_than_to_success():
"""A caller that does not recognise the failure must still not see a 0."""
assert exit_code_for("something_invented_next_year") == ExitCode.BAD_STATE
def test_reply_exit_code_reads_a_decoded_reply():
assert reply_exit_code({"ok": True, "id": "a"}) == 0
assert reply_exit_code({"ok": False, "error": "no_such_session"}) == 3
assert reply_exit_code({"ok": False}) == ExitCode.BAD_STATE
def test_client_errors_carry_their_own_code():
assert ClientError("daemon_unreachable", "no socket").exit_code == 5
assert ClientError("timeout", "no reply").exit_code == 7
+240
View File
@@ -0,0 +1,240 @@
"""Allowlist and input-bound tests (DESIGN.md §8 item 1).
The allowlist is the main control: without it, `start /bin/sh` makes every
other control decorative.
"""
from __future__ import annotations
import tomllib
from pathlib import Path
import pytest
from hhstreamd.policy import (
MAX_ARG_BYTES,
MAX_ARGV_BYTES,
MAX_ARGV_ITEMS,
Policy,
PolicyError,
check_app_name,
check_argv,
load_policy,
policy_from_dict,
)
CONFIG = """
[defaults]
max_sessions = 2
idle_timeout_s = 10800
max_idle_timeout_s = 86400
[apps.chromium]
exec = "/run/current-system/sw/bin/chromium"
args_allow = ["--new-window", "--app=*", "--user-data-dir=*"]
[apps.foot]
exec = "/usr/bin/foot"
[apps.blender]
exec = "/usr/bin/blender"
args = ["--background"]
args_allow = ["*.blend"]
idle_timeout_s = 60
"""
@pytest.fixture
def policy() -> Policy:
return policy_from_dict(tomllib.loads(CONFIG), source="test-config")
# ---------------------------------------------------------------------------
# the allowlist itself
# ---------------------------------------------------------------------------
def test_an_allowlisted_app_resolves_to_its_absolute_path(policy: Policy):
rule, argv = policy.resolve("chromium", ["--new-window"])
assert rule.name == "chromium"
assert argv == ["/run/current-system/sw/bin/chromium", "--new-window"]
def test_an_app_that_is_not_in_the_allowlist_is_refused_with_a_policy_error(policy: Policy):
with pytest.raises(PolicyError) as exc:
policy.resolve("sh", [])
assert exc.value.error == "not_allowlisted"
assert "sh" in exc.value.reason
assert "blender, chromium, foot" in exc.value.hint
@pytest.mark.parametrize("name", ["/bin/sh", "./sh", "../../bin/sh", "sh;rm -rf /", "a b"])
def test_a_path_or_a_shell_fragment_is_not_even_a_valid_app_name(policy: Policy, name: str):
"""The caller names an app; it never names a binary."""
with pytest.raises(PolicyError) as exc:
policy.resolve(name, [])
assert exc.value.error == "invalid_argument"
def test_an_empty_allowlist_refuses_everything_and_says_so():
empty = Policy(source="/etc/hhstream/config.toml")
with pytest.raises(PolicyError) as exc:
empty.resolve("foot", [])
assert exc.value.error == "not_allowlisted"
assert "the allowlist is empty" in exc.value.hint
def test_fixed_args_from_the_config_precede_the_callers(policy: Policy):
_, argv = policy.resolve("blender", ["scene.blend"])
assert argv == ["/usr/bin/blender", "--background", "scene.blend"]
def test_an_app_rule_may_set_its_own_default_idle_timeout(policy: Policy):
assert policy.apps["blender"].idle_timeout_s == 60
assert policy.apps["foot"].idle_timeout_s is None
# ---------------------------------------------------------------------------
# argument allowlisting
# ---------------------------------------------------------------------------
def test_arguments_are_matched_against_patterns(policy: Policy):
policy.resolve("chromium", ["--app=https://example.com", "--user-data-dir=/tmp/x"])
def test_an_argument_that_matches_no_pattern_is_refused(policy: Policy):
with pytest.raises(PolicyError) as exc:
policy.resolve("chromium", ["--no-sandbox"])
assert exc.value.error == "argument_refused"
assert "--no-sandbox" in exc.value.reason
def test_an_app_with_no_args_allow_accepts_no_caller_arguments(policy: Policy):
policy.resolve("foot", []) # fine
with pytest.raises(PolicyError) as exc:
policy.resolve("foot", ["-e", "sh"])
assert exc.value.error == "argument_refused"
# ---------------------------------------------------------------------------
# config validation
# ---------------------------------------------------------------------------
def test_a_relative_exec_path_is_rejected_at_load_time():
with pytest.raises(PolicyError) as exc:
policy_from_dict({"apps": {"foot": {"exec": "foot"}}})
assert "absolute path" in exc.value.reason
def test_an_app_with_no_exec_is_rejected():
with pytest.raises(PolicyError):
policy_from_dict({"apps": {"foot": {}}})
@pytest.mark.parametrize("field", ["args", "args_allow"])
def test_args_fields_must_be_lists_of_strings(field):
with pytest.raises(PolicyError):
policy_from_dict({"apps": {"foot": {"exec": "/usr/bin/foot", field: "oops"}}})
def test_a_missing_config_file_yields_an_empty_allowlist_rather_than_a_crash(tmp_path: Path):
"""A misconfigured deployment must not look like a daemon crash."""
policy = load_policy(tmp_path / "absent.toml")
assert policy.apps == {}
assert policy.source.endswith("absent.toml")
def test_a_config_file_is_read_from_disk(tmp_path: Path):
path = tmp_path / "config.toml"
path.write_text(CONFIG)
policy = load_policy(path)
assert set(policy.apps) == {"chromium", "foot", "blender"}
assert policy.max_sessions == 2
# ---------------------------------------------------------------------------
# input bounds on `start`
# ---------------------------------------------------------------------------
def test_argv_must_be_a_list_not_a_string():
"""A string here would be the beginning of a shell command line."""
with pytest.raises(PolicyError) as exc:
check_argv("--new-window https://example.com")
assert exc.value.error == "invalid_argument"
def test_argv_items_must_be_strings():
with pytest.raises(PolicyError):
check_argv(["--a", 7])
def test_argv_length_is_bounded():
check_argv(["x"] * MAX_ARGV_ITEMS)
with pytest.raises(PolicyError) as exc:
check_argv(["x"] * (MAX_ARGV_ITEMS + 1))
assert "more than" in exc.value.reason
def test_a_single_argument_is_bounded():
with pytest.raises(PolicyError):
check_argv(["x" * (MAX_ARG_BYTES + 1)])
def test_the_total_argv_size_is_bounded():
item = "x" * (MAX_ARG_BYTES - 1)
count = MAX_ARGV_BYTES // len(item) + 2
with pytest.raises(PolicyError) as exc:
check_argv([item] * count)
assert "in total" in exc.value.reason
def test_a_nul_byte_in_an_argument_is_refused():
"""execve would truncate at the NUL; refusing is the honest answer."""
with pytest.raises(PolicyError) as exc:
check_argv(["--url=http://x\0--no-sandbox"])
assert "NUL" in exc.value.reason
def test_app_names_are_bounded_and_charset_checked():
check_app_name("chromium")
check_app_name("my-app_2.0+x")
with pytest.raises(PolicyError):
check_app_name("")
with pytest.raises(PolicyError):
check_app_name("a" * 65)
with pytest.raises(PolicyError):
check_app_name(None)
def test_an_idle_timeout_above_the_configured_maximum_is_refused(policy: Policy):
assert policy.check_idle_timeout(None) == 10800.0
assert policy.check_idle_timeout(60) == 60.0
with pytest.raises(PolicyError) as exc:
policy.check_idle_timeout(86401)
assert exc.value.error == "refused"
def test_a_negative_or_non_numeric_idle_timeout_is_refused(policy: Policy):
with pytest.raises(PolicyError):
policy.check_idle_timeout(-1)
with pytest.raises(PolicyError):
policy.check_idle_timeout("soon")
def test_a_zero_idle_timeout_is_accepted_and_means_never_expire(policy: Policy):
assert policy.check_idle_timeout(0) == 0.0
def test_the_session_cap_is_enforced(policy: Policy):
policy.check_capacity(1)
with pytest.raises(PolicyError) as exc:
policy.check_capacity(2)
assert exc.value.error == "session_cap_reached"
assert "max_sessions is 2" in exc.value.reason
+383
View File
@@ -0,0 +1,383 @@
"""`/proc` parsing and the safety guards on signalling.
The identity check is what makes it safe to signal a pid the daemon did not
spawn, so it gets tests against a synthetic `/proc` as well as the real one.
BUG-02 removed one of its three conditions, so these tests have two jobs rather
than one: they pin down that an `exec`ing wrapper no longer trips the gate, and
they pin down — in the same file, from the same synthetic `/proc` — that a
genuinely recycled pid is still refused. Dropping a condition without the
second half would leave a check that cannot fail, which is worse than the bug.
"""
from __future__ import annotations
import hashlib
import os
import signal
import subprocess
import sys
import time
from pathlib import Path
import pytest
from hhstreamd import procmon
def fake_proc(root: Path, pid: int, *, comm: str, sid: int, pgid: int, starttime: int, cmdline: bytes):
"""Write a plausible `/proc/<pid>` for the parser to read."""
directory = root / str(pid)
directory.mkdir(parents=True, exist_ok=True)
fields = ["0"] * 50
fields[0] = "S" # field 3: state
fields[1] = "1" # field 4: ppid
fields[2] = str(pgid) # field 5
fields[3] = str(sid) # field 6
fields[19] = str(starttime) # field 22
(directory / "stat").write_text(f"{pid} ({comm}) " + " ".join(fields) + "\n")
(directory / "cmdline").write_bytes(cmdline)
return directory
def test_stat_parsing_survives_a_comm_containing_spaces_and_parentheses(tmp_path: Path):
"""`comm` is attacker-adjacent: a process can rename itself to ") 1 2 3 (".
Splitting the whole line on whitespace would read the wrong fields and the
daemon would then signal the wrong process group.
"""
fake_proc(
tmp_path,
42,
comm="evil (name) with ) spaces",
sid=7,
pgid=9,
starttime=12345,
cmdline=b"/bin/x\0",
)
stat = procmon.read_stat(42, proc=str(tmp_path))
assert stat.comm == "evil (name) with ) spaces"
assert stat.pgid == 9
assert stat.sid == 7
assert stat.starttime == 12345
def test_a_missing_process_reads_as_none(tmp_path: Path):
assert procmon.read_stat(999, proc=str(tmp_path)) is None
assert procmon.read_cmdline(999, proc=str(tmp_path)) is None
assert procmon.identity(999, proc=str(tmp_path)) is None
assert procmon.is_alive(999, proc=str(tmp_path)) is False
def test_a_truncated_stat_line_is_rejected_rather_than_misparsed(tmp_path: Path):
directory = tmp_path / "7"
directory.mkdir()
(directory / "stat").write_text("7 (short) S 1 2\n")
assert procmon.read_stat(7, proc=str(tmp_path)) is None
def test_cmdline_is_split_on_nul_without_a_trailing_empty_item(tmp_path: Path):
fake_proc(tmp_path, 8, comm="x", sid=8, pgid=8, starttime=1, cmdline=b"/bin/x\0-a\0-b\0")
assert procmon.cmdline_argv(8, proc=str(tmp_path)) == ["/bin/x", "-a", "-b"]
def test_the_cmdline_hash_is_of_the_raw_bytes(tmp_path: Path):
raw = b"/bin/x\0-a\0"
fake_proc(tmp_path, 9, comm="x", sid=9, pgid=9, starttime=1, cmdline=raw)
assert procmon.cmdline_hash(9, proc=str(tmp_path)) == hashlib.sha256(raw).hexdigest()
# ---------------------------------------------------------------------------
# the identity check (BUG-02)
# ---------------------------------------------------------------------------
def leader(root: Path, pid: int, *, starttime: int, cmdline: bytes, comm: str = "x"):
"""A session leader, as `start_new_session=True` produces: sid == pgid == pid."""
return fake_proc(
root, pid, comm=comm, sid=pid, pgid=pid, starttime=starttime, cmdline=cmdline
)
def test_the_gate_is_starttime_corroborated_by_a_leaders_sid_and_pgid(tmp_path: Path):
"""What is checked is stated explicitly, so weakening it cannot be silent."""
raw = b"/bin/x\0"
leader(tmp_path, 10, starttime=999, cmdline=raw)
verdict = procmon.check(
10,
proc=str(tmp_path),
starttime=999,
sid=10,
pgid=10,
cmdline_sha256=hashlib.sha256(raw).hexdigest(),
)
assert verdict.ok is True
assert verdict.reason is None
assert verdict.checked == ("starttime", "sid", "pgid")
assert verdict.mismatched == ()
assert verdict.cmdline_drifted is False
@pytest.mark.parametrize(
"live,mismatched",
[
# A different process was created on this pid: it cannot share the
# creation instant of the one that is recorded.
(dict(sid=10, pgid=10, starttime=1000), "starttime"),
# ...and if the clocks did collide, it is not in our session either.
(dict(sid=77, pgid=77, starttime=999), "sid"),
(dict(sid=10, pgid=77, starttime=999), "pgid"),
],
)
def test_a_recycled_pid_is_still_refused(tmp_path: Path, live: dict, mismatched: str):
"""The protection BUG-02 had to keep: a pid that is not ours is not signalled.
The brief's construction: a pid cannot be forced to be recycled on demand,
so the record is held fixed — a session leader on pid 10, created at tick
999 — and `/proc` is made to describe a process that **cannot** be that one
while it is alive. Each surviving condition is shown to refuse on its own.
"""
fake_proc(tmp_path, 10, comm="stranger", cmdline=b"/bin/other\0", **live)
verdict = procmon.check(10, proc=str(tmp_path), starttime=999, sid=10, pgid=10)
assert verdict.ok is False, "a pid that is not ours must never be signalled"
assert verdict.reason == "pid_reuse_detected"
assert verdict.unresolved is True
assert mismatched in verdict.mismatched
def test_the_gate_cannot_be_satisfied_by_the_cmdline_alone(tmp_path: Path):
"""A recycled pid is refused *even when the command line matches exactly*.
The converse of dropping the condition: it was never what made the check
safe, and removing it takes nothing away.
"""
raw = b"/bin/x\0"
fake_proc(tmp_path, 10, comm="x", sid=10, pgid=10, starttime=1000, cmdline=raw)
verdict = procmon.check(
10, proc=str(tmp_path), starttime=999, sid=10, pgid=10,
cmdline_sha256=hashlib.sha256(raw).hexdigest(),
)
assert verdict.ok is False
assert verdict.reason == "pid_reuse_detected"
assert verdict.cmdline_drifted is False, "identical argv, and still refused"
def test_a_wrapper_script_exec_does_not_look_like_pid_reuse(tmp_path: Path):
"""BUG-02 itself, at the level of the check.
The recorded argv is the wrapper's, the live argv is the real binary's, and
everything the kernel owns is unchanged. This is a healthy process.
"""
launched = b"/run/current-system/sw/bin/chromium\0--ozone-platform=wayland\0"
live = b"/nix/store/abc-chromium-unwrapped/libexec/chromium/chromium\0"
leader(tmp_path, 2661433, starttime=15780209, cmdline=live, comm="chromium")
verdict = procmon.check(
2661433,
proc=str(tmp_path),
starttime=15780209,
sid=2661433,
pgid=2661433,
cmdline_sha256=hashlib.sha256(launched).hexdigest(),
)
assert verdict.ok is True, "an exec'ing wrapper is not evidence of pid reuse"
assert verdict.cmdline_drifted is True, "the drift is still reported, just not acted on"
def test_the_cmdline_can_never_refuse_on_its_own(tmp_path: Path):
"""The condition that was dropped is dropped: no value of it refuses."""
raw = b"/bin/real\0"
leader(tmp_path, 10, starttime=999, cmdline=raw)
for recorded_hash in ["0" * 64, hashlib.sha256(b"/bin/wrapper\0").hexdigest(), ""]:
verdict = procmon.check(
10, proc=str(tmp_path), starttime=999, sid=10, pgid=10,
cmdline_sha256=recorded_hash,
)
assert verdict.ok is True
assert "cmdline_sha256" not in verdict.checked
def test_sid_and_pgid_are_only_checked_for_a_leader(tmp_path: Path):
"""An adopted non-leader may lawfully `setsid()`; that is not pid reuse.
`setsid()` and `setpgid()` both fail with EPERM for a session/group leader,
so the recorded values are immutable exactly when they equal the recorded
pid — and are checked exactly then.
"""
# Recorded as a member of someone else's session, then it calls setsid().
fake_proc(tmp_path, 10, comm="x", sid=5, pgid=5, starttime=999, cmdline=b"/bin/x\0")
verdict = procmon.check(10, proc=str(tmp_path), starttime=999, sid=5, pgid=5)
assert verdict.ok is True
assert verdict.checked == ("starttime",), "a non-leader's sid/pgid are not evidence"
def test_a_record_with_no_starttime_is_refused_rather_than_waved_through(tmp_path: Path):
"""BUG-02 item 5: no evidence means refuse loudly, never "probably fine"."""
leader(tmp_path, 10, starttime=999, cmdline=b"/bin/x\0")
for missing in (None, 0, -1):
verdict = procmon.check(10, proc=str(tmp_path), starttime=missing, sid=10, pgid=10)
assert verdict.ok is False
assert verdict.reason == "identity_unknown"
assert verdict.unresolved is True
def test_a_vanished_process_reports_gone_not_pid_reuse(tmp_path: Path):
"""The difference matters: one is routine, the other is alarming."""
verdict = procmon.check(404, proc=str(tmp_path), starttime=1, sid=404, pgid=404)
assert (verdict.ok, verdict.reason) == (False, "gone")
assert verdict.unresolved is False, "an exit is not something to shout about"
def test_a_zombie_is_gone_not_alive(tmp_path: Path):
"""A zombie holds its pid but can never be signalled into doing anything."""
directory = leader(tmp_path, 10, starttime=999, cmdline=b"/bin/x\0")
stat = (directory / "stat").read_text().replace(") S ", ") Z ", 1)
(directory / "stat").write_text(stat)
verdict = procmon.check(10, proc=str(tmp_path), starttime=999, sid=10, pgid=10)
assert (verdict.ok, verdict.reason) == (False, "gone")
def test_verify_still_answers_ok_and_reason_for_callers_that_want_no_detail(tmp_path: Path):
leader(tmp_path, 10, starttime=999, cmdline=b"/bin/x\0")
assert procmon.verify(10, proc=str(tmp_path), starttime=999, sid=10, pgid=10) == (True, None)
assert procmon.verify(10, proc=str(tmp_path), starttime=998, sid=10, pgid=10) == (
False,
"pid_reuse_detected",
)
def test_the_real_process_verifies_against_itself():
me = os.getpid()
identity = procmon.identity(me)
assert identity is not None
assert identity.sid == os.getsid(me)
assert identity.pgid == os.getpgid(me)
verdict = procmon.check(
me,
starttime=identity.starttime,
sid=identity.sid,
pgid=identity.pgid,
cmdline_sha256=identity.cmdline_sha256,
)
assert verdict.ok is True
assert "starttime" in verdict.checked
def test_a_real_exec_does_not_change_starttime_but_does_change_the_cmdline():
"""The claim the whole fix rests on, measured on a real process.
A process that `exec`s something else keeps its pid, its session id, its
process group and its `starttime`; only its command line is replaced. The
handshake over stdin is what makes it deterministic — the child is blocked
in `readline()` while the "before" identity is taken, so there is no race
with the `exec` and no sleep sized by guesswork. If this ever stops being
true, the gate is unsound and this test says so.
"""
child = (
"import os, sys\n"
"sys.stdout.write(str(os.getpid()) + chr(10)); sys.stdout.flush()\n"
"sys.stdin.readline()\n" # held here while the parent looks at us
"os.execv(sys.executable, [sys.executable, '-c', 'import time; time.sleep(30)'])\n"
)
process = subprocess.Popen(
[sys.executable, "-c", child],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
start_new_session=True,
)
try:
pid = int(process.stdout.readline())
before = procmon.identity(pid)
assert before is not None
assert before.sid == before.pgid == pid, "start_new_session gives a leader"
process.stdin.write(b"go\n")
process.stdin.flush()
deadline = time.monotonic() + 10.0
while True:
after = procmon.identity(pid)
assert after is not None, "the process exec'd into nothing"
if after.cmdline_sha256 != before.cmdline_sha256:
break
assert time.monotonic() < deadline, "the exec never happened"
time.sleep(0.01)
assert after.starttime == before.starttime, "starttime must survive exec"
assert after.sid == before.sid == pid, "the session id must survive exec"
assert after.pgid == before.pgid == pid, "the process group must survive exec"
# And the gate agrees: the identity recorded before the exec still
# verifies afterwards, with the drift reported but not acted on.
verdict = procmon.check(
pid,
starttime=before.starttime,
sid=before.sid,
pgid=before.pgid,
cmdline_sha256=before.cmdline_sha256,
)
assert verdict.ok is True
assert verdict.cmdline_drifted is True
finally:
process.kill()
process.wait(timeout=5)
# ---------------------------------------------------------------------------
# signalling guards
# ---------------------------------------------------------------------------
@pytest.mark.parametrize("sid", [0, 1, -1, None])
def test_the_sweep_refuses_session_ids_that_would_select_the_machine(sid):
"""sid 0 or 1 would match init and most of everything else."""
assert procmon.session_members(sid) == []
assert procmon.sweep_session(sid) == []
@pytest.mark.parametrize("pgid", [0, 1, -1, None])
def test_killpg_refuses_the_dangerous_process_groups(pgid):
"""killpg(0) means "every process in my own group" — including the daemon."""
assert procmon.killpg(pgid, signal.SIGTERM) is False
def test_the_sweep_finds_processes_by_session_id(tmp_path: Path):
for pid, sid in [(100, 55), (101, 55), (102, 66)]:
fake_proc(tmp_path, pid, comm="x", sid=sid, pgid=pid, starttime=1, cmdline=b"/bin/x\0")
assert procmon.session_members(55, proc=str(tmp_path)) == [100, 101]
assert procmon.session_members(66, proc=str(tmp_path)) == [102]
assert procmon.session_members(55, proc=str(tmp_path), exclude={100}) == [101]
def test_signalling_a_dead_pid_is_not_an_error():
assert procmon.kill_pid(2**22 - 1, signal.SIGKILL) in (False, True)
def test_pidfd_open_yields_none_rather_than_raising_when_it_cannot_be_used():
"""The caller falls back to a /proc poll on None, so it must never raise.
This covers both reasons for None: the kernel lacking the syscall, and the
process being gone between the liveness check and the open.
"""
assert procmon.pidfd_open(2**22 - 1) is None
fd = procmon.pidfd_open(os.getpid())
if fd is None:
assert not procmon.HAVE_PIDFD
else:
os.close(fd)
+110
View File
@@ -0,0 +1,110 @@
"""Protocol framing tests (DESIGN.md §3.4)."""
from __future__ import annotations
import json
import pytest
from hhstreamd import protocol
from hhstreamd.protocol import MAX_LINE_BYTES, PROTOCOL_VERSION, ProtocolError
def test_a_message_is_exactly_one_newline_terminated_line():
frame = protocol.encode({"ok": True, "id": "foot-aaaaaa"})
assert frame.endswith(b"\n")
assert frame.count(b"\n") == 1
assert json.loads(frame) == {"ok": True, "id": "foot-aaaaaa"}
def test_round_trip():
message = {"v": 1, "op": "start", "app": "foot", "argv": ["--a", "--b"]}
assert protocol.decode(protocol.encode(message)) == message
def test_embedded_newlines_cannot_break_the_framing():
"""A session id or an argv element must never be able to split a frame."""
message = {"ok": False, "reason": "line one\nline two\r\nline three"}
frame = protocol.encode(message)
assert frame.count(b"\n") == 1
assert protocol.decode(frame) == message
def test_non_ascii_is_escaped_so_the_frame_stays_ascii():
frame = protocol.encode({"reason": "café — ✓"})
frame.decode("ascii") # must not raise
assert protocol.decode(frame)["reason"] == "café — ✓"
def test_decode_tolerates_a_missing_trailing_newline():
assert protocol.decode(b'{"ok":true}') == {"ok": True}
@pytest.mark.parametrize("line", [b"", b"\n", b" \n"])
def test_an_empty_frame_is_a_bad_request(line):
with pytest.raises(ProtocolError) as exc:
protocol.decode(line)
assert exc.value.error == "bad_request"
def test_malformed_json_is_a_bad_request():
with pytest.raises(ProtocolError) as exc:
protocol.decode(b"{not json\n")
assert exc.value.error == "bad_request"
assert "malformed JSON" in exc.value.reason
@pytest.mark.parametrize("payload", [b"[1,2,3]", b'"a string"', b"42", b"null"])
def test_a_frame_that_is_not_an_object_is_rejected(payload):
with pytest.raises(ProtocolError) as exc:
protocol.decode(payload)
assert exc.value.error == "bad_request"
def test_an_oversized_frame_is_refused_on_both_sides():
with pytest.raises(ProtocolError) as exc:
protocol.decode(b"x" * (MAX_LINE_BYTES + 1))
assert exc.value.error == "message_too_large"
with pytest.raises(ProtocolError) as exc:
protocol.encode({"argv": ["x" * MAX_LINE_BYTES]})
assert exc.value.error == "message_too_large"
def test_decode_request_requires_an_op():
with pytest.raises(ProtocolError) as exc:
protocol.decode_request(b'{"v":1}')
assert exc.value.error == "bad_request"
assert "no 'op'" in exc.value.reason
def test_decode_request_rejects_an_unknown_protocol_version():
with pytest.raises(ProtocolError) as exc:
protocol.decode_request(b'{"v":99,"op":"list"}')
assert exc.value.error == "unsupported_version"
def test_a_request_without_a_version_is_assumed_to_be_current():
assert protocol.decode_request(b'{"op":"list"}')["op"] == "list"
def test_request_attaches_the_version_and_drops_unset_fields():
message = protocol.request("start", app="foot", argv=None, idle_timeout_s=30.0)
assert message == {"v": PROTOCOL_VERSION, "op": "start", "app": "foot", "idle_timeout_s": 30.0}
def test_error_replies_carry_both_a_machine_field_and_a_human_field():
reply = protocol.error(
"not_allowlisted", "app 'sh' is not in the allowlist", hint="see the config"
)
assert reply == {
"ok": False,
"error": "not_allowlisted",
"reason": "app 'sh' is not in the allowlist",
"hint": "see the config",
}
def test_ok_replies_are_flagged():
assert protocol.ok(id="a")["ok"] is True
+215
View File
@@ -0,0 +1,215 @@
"""Reaper unit tests — injected clock, no processes, no sleeping.
DESIGN.md §6.5 layer 1: "tests pass a fake clock and assert which session is
selected and when". This is where nearly all of the reaping logic is covered.
"""
from __future__ import annotations
import asyncio
from hhstreamd.clock import ManualClock
from hhstreamd.reaper import Reaper, deadlines, next_deadline, select_expired, sleep_for
from hhstreamd.registry import State
class FakeSession:
"""The minimum the reaper needs: an id, liveness and a deadline."""
def __init__(self, sid: str, last_activity: float, timeout: float, state: str = State.READY):
self.id = sid
self.last_activity_boottime = last_activity
self.idle_timeout_s = timeout
self.state = state
@property
def live(self) -> bool:
return self.state not in {State.EXITED, State.KILLED, State.REAPED}
def deadline(self):
if not self.live or not self.idle_timeout_s:
return None
return self.last_activity_boottime + self.idle_timeout_s
# ---------------------------------------------------------------------------
# selection
# ---------------------------------------------------------------------------
def test_empty_set_has_no_deadline_and_sleeps_indefinitely():
assert deadlines([]) == {}
assert next_deadline([]) is None
assert select_expired([], now=10_000.0) == []
# None means "sleep until woken" — zero wakeups on a quiet server.
assert sleep_for([], now=10_000.0) is None
def test_next_deadline_is_the_earliest_across_sessions():
sessions = [
FakeSession("a", last_activity=1000.0, timeout=10800.0), # 11800
FakeSession("b", last_activity=5000.0, timeout=60.0), # 5060 <- earliest
FakeSession("c", last_activity=2000.0, timeout=3600.0), # 5600
]
assert next_deadline(sessions) == 5060.0
assert sleep_for(sessions, now=5000.0) == 60.0
def test_select_expired_returns_only_the_passed_deadlines_oldest_first():
sessions = [
FakeSession("late", last_activity=100.0, timeout=10.0), # 110
FakeSession("later", last_activity=100.0, timeout=5.0), # 105
FakeSession("future", last_activity=100.0, timeout=1000.0), # 1100
]
assert select_expired(sessions, now=120.0) == ["later", "late"]
assert select_expired(sessions, now=104.0) == []
def test_deadline_is_inclusive_so_an_exactly_due_session_is_reaped():
session = FakeSession("a", last_activity=100.0, timeout=10.0)
assert select_expired([session], now=110.0) == ["a"]
assert sleep_for([session], now=110.0) == 0.0
def test_a_terminal_session_has_no_deadline():
for state in (State.EXITED, State.KILLED, State.REAPED):
session = FakeSession("a", last_activity=0.0, timeout=1.0, state=state)
assert session.deadline() is None
assert select_expired([session], now=1e9) == []
def test_a_zero_timeout_disables_expiry_rather_than_expiring_immediately():
session = FakeSession("a", last_activity=0.0, timeout=0.0)
assert next_deadline([session]) is None
assert select_expired([session], now=1e9) == []
def test_sleep_for_never_goes_negative():
session = FakeSession("a", last_activity=0.0, timeout=10.0)
assert sleep_for([session], now=1_000_000.0) == 0.0
# ---------------------------------------------------------------------------
# the Reaper object, with the clock injected at construction
# ---------------------------------------------------------------------------
def make_reaper(clock: ManualClock, sessions: list[FakeSession]):
reaped: list[str] = []
async def on_expire(session_id: str) -> None:
reaped.append(session_id)
for session in sessions:
if session.id == session_id:
session.state = State.REAPED
return Reaper(now=clock.boottime, sessions=lambda: sessions, on_expire=on_expire), reaped
def test_clock_is_injected_and_nothing_expires_until_it_moves():
clock = ManualClock(boottime=1000.0)
sessions = [FakeSession("a", last_activity=1000.0, timeout=10800.0)]
reaper, reaped = make_reaper(clock, sessions)
assert reaper.next_deadline() == 11800.0
assert reaper.sleep_for() == 10800.0
assert reaper.expired() == []
clock.advance(10799.0)
assert reaper.expired() == []
clock.advance(1.0) # exactly three hours of idleness
assert reaper.expired() == ["a"]
assert reaped == [] # selection alone does not tear anything down
def test_tick_reaps_the_expired_and_leaves_the_rest():
clock = ManualClock(boottime=0.0)
sessions = [
FakeSession("doomed", last_activity=0.0, timeout=60.0),
FakeSession("safe", last_activity=0.0, timeout=3600.0),
]
reaper, reaped = make_reaper(clock, sessions)
clock.advance(61.0)
assert asyncio.run(reaper.tick()) == ["doomed"]
assert reaped == ["doomed"]
assert sessions[1].state == State.READY
# The reaped session now has no deadline, so the next one is the survivor's.
assert reaper.next_deadline() == 3600.0
def test_adopt_with_an_already_expired_deadline_is_reaped_on_the_first_pass():
"""DESIGN.md §2.4 item 6: the arithmetic survives a daemon restart.
A record adopted at boottime 50000 whose last activity was at 1000 with a
one-hour timeout is already 13 hours overdue. It must be selected
immediately, not scheduled another hour out.
"""
clock = ManualClock(boottime=50_000.0)
sessions = [FakeSession("adopted", last_activity=1000.0, timeout=3600.0)]
reaper, reaped = make_reaper(clock, sessions)
assert reaper.sleep_for() == 0.0
assert asyncio.run(reaper.tick()) == ["adopted"]
assert reaped == ["adopted"]
def test_a_touch_pushes_the_deadline_out():
clock = ManualClock(boottime=0.0)
session = FakeSession("a", last_activity=0.0, timeout=100.0)
reaper, _ = make_reaper(clock, [session])
clock.advance(99.0)
assert reaper.expired() == []
session.last_activity_boottime = clock.boottime() # what `touch` does
clock.advance(99.0)
assert reaper.expired() == []
clock.advance(2.0)
assert reaper.expired() == ["a"]
def test_run_reaps_when_the_deadline_arrives_and_then_sleeps_indefinitely():
"""Exercise the real asyncio task without waiting for a real timeout."""
async def scenario():
clock = ManualClock(boottime=0.0)
sessions = [FakeSession("a", last_activity=0.0, timeout=10_000.0)]
reaper, reaped = make_reaper(clock, sessions)
task = reaper.start()
await asyncio.sleep(0) # let the first pass run
assert reaped == []
clock.advance(10_001.0) # three hours pass in a nanosecond
reaper.wake()
for _ in range(20):
await asyncio.sleep(0)
if reaped:
break
assert reaped == ["a"]
# Nothing left to expire: the task is parked, not spinning.
assert reaper.sleep_for() is None
await reaper.stop()
assert task.done()
asyncio.run(scenario())
def test_a_failing_teardown_does_not_kill_the_reaper_task():
async def scenario():
clock = ManualClock(boottime=0.0)
sessions = [FakeSession("a", last_activity=0.0, timeout=1.0)]
calls = []
async def on_expire(session_id: str) -> None:
calls.append(session_id)
raise RuntimeError("teardown exploded")
reaper = Reaper(now=clock.boottime, sessions=lambda: sessions, on_expire=on_expire)
clock.advance(2.0)
assert await reaper.tick() == [] # the failure is swallowed and logged
assert calls == ["a"]
assert reaper.expired() == ["a"] # still expired; it will be retried
asyncio.run(scenario())
+270
View File
@@ -0,0 +1,270 @@
"""Registry unit tests: atomic persistence, adoption, tombstones."""
from __future__ import annotations
import base64
import json
import os
from pathlib import Path
import pytest
from hhstreamd.clock import ManualClock
from hhstreamd.registry import (
Registry,
Session,
State,
atomic_write_json,
mint_id,
)
def make_session(sid: str = "foot-aaaaaa", **overrides) -> Session:
base = dict(
id=sid,
app="foot",
argv=["/usr/bin/foot"],
pid=4242,
pgid=4242,
sid=4242,
pid_starttime=918342,
cmdline_sha256="9f2c",
home="/home/someone",
runtime_dir="/run/hhstream/sessions/" + sid,
created_at="2026-10-01T09:14:02Z",
created_boottime=1000.0,
last_activity_at="2026-10-01T09:14:02Z",
last_activity_boottime=1000.0,
idle_timeout_s=10800.0,
state=State.READY,
)
base.update(overrides)
return Session(**base)
@pytest.fixture
def registry(tmp_path: Path) -> Registry:
reg = Registry(tmp_path, ManualClock(boottime=1000.0))
reg.ensure_dirs()
return reg
# ---------------------------------------------------------------------------
# atomic persistence
# ---------------------------------------------------------------------------
def test_atomic_write_leaves_no_temporary_files_behind(tmp_path: Path):
target = tmp_path / "rec.json"
atomic_write_json(target, {"hello": "world"})
assert json.loads(target.read_text()) == {"hello": "world"}
assert [p.name for p in tmp_path.iterdir()] == ["rec.json"]
def test_a_failed_write_leaves_the_previous_record_intact(tmp_path: Path, monkeypatch):
"""The point of tmp + os.replace: a reader never sees a half-written record."""
target = tmp_path / "rec.json"
atomic_write_json(target, {"generation": 1})
def explode(*args, **kwargs):
raise OSError("disk full")
monkeypatch.setattr(json, "dump", explode)
with pytest.raises(OSError):
atomic_write_json(target, {"generation": 2})
assert json.loads(target.read_text()) == {"generation": 1}
assert [p.name for p in tmp_path.iterdir()] == ["rec.json"] # temp cleaned up
def test_the_temp_file_is_written_in_the_target_directory(tmp_path: Path, monkeypatch):
"""os.replace is only atomic within one filesystem, so the temp must be adjacent."""
seen: list[str] = []
real_replace = os.replace
def spy(src, dst, *args, **kwargs):
seen.append(str(src))
return real_replace(src, dst, *args, **kwargs)
monkeypatch.setattr(os, "replace", spy)
target = tmp_path / "deep" / "rec.json"
atomic_write_json(target, {"a": 1})
assert Path(seen[0]).parent == target.parent
def test_persist_writes_the_record_and_it_round_trips(registry: Registry):
session = registry.add(make_session())
path = registry.record_path(session.id)
assert path.exists()
restored = Session.from_dict(json.loads(path.read_text()))
assert restored == session
def test_a_mirror_write_failure_never_fails_the_command(registry: Registry, monkeypatch, caplog):
session = registry.add(make_session())
monkeypatch.setattr(
"hhstreamd.registry.atomic_write_json",
lambda *a, **k: (_ for _ in ()).throw(OSError("read-only filesystem")),
)
registry.touch(session, force_flush=True) # must not raise
assert session.command_count == 1
assert "mirror write failed" in caplog.text
def test_activity_flushes_are_rate_limited_but_transitions_are_not(registry: Registry):
clock: ManualClock = registry.clock # type: ignore[assignment]
session = registry.add(make_session())
registry.touch(session) # within the 15 s window: memory only
on_disk = Session.from_dict(json.loads(registry.record_path(session.id).read_text()))
assert session.command_count == 1
assert on_disk.command_count == 0
clock.advance(16.0)
registry.touch(session)
on_disk = Session.from_dict(json.loads(registry.record_path(session.id).read_text()))
assert on_disk.command_count == 2
# ---------------------------------------------------------------------------
# handles
# ---------------------------------------------------------------------------
def test_a_colliding_draw_is_discarded_and_another_taken():
"""Handles are never reused: a draw already in the set is refused."""
class TwoDraws:
def __init__(self):
self.calls = 0
def token_bytes(self, n):
self.calls += 1
return b"\x00" * n if self.calls == 1 else b"\xff" * n
rng = TwoDraws()
collision = f"foot-{base64.b32encode(b'\x00' * 5).decode().rstrip('=').lower()[:6]}"
minted = mint_id("foot", {collision}, rng=rng)
assert rng.calls == 2
assert minted != collision
assert minted.startswith("foot-")
def test_a_tombstoned_id_still_blocks_reuse(registry: Registry):
session = registry.add(make_session())
registry.terminate(session, State.KILLED)
assert session.id in registry.taken_ids()
assert registry.new_id("foot") != session.id
def test_the_slug_is_sanitised_but_never_parsed():
assert mint_id("Chromium", set()).startswith("chromium-")
assert mint_id("/bin/odd name", set()).startswith("bin-odd-name-")
assert mint_id("!!!", set()).startswith("app-")
# ---------------------------------------------------------------------------
# tombstones
# ---------------------------------------------------------------------------
def test_terminate_writes_a_tombstone_with_an_expiry(registry: Registry):
clock: ManualClock = registry.clock # type: ignore[assignment]
session = registry.add(make_session())
registry.terminate(session, State.EXITED, exit_code=0, exit_detected_by="sigchld")
assert not session.live
assert session.exit_code == 0
assert session.tombstoned_boottime == clock.boottime()
assert session.tombstone_expires() == clock.boottime() + 3600.0
assert session.deadline() is None # a tombstone can never be idle-reaped
assert registry.tombstones() == [session]
assert registry.live() == []
def test_an_adopted_exit_records_no_exit_code_rather_than_inventing_a_zero(registry: Registry):
session = registry.add(make_session(adopted=True))
registry.terminate(session, State.EXITED, exit_detected_by="pidfd")
assert session.exit_code is None
assert session.exit_detected_by == "pidfd"
def test_drop_removes_the_record_and_its_mirror(registry: Registry):
session = registry.add(make_session())
path = registry.record_path(session.id)
registry.drop(session.id)
assert not path.exists()
assert registry.get(session.id) is None
# ---------------------------------------------------------------------------
# loading / adoption
# ---------------------------------------------------------------------------
def test_load_records_reads_back_what_was_persisted(tmp_path: Path):
writer = Registry(tmp_path, ManualClock())
writer.ensure_dirs()
writer.add(make_session("foot-aaaaaa"))
writer.add(make_session("foot-bbbbbb", pid=99))
reader = Registry(tmp_path, ManualClock())
records, quarantined = reader.load_records()
assert quarantined == []
assert sorted(r.id for r in records) == ["foot-aaaaaa", "foot-bbbbbb"]
def test_a_corrupt_record_is_quarantined_not_deleted_and_not_acted_on(tmp_path: Path, caplog):
registry = Registry(tmp_path, ManualClock())
registry.ensure_dirs()
bad = registry.sessions_dir / "foot-broken.json"
bad.write_text("{this is not json")
records, quarantined = registry.load_records()
assert records == []
assert len(quarantined) == 1
assert not bad.exists()
moved = list(registry.quarantine_dir.iterdir())
assert len(moved) == 1
assert moved[0].read_text() == "{this is not json"
assert "quarantining unreadable record" in caplog.text
def test_a_record_that_is_not_an_object_is_quarantined(tmp_path: Path):
registry = Registry(tmp_path, ManualClock())
registry.ensure_dirs()
(registry.sessions_dir / "x.json").write_text("[1, 2, 3]")
records, quarantined = registry.load_records()
assert records == [] and len(quarantined) == 1
def test_unknown_fields_from_a_newer_schema_are_dropped_not_fatal(caplog):
data = make_session().to_dict()
data["invented_by_a_later_version"] = True
session = Session.from_dict(data)
assert session.id == "foot-aaaaaa"
assert "dropping unknown record fields" in caplog.text
def test_a_record_from_a_previous_boot_is_recognised_as_stale(tmp_path: Path):
"""Pids restart from zero at boot, so /proc cannot be trusted about them."""
registry = Registry(tmp_path, ManualClock())
this_boot = make_session("foot-aaaaaa")
this_boot.boot_id = registry._boot_id
previous_boot = make_session("foot-bbbbbb")
previous_boot.boot_id = "00000000-0000-0000-0000-000000000000"
assert registry.stale_boot(this_boot) is False
assert registry.stale_boot(previous_boot) is True
def test_derived_fields_reported_to_callers(registry: Registry):
session = registry.add(make_session())
public = session.public(now=1000.0 + 600.0)
assert public["idle_s"] == 600.0
assert public["expires_in_s"] == 10800.0 - 600.0
assert public["live"] is True
assert public["schema"] == 2
+67
View File
@@ -0,0 +1,67 @@
"""Bounded tail reads.
Both the audit log and an application's captured output are unbounded in M0, and
both are read by verbs that must stay cheap.
"""
from __future__ import annotations
from pathlib import Path
from hhstreamd.textio import tail_lines
def test_the_last_n_lines_are_returned(tmp_path: Path):
path = tmp_path / "f.log"
path.write_text("".join(f"line {i}\n" for i in range(100)))
lines, truncated, size = tail_lines(path, 3)
assert lines == ["line 97", "line 98", "line 99"]
assert truncated is True # there was more above the window
assert size == path.stat().st_size
def test_a_short_file_is_returned_whole_and_not_marked_truncated(tmp_path: Path):
path = tmp_path / "f.log"
path.write_text("a\nb\n")
assert tail_lines(path, 10) == (["a", "b"], False, 4)
def test_a_large_file_is_never_read_whole(tmp_path: Path):
"""A gigabyte of Chromium stderr must not become a daemon-sized allocation."""
path = tmp_path / "big.log"
path.write_text("".join(f"{i:07d} padding padding padding\n" for i in range(40_000)))
assert path.stat().st_size > 1_000_000
lines, truncated, size = tail_lines(path, 5, max_bytes=4096)
assert len(lines) == 5
assert lines[-1] == "0039999 padding padding padding"
assert truncated is True
assert size == path.stat().st_size
def test_a_partial_first_line_in_the_window_is_dropped_not_reported_as_whole(tmp_path: Path):
path = tmp_path / "f.log"
path.write_text("aaaaaaaaaaaaaaaaaaaa\nbbbb\ncccc\n")
lines, truncated, _ = tail_lines(path, 10, max_bytes=12)
assert "aaaa" not in "".join(lines)
assert lines == ["cccc"] or lines == ["bbbb", "cccc"]
assert truncated is True
def test_a_missing_file_is_empty_rather_than_an_error(tmp_path: Path):
assert tail_lines(tmp_path / "absent.log", 10) == ([], False, 0)
def test_asking_for_no_lines_still_reports_the_size(tmp_path: Path):
path = tmp_path / "f.log"
path.write_text("hello\n")
assert tail_lines(path, 0) == ([], False, 6)
def test_invalid_utf8_is_replaced_not_fatal(tmp_path: Path):
path = tmp_path / "f.log"
path.write_bytes(b"ok\n\xff\xfe bad\n")
lines, _, _ = tail_lines(path, 10)
assert lines[0] == "ok"
assert "bad" in lines[1]