From 0f2810afaadf1a5ae56b8c4eeef096a61ac3f2f8 Mon Sep 17 00:00:00 2001 From: Hermenix Date: Sun, 11 Oct 2026 22:46:54 +0000 Subject: [PATCH 01/12] docs(sessions)!: sessions-v4, the per-session live stream MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `docs/CONTRACT-sessions.md` goes to sessions-v4 with a change table at the top, for the reason mcp-v2 got one: everything in it is an addition, and a reader who learned v3 should be told what moved rather than diff it. The new §3.5 is the interface that needed freezing most, because it is the only one in this build that two *processes* have to agree on: `/run/hhstream/live.sock`, `LiveStreams.attach/detach/list/viewers`, the four JSON ops on the wire, and the `Attachment` the broker gets back. Its shape is §3.4's deliberately -- the broker holds the decision (which session streams, under which phrase), the daemon holds the capture -- so a reader who knows the pointer driver already knows this. Four rules in §3.5 are written out because each one is a thing a reasonable implementation would get wrong: * the daemon holds no session state and no idle clock; `viewers` is a reading, and what the broker does with it is the broker's; * `fps` is optional and omitting it is normal -- the default is the daemon's, from its own unit's argv, for the same reason the port is. A broker that sent its own default on every attach would turn `sessions.stream.fps` into a setting that changed nothing, which is the failure `--xkb-layout` exists to avoid; * `attach` is idempotent on the phrase, so a re-`enable` after a broker restart does not double the `grim` rate on one output; * the phrase is never logged by either side. It is the stream's only credential. §4 gains four verbs, the three `start` arguments, and the one genuinely new rule in this version: **a connected viewer is activity**. That is the first exception to "`touch` is the only mark of activity" since that rule was written, and it is there for one failure -- an operator watching a long job on his phone, saying nothing to the broker, and losing the session at three hours *because* watching was all he was doing. It cannot hold a session open by itself: an enabled stream with no viewer runs no `grim` and stamps nothing. No new exit numbers. The three error names map onto codes the frozen table already has (`stream_unavailable` -> 10, `stream_busy` -> 8, `stream_budget` -> 6), because no new *kind* of failure appeared: no container output is a compositor-side failure, a phrase collision is a busy, a viewer cap is a budget. §6 records schema 3 -> 4 (the record gains `stream`, older records migrate to `null`), and §5 names the socket, the unit and port 9988 beside the other frozen paths. What v4 deliberately withholds from the broker is the port, the bind address and the capture command. The broker learns the URL prefix *from* the daemon, in the `attach` reply -- a URL it assembled itself from a port it was told about once is a URL that lies the first time an operator moves the listener. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_011ZeH65zd6pFTgBSu52W9PE --- docs/CONTRACT-sessions.md | 173 +++++++++++++++++++++++++++++++++++--- 1 file changed, 162 insertions(+), 11 deletions(-) diff --git a/docs/CONTRACT-sessions.md b/docs/CONTRACT-sessions.md index 455ff0c..eb17fc6 100644 --- a/docs/CONTRACT-sessions.md +++ b/docs/CONTRACT-sessions.md @@ -1,4 +1,4 @@ -# CONTRACT — sessions build (sessions-v3) +# CONTRACT — sessions build (sessions-v4) This file is the interface for the **session** work: the broker, containers, compositor IPC and pointer **and keyboard** input. It is the sibling of `docs/CONTRACT.md`, which @@ -9,11 +9,31 @@ else. If something here is wrong or missing, **stop and report it in your final message** — the parent edits this file, not you. Do not widen your slice to work around a gap. -Version: sessions-v3 (2026-10-04). v1 was the freeze the four slices built +Version: sessions-v4 (2026-10-11). v1 was the freeze the four slices built against; v2 recorded the decisions their reports asked for; v3 adds the -keyboard. Every change in both has been an addition or a clarification — **no -v1 or v2 signature changed meaning** — so earlier code stays valid except where -noted. +keyboard; v4 adds the **per-session live stream**. Every change in all of them +has been an addition or a clarification — **no v1, v2 or v3 signature changed +meaning** — so earlier code stays valid except where noted. + +v4 changes, all of them the stream +(`docs/plans/2026-10-04-session-stream-webapp.md`, Stage 2): + +| § | Change | Why | +| --- | --- | --- | +| §3.5 | New: `hhstreamd/live.py`, the broker's end of `/run/hhstream/live.sock`, and that socket's wire format. | The stream is captured by `hhstream-live`, a third executor beside the pointer driver. `grim` blocks tens to hundreds of milliseconds, so it must not run in the broker, and a LAN-facing HTTP parser must not run in the frame server whose one-socket contract cannot grow one. The broker therefore *tells* a process what to publish, exactly as it tells the driver where to warp. | +| §4 | Verbs `stream.enable`, `stream.disable`, `stream.list`, `stream.status`. `enable` and `disable` are in the mutation set; `list` and `status` are not. | `enable` mints or rotates a phrase and opens a capture — a retried `enable` that minted a second phrase would invalidate the URL somebody is already holding. | +| §4 | `start` gains `stream?`, `stream_phrase?` and `stream_fps?`. | A stream needs the session's container output, which `start` creates; sequencing "start, then enable" in a *caller* means every caller repeats the ordering and the failure handling. One request, one answer, and the phrase comes back with the session. | +| §4 | **A connected viewer is activity.** The broker asks the daemon for viewer counts about once a minute and stamps `touch` when any is non-zero. | The one new way the idle clock moves, and the only exception to "`touch` is the only mark of activity" since that rule was written. A session an operator is watching on his phone must not be reaped at three hours *because* watching is all he is doing. An enabled stream with **no** viewer runs no `grim` and keeps nothing alive, so this cannot hold a session open by itself. | +| §4 | Error names `stream_unavailable` (exit 10), `stream_busy` (exit 8), `stream_budget` (exit 6). **No new exit numbers.** | The failures are the ones the table already has: no container is a compositor-side failure, a phrase collision is a busy, a viewer cap is a policy budget. | +| §5 | `/run/hhstream/live.sock`, the unit `hhstream-live`, and the stream port 9988. | Named here with the other frozen paths, because two processes have to agree on them. | +| §6 | Registry **schema 3 → 4**: the record gains `stream`. | A phrase has to survive a broker restart and a re-enable, so it lives in the record. Older records migrate to `stream: null` and are adopted, not refused. | + +v4 deliberately does **not** give the broker the port, the bind address or the +capture command. Those are the daemon's, from its own unit's argv, for the +reason the keyboard layout is the driver's: the broker's TOML is read by +`hhstreamd`, and a setting an operator could change in it while the listener +stayed where it was would be worse than no setting at all. The broker learns +the URL prefix *from* the daemon, in the `attach` reply. v3 changes, all of them keyboard (`docs/plans/2026-10-03-keyboard-input.md` as amended by `docs/plans/2026-10-04-keyboard-verification.md`): @@ -375,6 +395,79 @@ the lease; a real-compositor case is still missing — see lease state** — the broker owns the lease; the driver is an executor that must never be trusted to serialise anything. +### 3.5 `hhstreamd/live.py` (v4) — the broker's end of the stream daemon + +A third executor beside the pointer driver, and the same shape as §3.4: the +broker holds the *decision* (which session streams, under which phrase) and the +daemon holds the *capture*. `hhstream-live` owns the port, the bind address and +the capture command, from its own unit's argv; the broker never passes them and +never reads them from its own TOML. + +```python +class LiveUnavailable(Exception): + """The stream daemon could not be reached, or did not answer intelligibly.""" + +class LiveRejected(Exception): + """The daemon refused. `kind` maps to an exit code, as the driver's does.""" + def __init__(self, message: str, *, kind: str | None = None) -> None: ... + +@dataclass(frozen=True) +class Attachment: + phrase: str; output: str; fps: int; session: str | None + url: str # the daemon's own prefix, e.g. http://127.0.0.1:9988// + viewers: int + +class LiveStreams: + def __init__(self, socket_path: Path | str, *, timeout_s: float = 2.0) -> None: ... + def attach(self, *, phrase: str, output: str, fps: int, + session: str | None = None, label: str = "") -> Attachment: ... + def detach(self, phrase: str) -> bool: ... # False if it was not attached + def list(self) -> list[Attachment]: ... + def viewers(self) -> dict[str, int]: ... # phrase -> connected viewers + def close(self) -> None: ... +``` + +Private socket protocol, newline-delimited JSON, the driver's conventions +exactly — `{"ok": true, …}` or `{"ok": false, "error": "…", "kind": "…"}`: + +```json +{"op": "attach", "phrase": "amber-otter-lantern-harbor-7k2qp", "output": "hh-x", + "session": "x", "label": "chromium", "fps": 5} // fps is OPTIONAL + -> {"ok": true, "stream": {"phrase": "…", "output": "hh-x", + "fps": 5, "session": "x", "url": "http://…/", "viewers": 0}} +{"op": "detach", "phrase": "…"} -> {"ok": true, "detached": true} +{"op": "list"} -> {"ok": true, "streams": [ {…}, … ]} +{"op": "viewers"} -> {"ok": true, "viewers": {"": 2}} +``` + +Four rules that are not interchangeable: + +- **The daemon holds no session state and no idea of an idle clock.** It + publishes what it was told to publish. `viewers` is a *reading*, and what the + broker does with it (§4) is the broker's. +- **`fps` is optional, and omitting it is the normal case.** The default rate is + the daemon's, from its own unit's argv (`sessions.stream.fps`), for the same + reason the port is. The broker sends one only when a caller named one, or + when re-attaching a stream that already had one. A broker that sent its own + default on every attach would make the operator's option a setting that + changed nothing — the same failure `--xkb-layout` (§3.4) exists to avoid. +- **`attach` is idempotent on the phrase.** Re-attaching the same phrase to the + same output returns the existing stream rather than starting a second + capture, because a re-`enable` after a broker restart must not double the + `grim` rate on one output. The same phrase aimed at a *different* output is + `{"kind": "invalid_argument"}`. +- **The phrase is never logged by either side.** It is the stream's only + credential. The daemon logs the output and the viewer count; the broker's + audit log records the session and the output (§4), and neither records the + phrase. + +`url` comes back from the daemon rather than being built by the broker, for the +reason the port is the daemon's: a URL the broker assembled from a port it was +told about once is a URL that silently lies the first time an operator moves +the listener. + +--- + --- ## 4. Wire format (broker control socket) — B, C and D do not implement this @@ -395,6 +488,44 @@ code will be called with): | `pointer.state` | — | `holder`, `since`, `lease_id`, `buttons[]`, `keys[]` | | `keyboard.key` (v3) | `id`, `lease_id`, `key`, `state` | `keys[]` | | `keyboard.type` (v3) | `id`, `lease_id`, `text` | `chars`, `keys[]` | +| `stream.enable` (v4) | `id`, `phrase?`, `fps?` | `stream` | +| `stream.disable` (v4) | `id` | `stream`, `detached` | +| `stream.list` (v4) | — | `streams[]`, `available` | +| `stream.status` (v4) | `id?` | `stream` \| `streams[]` | + +**v4, the stream.** `stream.enable` and `stream.disable` are in the +`request_id` mutation set; `stream.list` and `stream.status` are not. +`start` additionally accepts `stream?` (bool), `stream_phrase?` and +`stream_fps?`, and when `stream` is true its reply's session carries the same +`stream` object `stream.enable` would have returned. + +The `stream` object is the record's, with the daemon's `url` and `viewers` +merged in when it could be asked: + +```json +{"enabled": true, "phrase": "amber-otter-lantern-harbor-7k2qp", + "output": "hh-x", "fps": 5, "port": 9988, + "url": "http://127.0.0.1:9988/amber-otter-lantern-harbor-7k2qp/", + "viewers": 0, "enabled_at": "2026-10-11T…Z", "attached": true} +``` + +`attached` is its own field and is **not** folded into `enabled`: a record may +say a session is enabled while the daemon is down, and "the operator asked for +this stream" and "a capture is running right now" are two different facts. A +caller that wants the second one reads `attached`; `url` and `viewers` are +absent when it is false, rather than stale. + +**`stream.enable` requires the session's container output.** Without one it is +`stream_unavailable` (exit 10). A stream never falls back to the main output: +that output carries every other session, and putting it on a phone is the one +mistake this feature must not make. Re-enabling an enabled session returns the +same phrase; a `phrase` that differs from the stored one is `stream_busy` +(exit 8). `disable` then `enable` rotates it. + +**Teardown order is load-bearing.** `kill` detaches the stream *before* +`hyprctl output remove`, because a capture aimed at a removed output fails every +time and the daemon's own failure budget would close the viewers with +"unknown output" instead of "the session ended". **v3, the keyboard.** `keyboard.*` verbs take the **pointer** lease: there is no `keyboard.acquire` and no second lease id. One `wl_seat` means one keyboard @@ -417,10 +548,26 @@ explicitly rather than inheriting. | 0 OK | 2 USAGE | 3 NO_SUCH_SESSION | 4 BAD_STATE | 5 DAEMON_UNREACHABLE | | 6 POLICY | 7 TIMEOUT | 8 BUSY | **9 NO_LEASE (new)** | **10 COMPOSITOR (new)** | -Activity accounting is unchanged and load-bearing: **`touch` is the only mark of -activity**; `list`, `status`, `logs`, `gc`, and every `pointer.*` **and -`keyboard.*`** verb do **not** reset the idle clock. A session being typed into -is still a session nobody has said they want kept. +**v4 adds error names and no numbers.** `stream_unavailable` → 10 (no +container, or the daemon is unreachable — the session itself is fine either +way), `stream_busy` → 8 (a `phrase` that is not this session's), `stream_budget` +→ 6 (the daemon's viewer or stream cap). A bad `fps` is `invalid_argument` → 2, +as every other bad argument is. + +Activity accounting is load-bearing and has **one** exception as of v4: +**`touch` is the only mark of activity a caller can make**; `list`, `status`, +`logs`, `gc`, every `pointer.*` **and `keyboard.*`** verb, and every +`stream.*` verb do **not** reset the idle clock. A session being typed into is +still a session nobody has said they want kept. + +The exception is **a connected viewer**. About once a minute the broker asks +the daemon for viewer counts and stamps `touch` on every session whose stream +has at least one; the audit entry names the session, the output and the count, +never the phrase. This is not a caller's verb and cannot be invoked — it is the +broker observing that a person is *looking*, which is the one form of "still +wanted" that does not come down the control socket. It cannot hold a session +open on its own: an enabled stream with no viewer runs no `grim` and stamps +nothing. The *lease* clock is a different one, and v3 widens it: every `pointer.*` and `keyboard.*` gesture verb refreshes the lease's idle deadline, because they all @@ -438,14 +585,18 @@ go through the same gate. `pointer acquire`'s own stderr says so. | Frame socket (published) | `/stream.sock` — default `/run/hypr-stream/stream.sock`. Also frozen in `docs/CONTRACT.md`, which v1 of this file left disagreeing with it. | | Broker control socket | `/run/hhstream/control.sock` | | Broker↔driver socket | `/run/hhstream/pointer.sock` | +| Broker↔stream socket | `/run/hhstream/live.sock` (v4) | +| Stream port | **option default** `9988` (`sessions.stream.port`), fixed and single — no range, no fallback. The daemon owns it; the broker learns the URL from the daemon (§3.5). | +| Stream bind | **option default** `127.0.0.1` (`sessions.stream.bind`) | | Broker state dir | **option default** `/var/lib/hhstream`; this machine sets `/var/lib/hermes/hhstream-sessions` (`sessions.stateDir`). v1 listed the deployed value, which read as a contradiction of the module. | | Session runtime dir | `/run/hhstream/sessions/` | | Container output name | `hh-` | | Container workspace band | `1000 + slot`, allocated from the registry's free list | | Container mode | `1280x720` | -| Units | `hyprland-headless`, `hyprland-headless-stream`, `hhstream-sessions`, `hhstream-pointer` | +| Units | `hyprland-headless`, `hyprland-headless-stream`, `hhstream-sessions`, `hhstream-pointer`, `hhstream-live` (v4) | | Keyboard layout | **option default** `us` (`sessions.keyboard.layout`), reaching the driver as `--xkb-layout`. One group: a comma is refused. Hyprland's `input:kb_layout` is not read. | -| Binaries | `hhstream`, `hhstreamd`, `hhstream-server`, `hhstream-client`, `hhstream-pointer` | +| Binaries | `hhstream`, `hhstreamd`, `hhstream-server`, `hhstream-client`, `hhstream-pointer`, `hhstream-live` | +| Registry schema | **4** (v4; 3 → 4 adds `stream`, migrating to `null`) | | Service account | `hermes`, supplementary group `hhstream` | --- From 8c99518a6220dcb6243459e6b075b8ac0724ce2e Mon Sep 17 00:00:00 2001 From: Hermenix Date: Sun, 11 Oct 2026 22:47:16 +0000 Subject: [PATCH 02/12] feat(live): live.sock, both ends MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The socket CONTRACT-sessions.md §3.5 freezes, implemented on both sides and by nobody else: `hhstream_live.control` answers it, `hhstreamd.live` asks it. `control.Control` is the four ops and nothing more -- attach, detach, list, viewers -- with the decisions the daemon is allowed to make and no others. It owns the two things only this process knows: how to build a capture for an output (the capture command, from this unit's argv) and what a URL looks like (the bound port). Both are injected as callables by `__main__.control_server`, which is also why `attach` replies with a URL rather than letting the broker assemble one from a port it was told about once. `attach` is idempotent on the phrase and returns the existing stream, so a re-`enable` after a broker restart cannot double the `grim` rate on an output. The same phrase aimed at a *different* output is `invalid_argument`, because that is a mistake and not a request. Two details that are the point rather than polish: * `_phrase_of` holds a caller's phrase to the generated shape and raises a shape-only message, with `from None`. `phrase.validate` names the value it rejected, which is correct for a library and wrong here: this value is a password, and the refusal travels back over a socket and into a log. * `ControlServer.serve_forever` reads `self._sock` into a local before using it. `close()` runs on the main thread while this one is in `accept()`, and the attribute it was reading can be gone by the time it is dereferenced. `hhstreamd.live` is the broker's client: one op at a time behind a lock, a 2-second timeout, `LiveUnavailable` for "could not be reached or did not answer intelligibly" and `LiveRejected` carrying the `kind` the broker maps to an exit code -- the pointer driver's client, verb for verb, because the broker already knows how to hold one of those. `Attachment.parse` refuses a malformed reply rather than defaulting its way past one. `fps` is omitted from an `attach` unless a caller named one, which is what makes `sessions.stream.fps` on the daemon's unit authoritative instead of decorative. And `_safe()` renders a phrase as `` for the one log line that mentions a stream at all: the shape, never the words. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_011ZeH65zd6pFTgBSu52W9PE --- src/hhstream_live/__main__.py | 58 ++++++- src/hhstream_live/control.py | 317 ++++++++++++++++++++++++++++++++++ src/hhstreamd/live.py | 301 ++++++++++++++++++++++++++++++++ 3 files changed, 673 insertions(+), 3 deletions(-) create mode 100644 src/hhstream_live/control.py create mode 100644 src/hhstreamd/live.py diff --git a/src/hhstream_live/__main__.py b/src/hhstream_live/__main__.py index 3c0bb80..c9f4c29 100644 --- a/src/hhstream_live/__main__.py +++ b/src/hhstream_live/__main__.py @@ -88,6 +88,13 @@ def build_parser() -> argparse.ArgumentParser: serve.add_argument("--stream", action="append", default=[], type=parse_stream, metavar="PHRASE=OUTPUT", help="enable a stream on this output; repeatable") + serve.add_argument( + "--control-socket", + metavar="PATH", + help="serve the broker's control socket here (CONTRACT-sessions.md " + "§3.5), so `hhstream session stream enable` can attach a stream. " + "Omitted, the app is standalone and only --stream puts anything on it.", + ) serve.add_argument("--capture-command", action="append", default=None, metavar="WORD", help="capture argv word; repeat (a repeated flag is the " @@ -139,6 +146,7 @@ def cmd_serve(args: argparse.Namespace) -> int: capture_base = args.capture_command or DEFAULT_CAPTURE registry = make_registry(args.stream, capture_base, args.fps) server = LiveServer(registry, host=args.bind, port=args.port) + control = control_server(args, registry, server, capture_base) # A service is stopped with SIGTERM, not with Ctrl-C: without a handler the # default action kills the process outright and `server.stop` never runs, so @@ -159,14 +167,25 @@ def cmd_serve(args: argparse.Namespace) -> int: try: server.start() # raises OSError loudly if the port is taken + if control is not None: + # After the listener, because `attach` replies with a URL and the + # URL has no answer until the port is bound -- a requested port of + # 0 least of all. + control.listen() + threading.Thread( + target=control.serve_forever, + name="hhstream-live.control", + daemon=True, + ).start() + print(f"control socket: {control.socket_path}", file=sys.stderr) # Printed after `start`, because a requested port of 0 only has an answer # once it is bound. `describe_bind` names the real bind, so `0.0.0.0` is # not reported as loopback just because that is the address you click. print(server.describe_bind(), file=sys.stderr) print(f"landing page: http://{server.address}:{server.port}/", file=sys.stderr) - if not args.stream: - print("no --stream given; only the landing page will answer", - file=sys.stderr) + if not args.stream and control is None: + print("no --stream and no --control-socket; only the landing page " + "will answer", file=sys.stderr) for phrase in registry.phrases(): print(server.url_for(phrase)) sys.stdout.flush() @@ -177,10 +196,43 @@ def cmd_serve(args: argparse.Namespace) -> int: finally: if previous is not None: signal.signal(signal.SIGTERM, previous) + if control is not None: + # Before the HTTP listener: the broker's next `attach` should fail + # to connect rather than be accepted by a process on its way out. + control.close() server.stop() return 0 +def control_server( + args: argparse.Namespace, + registry: StreamRegistry, + server: LiveServer, + capture_base: list[str], +) -> "ControlServer | None": + """The broker's control socket, or ``None`` for a standalone run. + + The two things only this process knows are injected here and nowhere else: + how to build a capture for an output (the capture command, from this unit's + argv) and how to name a URL (the bound port). That is the whole reason + `attach` replies with a URL instead of letting the broker assemble one. + """ + + if not getattr(args, "control_socket", None): + return None + from .control import Control, ControlServer + + from hhstream.capture import CommandCapture + + control = Control( + registry, + capture_for=lambda output: CommandCapture(capture_argv(capture_base, output)), + url_for=server.url_for, + default_fps=args.fps, + ) + return ControlServer(args.control_socket, control) + + def main(argv: list[str] | None = None) -> int: args = parse_args(argv) logging.basicConfig( diff --git a/src/hhstream_live/control.py b/src/hhstream_live/control.py new file mode 100644 index 0000000..b24476e --- /dev/null +++ b/src/hhstream_live/control.py @@ -0,0 +1,317 @@ +"""``/run/hhstream/live.sock`` — the broker tells this process what to publish. + +The daemon's end of `docs/CONTRACT-sessions.md` §3.5. Four ops (`attach`, +`detach`, `list`, `viewers`) over newline-delimited JSON, in the shape the +pointer driver's socket already uses, because a second convention for the same +job would be a second thing to remember. + +**This process decides nothing about sessions.** It does not know what an idle +clock is, which application a session runs, or whether a stream *should* be on. +It is told a phrase, an output and a rate, and it publishes. `viewers` is a +counter read; what it means is the broker's business +(`docs/CONTRACT-sessions.md` §4). + +**It does own the port, the bind address and the capture command**, from its own +unit's argv — which is why `attach` replies with the URL rather than letting the +broker assemble one. A URL built from a port somebody was told about once is a +URL that lies the first time the listener moves. + +**`attach` is idempotent on the phrase.** The same phrase aimed at the same +output returns the running stream instead of starting a second capture: the +broker re-attaches after a restart, and doubling the ``grim`` rate on one output +because of a reconnect would be a self-inflicted load problem. The same phrase +aimed somewhere *else* is refused — a phrase is a credential for one view, and +silently re-pointing it would show a person a desktop they were not given. + +**The phrase is never logged**, here or in the broker. Every line below names +the output and the session. +""" + +from __future__ import annotations + +import json +import logging +import os +import socket +import threading +from pathlib import Path +from typing import Any, Callable + +from . import phrase as phrase_mod +from .publisher import DEFAULT_FPS, FPS_CAP, StreamRegistry, StreamSpec + +log = logging.getLogger("hhstream.live.control") + +#: Matches ``hhstreamd.protocol.MAX_LINE_BYTES`` and the pointer driver's. +MAX_LINE_BYTES = 1 << 20 +LISTEN_BACKLOG = 8 + +#: A rejection the broker maps to exit 2 rather than exit 10 -- the caller's +#: mistake, not a failure of this process. Same vocabulary as the driver's. +INVALID = "invalid_argument" + + +class Control: + """The op handler. Holds the registry and the two things only this process knows. + + ``capture_for`` turns an output name into a capture object, and ``url_for`` + turns a phrase into this listener's URL. Both are injected rather than + built here so the tests can drive the whole protocol without a compositor + and without binding a port. + """ + + def __init__( + self, + registry: StreamRegistry, + *, + capture_for: Callable[[str], Any], + url_for: Callable[[str], str], + default_fps: int = DEFAULT_FPS, + ) -> None: + self.registry = registry + self.capture_for = capture_for + self.url_for = url_for + #: The rate an `attach` with no `fps` gets. This process's, from its + #: own argv, because it is the one that has to keep up with it -- and + #: so an operator's `sessions.stream.fps` reaches every stream rather + #: than only the ones somebody named a rate for. + self.default_fps = int(default_fps) + self._lock = threading.Lock() + + # -- the ops ---------------------------------------------------------- + + def handle(self, request: Any) -> dict[str, Any]: + if not isinstance(request, dict): + return _no(f"request is not an object: {type(request).__name__}", INVALID) + op = request.get("op") + handler = { + "attach": self._attach, + "detach": self._detach, + "list": self._list, + "viewers": self._viewers, + }.get(op if isinstance(op, str) else "") + if handler is None: + return _no(f"unknown op: {op!r}", INVALID) + # One op at a time: `attach` reads the registry and then mutates it, and + # two brokers (or a broker and a restart sweep) must not interleave + # there and start two captures for one phrase. + with self._lock: + try: + return handler(request) + except Rejected as refused: + return _no(str(refused), refused.kind) + except (ValueError, TypeError) as exc: + # The registry's own validation (`fps must be a positive int`, + # `phrase ... is already enabled`) surfaces here. It is the + # caller's mistake, so it is exit 2 and not a daemon failure. + return _no(str(exc), INVALID) + + def _attach(self, request: dict[str, Any]) -> dict[str, Any]: + phrase = _phrase_of(request) + output = _text(request, "output") + fps = _fps(request, self.default_fps) + session = request.get("session") + session = None if session is None else str(session) + label = str(request.get("label") or "") + + existing = self.registry.get(phrase) + if existing is not None: + if existing.spec.output != output: + raise Rejected( + f"that phrase is already publishing output " + f"{existing.spec.output!r}, not {output!r}; disable it first", + INVALID, + ) + # Idempotent: the running stream, with whatever rate it was started + # at. A new `fps` is deliberately NOT applied here -- changing the + # rate under a connected viewer is a `disable`/`enable`, which is + # one record write and one obvious act, rather than a silent + # re-rate that no log line would explain. + log.info("re-attached output %s for %s", output, session or "a standalone stream") + return {"ok": True, "stream": self._describe(existing)} + + spec = StreamSpec( + phrase=phrase, output=output, fps=fps, session=session, label=label or output + ) + stream = self.registry.add(spec, self.capture_for(output)) + log.info( + "attached output %s for %s at %s fps", + output, + session or "a standalone stream", + fps, + ) + return {"ok": True, "stream": self._describe(stream)} + + def _detach(self, request: dict[str, Any]) -> dict[str, Any]: + phrase = _phrase_of(request) + stream = self.registry.remove(phrase) + if stream is None: + # Not an error. `disable` on a session whose daemon restarted since + # `enable` is the ordinary case, and the broker has to clear its + # record either way. + return {"ok": True, "detached": False} + log.info( + "detached output %s for %s", + stream.spec.output, + stream.spec.session or "a standalone stream", + ) + return {"ok": True, "detached": True} + + def _list(self, request: dict[str, Any]) -> dict[str, Any]: + return {"ok": True, "streams": [self._describe(s) for s in self.registry.all()]} + + def _viewers(self, request: dict[str, Any]) -> dict[str, Any]: + return { + "ok": True, + # `Stream.viewers` is a property, not a method. + "viewers": {s.spec.phrase: s.viewers for s in self.registry.all()}, + } + + def _describe(self, stream: Any) -> dict[str, Any]: + public = dict(stream.spec.public()) + public["url"] = self.url_for(stream.spec.phrase) + public["viewers"] = stream.viewers + return public + + +class Rejected(Exception): + """A refusal with a ``kind`` the broker maps to an exit code.""" + + def __init__(self, message: str, kind: str | None = None) -> None: + super().__init__(message) + self.kind = kind + + +def _no(error: str, kind: str | None = None) -> dict[str, Any]: + reply: dict[str, Any] = {"ok": False, "error": error} + if kind: + reply["kind"] = kind + return reply + + +def _text(request: dict[str, Any], field: str) -> str: + value = request.get(field) + if not isinstance(value, str) or not value.strip(): + raise Rejected(f"{field} must be a non-empty string", INVALID) + return value.strip() + + +def _phrase_of(request: dict[str, Any]) -> str: + """The request's phrase, held to the generated shape. + + ``phrase.validate`` and not a looser check, for the reason it exists: the + phrase is this stream's only credential, so anything that reaches the + registry has to be a password rather than ``a-b-c``. The refusal never + echoes the phrase back. + """ + + value = request.get("phrase") + if not isinstance(value, str): + raise Rejected("phrase must be a string", INVALID) + try: + return phrase_mod.validate(value) + except ValueError: + # Not `{exc}`: the validator names the value it refused, and this + # message travels back over the socket into the broker's log. The + # shape, never the phrase. + raise Rejected( + "the phrase given is not of the expected shape: four words of at " + "least three letters plus a five-character tail", + INVALID, + ) from None + + +def _fps(request: dict[str, Any], default: int) -> int: + value = request.get("fps") + if value is None: + return default + if isinstance(value, bool) or not isinstance(value, int): + raise Rejected(f"fps must be an int, got {type(value).__name__}", INVALID) + if value <= 0 or value > FPS_CAP: + raise Rejected(f"fps must be between 1 and {FPS_CAP}, got {value}", INVALID) + return value + + +class ControlServer: + """The accept loop for ``live.sock``. One connection at a time. + + The same shape as ``hhstream_pointer.server.PointerServer``: the broker is + the only peer, the ops are one small line each, and an async server would + buy nothing but a way to interleave two attaches. + """ + + def __init__( + self, + socket_path: Path | str, + control: Control, + *, + socket_mode: int = 0o660, + ) -> None: + self.socket_path = Path(socket_path) + self.control = control + self.socket_mode = socket_mode + self._sock: socket.socket | None = None + self.ready = threading.Event() + + def listen(self) -> None: + """Bind and listen. A stale socket file is replaced, not refused: + a crashed daemon leaves one behind and the unit must still start.""" + + self.socket_path.parent.mkdir(parents=True, exist_ok=True) + if self.socket_path.is_socket(): + self.socket_path.unlink() + sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + sock.bind(str(self.socket_path)) + os.chmod(self.socket_path, self.socket_mode) + sock.listen(LISTEN_BACKLOG) + self._sock = sock + self.ready.set() + log.info("control socket listening on %s", self.socket_path) + + def serve_forever(self) -> None: + assert self._sock is not None, "listen() first" + while True: + # Read into a local first. `close()` runs on another thread and + # sets `_sock` to None, so reading the attribute twice can find a + # socket and then find nothing -- measured, as an AttributeError + # out of a daemon thread at shutdown rather than the clean return + # this loop is supposed to make. + sock = self._sock + if sock is None: + return + try: + conn, _ = sock.accept() + except OSError: + # The other way `close()` shows up: the listener was closed + # while this thread was blocked in `accept`. + return + with conn: + self._serve_connection(conn) + + def close(self) -> None: + sock, self._sock = self._sock, None + if sock is not None: + sock.close() + self.socket_path.unlink(missing_ok=True) + self.ready.clear() + + def _serve_connection(self, conn: socket.socket) -> None: + with conn.makefile("rwb") as stream: + while True: + line = stream.readline(MAX_LINE_BYTES + 1) + if not line: + return + if len(line) > MAX_LINE_BYTES: + _write(stream, _no("request line too long", INVALID)) + return + try: + request = json.loads(line) + except json.JSONDecodeError as exc: + _write(stream, _no(f"malformed JSON: {exc}", INVALID)) + continue + _write(stream, self.control.handle(request)) + + +def _write(stream, response: dict[str, Any]) -> None: + stream.write(json.dumps(response, separators=(",", ":")).encode() + b"\n") + stream.flush() diff --git a/src/hhstreamd/live.py b/src/hhstreamd/live.py new file mode 100644 index 0000000..c9364f1 --- /dev/null +++ b/src/hhstreamd/live.py @@ -0,0 +1,301 @@ +"""The broker's end of the stream daemon's socket (CONTRACT-sessions.md §3.5). + +`hhstream-live` is a third executor beside the pointer driver, and this module +is the client that talks to it. The split is the driver's, for the driver's +reasons: ``grim`` blocks tens to hundreds of milliseconds, so the capture must +not run where ``touch`` and ``kill`` are answered, and a LAN-facing HTTP parser +must not run in the frame server whose one-socket contract cannot grow one. + +**The broker holds the decision; the daemon holds the capture.** Which session +streams, and under which phrase, is the broker's — it is in the session record +and it survives a restart. The port, the bind address, the capture command and +the URL prefix are the daemon's, from its own unit's argv. So nothing here +passes a port and nothing here builds a URL: :attr:`Attachment.url` comes back +from the daemon, because a URL assembled from a port the broker was told about +once is a URL that lies the first time an operator moves the listener. + +**The phrase is never logged.** It is the stream's only credential, in +cleartext in a URL on whatever the bind allows. Every log line here names the +output and the session; `_safe` is what keeps a phrase out of the ones that +would otherwise carry it. + +Transport, lifetime and failure semantics are :mod:`hhstreamd.driver`'s, down to +the held connection and the EOF check before each write. One difference in +what a failure *means*: a dropped `release_all` leaves a key down in the +compositor, which is why the driver never retries; a dropped `attach` leaves +nothing behind but an unattached phrase, so `attach` is **idempotent on the +daemon's side** and re-sending it is safe. That is the daemon's guarantee, not +this module's, and it is what makes re-enabling after a restart cheap. +""" + +from __future__ import annotations + +import dataclasses +import json +import logging +import select +import socket +import threading +from pathlib import Path +from typing import Any + +log = logging.getLogger("hhstreamd.live") + +#: The daemon answers from memory for every op here -- it does not capture a +#: frame to answer `attach`, and `viewers` is a counter read. This only bounds +#: a daemon that has stopped answering at all. +DEFAULT_TIMEOUT_S = 2.0 + +#: Matches the daemon's own ceiling. +MAX_LINE_BYTES = 1 << 20 + + +class LiveUnavailable(Exception): + """The daemon could not be reached, or did not answer intelligibly. + + Distinct from :class:`LiveRejected`: this one means nothing is known about + whether the stream is attached, so the caller reports a *degraded* stream + rather than an absent one. The session is unaffected either way -- a + stream that cannot be reached is a stream, not a dead application -- which + is why the manager maps this to exit 10 and never to a session failure. + """ + + +class LiveRejected(Exception): + """The daemon refused. ``kind`` decides the exit code, as the driver's does.""" + + def __init__(self, message: str, *, kind: str | None = None) -> None: + super().__init__(message) + self.kind = kind + + +@dataclasses.dataclass(frozen=True) +class Attachment: + """One stream, as the daemon reports it.""" + + phrase: str + output: str + fps: int + session: str | None + #: The daemon's own prefix, e.g. ``http://127.0.0.1:9988//``. + url: str + viewers: int + + @classmethod + def parse(cls, data: Any) -> "Attachment": + if not isinstance(data, dict): + raise ValueError(f"stream is not an object: {type(data).__name__}") + try: + return cls( + phrase=str(data["phrase"]), + output=str(data["output"]), + fps=int(data["fps"]), + session=None if data.get("session") is None else str(data["session"]), + url=str(data.get("url") or ""), + viewers=int(data.get("viewers") or 0), + ) + except (KeyError, TypeError, ValueError) as exc: + raise ValueError(f"malformed stream in the daemon's reply: {exc}") from exc + + +def _safe(phrase: str) -> str: + """A phrase, for a log line. The shape, never the words. + + ``amber-otter-lantern-harbor-7k2qp`` becomes ````. A log + is read by whoever can read logs, and this one is a password. + """ + + return f"" + + +class LiveStreams: + """A client for one stream daemon's socket. Thread-safe; one op at a time.""" + + def __init__(self, socket_path: Path | str, *, timeout_s: float = DEFAULT_TIMEOUT_S) -> None: + self.socket_path = Path(socket_path) + self.timeout_s = float(timeout_s) + self._lock = threading.Lock() + self._sock: socket.socket | None = None + self._stream = None + + # -- the ops ---------------------------------------------------------- + + def attach( + self, + *, + phrase: str, + output: str, + fps: int | None = None, + session: str | None = None, + label: str = "", + ) -> Attachment: + """Publish ``output`` under ``phrase``. Idempotent on the phrase. + + The daemon returns the existing stream if the same phrase is already + aimed at the same output, rather than starting a second capture: a + re-enable after a broker restart must not double the ``grim`` rate on + one output. The same phrase aimed somewhere else is a rejection. + + ``fps`` is **omitted** when nobody asked for one, and that is the point: + the default rate is the daemon's, from its own unit's argv, for the same + reason the port is. A broker that sent its own default on every attach + would make the operator's ``sessions.stream.fps`` a setting that changed + nothing -- which is exactly the failure the keyboard layout's + ``--xkb-layout`` comment in this tree is about. + """ + + request: dict[str, Any] = { + "op": "attach", + "phrase": phrase, + "output": output, + "session": session, + "label": label, + } + if fps is not None: + request["fps"] = int(fps) + reply = self.request(request) + attachment = Attachment.parse(reply.get("stream")) + log.info( + "attached %s to output %s at %s fps (%s)", + session or "a standalone stream", + output, + attachment.fps, + _safe(phrase), + ) + return attachment + + def detach(self, phrase: str) -> bool: + """Stop publishing ``phrase``. ``False`` if it was not attached. + + Not an error: ``disable`` on a session whose daemon restarted since + ``enable`` is the ordinary case, and the caller's job is to clear the + record either way. + """ + + reply = self.request({"op": "detach", "phrase": phrase}) + return bool(reply.get("detached")) + + def list(self) -> list[Attachment]: + """Every stream the daemon is publishing, in its own order.""" + + reply = self.request({"op": "list"}) + streams = reply.get("streams") + if not isinstance(streams, list): + raise LiveUnavailable( + f"stream daemon {self.socket_path}: `list` returned " + f"{type(streams).__name__}, not a list" + ) + try: + return [Attachment.parse(entry) for entry in streams] + except ValueError as exc: + raise LiveUnavailable(f"stream daemon {self.socket_path}: {exc}") from exc + + def viewers(self) -> dict[str, int]: + """Phrase to connected viewers, for the activity sweep (§4). + + A reading, and nothing more: what it means for the idle clock is the + manager's, and this module deliberately has no opinion about it. + """ + + reply = self.request({"op": "viewers"}) + counts = reply.get("viewers") + if not isinstance(counts, dict): + raise LiveUnavailable( + f"stream daemon {self.socket_path}: `viewers` returned " + f"{type(counts).__name__}, not an object" + ) + read: dict[str, int] = {} + for phrase, count in counts.items(): + try: + read[str(phrase)] = int(count) + except (TypeError, ValueError) as exc: + raise LiveUnavailable( + f"stream daemon {self.socket_path}: viewer count for " + f"{_safe(str(phrase))} is not a number: {exc}" + ) from exc + return read + + # -- transport -------------------------------------------------------- + + def request(self, message: dict[str, Any]) -> dict[str, Any]: + """One request, one reply. Raises rather than returning a guess.""" + + with self._lock: + try: + return self._exchange(message) + except (OSError, ValueError) as exc: + # The connection is now of unknown state -- a half-written + # request, a partial reply -- so it is dropped rather than + # reused for the next op. + self._disconnect() + raise LiveUnavailable(f"stream daemon {self.socket_path}: {exc}") from exc + + def close(self) -> None: + with self._lock: + self._disconnect() + + def _exchange(self, message: dict[str, Any]) -> dict[str, Any]: + stream = self._connect() + line = json.dumps(message, separators=(",", ":")).encode("utf-8") + b"\n" + stream.write(line) + stream.flush() + reply_line = stream.readline(MAX_LINE_BYTES + 1) + if not reply_line: + raise OSError("the daemon closed the connection without answering") + if len(reply_line) > MAX_LINE_BYTES: + raise ValueError("reply exceeds the frame limit") + reply = json.loads(reply_line) + if not isinstance(reply, dict): + raise ValueError(f"reply is not an object: {type(reply).__name__}") + if not reply.get("ok"): + kind = reply.get("kind") + raise LiveRejected( + str(reply.get("error") or "the stream daemon refused the request"), + kind=kind if isinstance(kind, str) else None, + ) + return reply + + def _connect(self): + if self._stream is not None and not self._stale(): + return self._stream + self._disconnect() + sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + try: + sock.settimeout(self.timeout_s) + sock.connect(str(self.socket_path)) + except OSError: + sock.close() + raise + self._sock = sock + self._stream = sock.makefile("rwb") + log.debug("connected to the stream daemon at %s", self.socket_path) + return self._stream + + def _stale(self) -> bool: + """True if the held connection has died since the last op. + + The daemon never speaks unsolicited, so a readable socket can only be + at EOF. One non-blocking ``select``, and it is what makes reconnecting + safe: a request is written only to a connection that was alive when it + was written. + """ + + sock = self._sock + if sock is None: + return True + try: + readable, _, _ = select.select([sock], [], [], 0) + except (OSError, ValueError): + return True + return bool(readable) + + def _disconnect(self) -> None: + stream, self._stream = self._stream, None + sock, self._sock = self._sock, None + for closeable in (stream, sock): + if closeable is None: + continue + try: + closeable.close() + except OSError: + pass From 7d8ca6191bb908ed2bdfb73357b739bda6792766 Mon Sep 17 00:00:00 2001 From: Hermenix Date: Sun, 11 Oct 2026 22:47:43 +0000 Subject: [PATCH 03/12] feat(sessions)!: the four stream verbs, schema 4, and the viewer sweep The broker half of sessions-v4: `stream.enable`, `stream.disable`, `stream.list`, `stream.status`, `start`'s three new arguments, and the one rule that is genuinely new rather than additive -- a connected viewer is activity. The decisions worth reading the diff for: **A stream never falls back to the main output.** `_stream_output` refuses when a session has no container, and the refusal is stronger than the pointer's for a reason the hint states: the main output carries *every other session*, so a fallback would put somebody else's desktop on a phone behind a phrase that was issued for this one. **The phrase lives in the record, so re-enabling returns the same URL.** A different `--phrase` on an already-enabled session is `stream_busy` (exit 8), not a silent rotation: rotating under a caller who is holding the old URL is the one outcome neither answer can be mistaken for. `disable` then `enable` is how it rotates, which is explicit. **`_checked_phrase` raises the shape and never the value**, with `from None`, for `control._phrase_of`'s reason: `phrase.validate` names what it rejected, and this value is a password on its way into an error reply and a log. **`fps` is only sent when a caller named one** (`_checked_fps` returns `None` otherwise), which is what makes the daemon's own default authoritative instead of decorative. The broker holds the daemon to the frozen cap before sending, so a nonsense rate is exit 2 from the verb rather than a surprise from the socket. **`attached` is its own field and is not folded into `enabled`.** A record can say the operator asked for a stream while the daemon is down, and those are two different facts; `url` and `viewers` are *absent* rather than stale when it is false, because a URL that no longer serves anything is worse than no URL. `stream.list` also reports `available`, so "no streams" is distinguishable from "no daemon" without parsing an error. **The viewer sweep** (`_sweep_streams`, every `STREAM_SWEEP_S = 60`s) asks the daemon for viewer counts and stamps `touch` on every session somebody is watching. It is the only thing in this daemon that treats an observation as an intention, and it exists for one failure: an operator watching a long job on his phone and losing the session at three hours *because* watching was all he was doing. It cannot hold a session open by itself -- an enabled stream with no viewer runs no `grim` and stamps nothing -- which is why it is safe to run unconditionally. **Teardown detaches before it destroys the container.** The other order makes every capture fail against a removed output, which is a stream that gives up ten frames later and a log that blames `grim`. `start` takes `stream`, `stream_phrase` and `stream_fps` and reports a stream it could not open in `stream_error` beside a session that *did* start, because the exit code is 0 and a caller who asked for a stream must not read that as one. Registry schema 3 -> 4: the record gains `stream`, older records migrate to `null` rather than being refused, and the viewer count is deliberately not stored -- a viewer count on disk is a number that is wrong the moment it is written. No new exit numbers: `stream_busy` -> 8, `stream_budget` -> 6, `stream_unavailable` -> 10. The last one is 10 for `driver_refused`'s reason -- the session keeps running, and a caller that read an unreachable stream as a dead session would kill an application over a web page. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_011ZeH65zd6pFTgBSu52W9PE --- src/hhstreamd/manager.py | 472 +++++++++++++++++++++++++++++++++++++- src/hhstreamd/protocol.py | 21 ++ src/hhstreamd/registry.py | 63 ++++- src/hhstreamd/server.py | 33 +++ 4 files changed, 579 insertions(+), 10 deletions(-) diff --git a/src/hhstreamd/manager.py b/src/hhstreamd/manager.py index 1cf3def..d37a6f8 100644 --- a/src/hhstreamd/manager.py +++ b/src/hhstreamd/manager.py @@ -57,6 +57,7 @@ from .container_host import ( ContainerHost, ) from .driver import DriverRejected, DriverUnavailable, PointerDriver +from .live import Attachment, LiveRejected, LiveStreams, LiveUnavailable from .ipc import CompositorUnavailable from .launcher import Launcher from .pointer import Bookmark, LeaseBusy, LeaseInvalid, PointerLease @@ -70,6 +71,7 @@ from .registry import ( State, container_record, pointer_record, + stream_record, ) from .textio import tail_lines @@ -110,6 +112,16 @@ READBACK_INTERVAL_S = 0.05 #: as long as nobody asks. POINTER_SWEEP_S = 1.0 +#: How often the broker asks the stream daemon for viewer counts, and stamps +#: `touch` on every session that has one (CONTRACT-sessions.md §4). +#: +#: A minute, not a second: this is the one exception to "`touch` is the only +#: mark of activity", and the thing it exists to prevent is a three-hour idle +#: timeout firing under somebody's eyes. Against three hours a minute of +#: granularity is free, and a tighter sweep would be one `grim`-adjacent round +#: trip per second for a fact that changes when a browser tab opens. +STREAM_SWEEP_S = 60.0 + #: The workspace band (CONTRACT-sessions.md §5) and how many slots of it this #: broker will hand out. The real ceiling is a CPU question the plan defers to #: a measurement (PLAN §5.5); the band is the part that can be enforced here, @@ -199,8 +211,17 @@ class ManagerConfig: pointer_ttl_s: float = 30.0 pointer_max_s: float = 300.0 pointer_sweep_s: float = POINTER_SWEEP_S + #: The broker↔stream socket (CONTRACT-sessions.md §3.5). Derived from the + #: runtime dir for the reason `pointer_socket` is: the path is frozen, so a + #: flag for it would be a flag for a value that may not vary. + live_socket: Path | None = None + #: How often the viewer sweep runs. The one thing that moves an idle clock + #: without a caller asking (CONTRACT-sessions.md §4). + stream_sweep_s: float = STREAM_SWEEP_S def __post_init__(self) -> None: + if self.live_socket is None: + self.live_socket = Path(self.runtime_dir) / "live.sock" if self.pointer_socket is None: # `/pointer.sock` is the frozen path # (CONTRACT-sessions.md §5: /run/hhstream/pointer.sock), and @@ -221,6 +242,7 @@ class SessionManager: audit: AuditLog | None = None, host: ContainerHost | None = None, driver: PointerDriver | None = None, + live: LiveStreams | None = None, ) -> None: self.config = config self.clock = clock or Clock() @@ -251,6 +273,10 @@ class SessionManager: ttl_s=config.pointer_ttl_s, max_s=config.pointer_max_s, clock=self.clock.boottime ) self.driver = driver + #: The stream daemon's client, or ``None`` with no socket configured -- + #: in which case every `stream.*` verb answers `stream_unavailable` + #: (exit 10) rather than pretending there is a stream. + self.live = live # Per-session asyncio plumbing, never persisted. self._exited: dict[str, asyncio.Event] = {} self._watchers: dict[str, asyncio.Task] = {} @@ -259,6 +285,7 @@ class SessionManager: #: Held so the loop does not collect a task mid-flight. self._background: set[asyncio.Task] = set() self._pointer_sweeper: asyncio.Task | None = None + self._stream_sweeper: asyncio.Task | None = None #: Set while a lease is held with a caller-supplied TTL, so the default #: comes back when the lease ends (the frozen ``acquire`` takes no TTL). self._pointer_ttl_override = False @@ -339,6 +366,13 @@ class SessionManager: self._pointer_sweeper = asyncio.create_task( self._sweep_pointer(), name="hhstreamd.pointer-sweep" ) + # Unconditional, even with no stream enabled anywhere: the sweep's own + # first act is to ask the daemon, and a broker that only started it on + # the first `enable` would be a broker whose adopted records were not + # being watched after a restart. + self._stream_sweeper = asyncio.create_task( + self._sweep_streams(), name="hhstreamd.stream-sweep" + ) return summary # ------------------------------------------------------------------ @@ -632,11 +666,14 @@ class SessionManager: """ await self.reaper.stop() - if self._pointer_sweeper is not None: - self._pointer_sweeper.cancel() + for name in ("_pointer_sweeper", "_stream_sweeper"): + sweeper = getattr(self, name) + if sweeper is None: + continue + sweeper.cancel() with contextlib.suppress(asyncio.CancelledError, Exception): - await self._pointer_sweeper - self._pointer_sweeper = None + await sweeper + setattr(self, name, None) for task in list(self._background): task.cancel() for task in list(self._background): @@ -650,6 +687,12 @@ class SessionManager: with contextlib.suppress(Exception): await self._release_all_buttons() self.driver.close() + if self.live is not None: + # Nothing to lift on this one: a stream left attached across a + # broker restart is re-attached by `enable`, and the daemon keeps + # publishing meanwhile -- which is the behaviour an operator + # watching a long job wants. + self.live.close() for task in list(self._watchers.values()): task.cancel() for task in list(self._watchers.values()): @@ -878,6 +921,9 @@ class SessionManager: wait_ready_s: float | None = None, home: str | None = None, ephemeral_home: bool = False, + stream: bool = False, + stream_phrase: str | None = None, + stream_fps: int | None = None, ) -> dict[str, Any]: policy = self.config.policy policy.check_capacity(len(self.registry.live())) @@ -971,8 +1017,38 @@ class SessionManager: ready = await self._settle(session, wait_ready) if session.live: await self._read_back(session, app_id) + stream_error: str | None = None + if stream and session.live: + # After the read-back, because that is what settles the container's + # output, and a stream has to have one to publish + # (`_stream_output`). Before the record below, so the phrase comes + # back with the session the caller asked for -- which is the whole + # reason `start` takes these arguments instead of making every + # caller sequence "start, then enable" and handle the half-done + # case in between. + # + # A stream that cannot be opened does NOT fail the start. The + # application is on the screen and killing it over a web page would + # be the wrong trade; the refusal is reported on the session's + # `stream_error` and the caller can retry `stream.enable`. + try: + await self.stream_enable( + session.id, phrase=stream_phrase, fps=stream_fps + ) + except ManagerError as exc: + log.warning("could not enable %s's stream: %s", session.id, exc.reason) + session.last_error = session.last_error or exc.reason + # A local, not an attribute: two starts can be in flight at + # once and an attribute would hand one of them the other's + # failure. + stream_error = exc.reason self.reaper.wake() record = session.public(self.now()) + if stream: + # Present whenever a stream was asked for, `None` when it worked: + # a key that appears only on failure is a key every caller has to + # remember to look for. + record["stream_error"] = stream_error if session.state == State.EXITED: # The app exited during the settle: surface it rather than handing # back a healthy-looking id (DESIGN.md §7 #3). @@ -1235,7 +1311,17 @@ class SessionManager: # has four values (lease|timeout|crash|kill), and a reap is the # broker killing the session. await self._drop_lease_for(session, reason="kill") + # The stream goes BEFORE the output does, and the order is + # load-bearing (CONTRACT-sessions.md §4): a capture aimed at a + # removed output fails every time, so the daemon's own failure + # budget would close the viewers with "unknown output" instead of + # the session simply ending. Never raises -- a daemon that cannot + # be reached must not stop a session being killed. + stream_detached = await self._detach_stream(session) + if (session.stream or {}).get("phrase"): + session.stream = None container_result = self._destroy_container(session) + container_result["stream_detached"] = stream_detached if refused is None: state_now = state @@ -1929,6 +2015,368 @@ class SessionManager: # -- lease internals -------------------------------------------------- + # ------------------------------------------------------------------ + # the live stream (CONTRACT-sessions.md §4, v4) + # ------------------------------------------------------------------ + + async def stream_enable( + self, session_id: str, *, phrase: str | None = None, fps: int | None = None + ) -> dict[str, Any]: + """Publish this session's container output under a spoken phrase. + + **Does not** count as activity, like every `pointer.*` verb: switching + a stream on is not saying the session is wanted. What *is* activity is + somebody watching it, and that is :meth:`_sweep_streams`. + + The phrase is minted here and stored in the record, so it survives a + broker restart and a re-enable: an operator who read four words out to + somebody must not have them change because the daemon bounced. A + caller's own `phrase` is held to the generated shape -- it is this + stream's only credential, so anything a caller may set is a password, + and a looser check would accept `a-b-c`. + + Re-enabling an enabled session returns the same phrase. A different + `phrase` is `stream_busy` (exit 8) rather than a silent rotation, + because rotating under a caller who is holding the old URL is the one + outcome neither answer can be mistaken for. + """ + + session = self._require_live(session_id) + output = self._stream_output(session) + rate = self._checked_fps(fps) + stored = session.stream or {} + wanted = self._checked_phrase(phrase) + + if stored.get("phrase") and wanted and wanted != stored["phrase"]: + raise ManagerError( + "stream_busy", + f"session {session.id} already has a stream under a different phrase", + hint="`stream disable` then `stream enable` rotates it; a stream " + "cannot have two phrases, because the old one is a URL somebody " + "may be holding", + id=session.id, + ) + settled = stored.get("phrase") or wanted or self._mint_phrase() + # The stored rate wins on a re-enable with no `fps`: a re-attach after a + # restart must not quietly re-rate a stream somebody is watching. With + # neither, the rate is left UNSET and the daemon applies its own + # configured default -- the broker does not have one, on purpose. + rate = rate or stored.get("fps") + + attachment = await self._attach_stream( + session, phrase=settled, output=output, fps=None if rate is None else int(rate) + ) + record = stream_record( + phrase=settled, + output=output, + fps=attachment.fps, + enabled_at=stored.get("enabled_at") or iso(self.clock.wall()), + ) + session.stream = record + # `force=True`: the phrase is the one field here that cannot be + # re-derived, so it is on disk before the caller is told what it is. + self.registry.persist(session, force=True) + # The phrase is NOT in the audit entry (CONTRACT-sessions.md §3.5): the + # log is read by whoever can read logs, and this one is a password. + self.audit.emit( + "stream_enable", id=session.id, output=output, fps=attachment.fps + ) + return {"id": session.id, "stream": self._stream_public(record, attachment)} + + async def stream_disable(self, session_id: str) -> dict[str, Any]: + """Stop publishing, and clear the record. + + Detaching a stream the daemon has already forgotten is **not** an + error: `disable` after a daemon restart is the ordinary case, and the + record has to be cleared either way. `detached` says which happened. + """ + + session = self._require_live(session_id) + stored = session.stream or {} + if not stored.get("phrase"): + raise ManagerError( + "stream_unavailable", + f"session {session.id} has no stream to disable", + hint="`stream enable` first", + id=session.id, + ) + detached = await self._detach_stream(session) + session.stream = None + self.registry.persist(session, force=True) + self.audit.emit( + "stream_disable", id=session.id, output=stored.get("output"), detached=detached + ) + return {"id": session.id, "stream": None, "detached": detached} + + def stream_list(self) -> dict[str, Any]: + """Every session with a stream, with the daemon's view merged in. + + Does not count as activity. ``available`` is whether the daemon could + be asked at all, so a caller can tell "no streams" from "no daemon" + without parsing an error. + """ + + attachments, available = self._attachments() + streams = [] + for session in self.registry.live(): + stored = session.stream or {} + if not stored.get("phrase"): + continue + streams.append( + { + "id": session.id, + "app": session.app, + **self._stream_public(stored, attachments.get(stored["phrase"])), + } + ) + return {"streams": streams, "available": available} + + def stream_status(self, session_id: str | None = None) -> dict[str, Any]: + """One session's stream, or every one of them. Not activity.""" + + if session_id is None: + return self.stream_list() + session = self._require(session_id) + stored = session.stream or {} + if not stored.get("phrase"): + return {"id": session.id, "stream": None} + attachments, _ = self._attachments() + return { + "id": session.id, + "stream": self._stream_public(stored, attachments.get(stored["phrase"])), + } + + # -- the pieces the four verbs share --------------------------------- + + def _stream_output(self, session: Session) -> str: + """The output a stream publishes: the session's own container. + + Refused rather than defaulted, and the reason is stronger than the + pointer's: the main output carries *every other session*, so falling + back to it would put somebody else's desktop on a phone behind a phrase + that was issued for this one. + """ + + output = (session.container or {}).get("output") + if output: + return str(output) + raise ManagerError( + "stream_unavailable", + f"session {session.id} has no container output to publish", + hint="a stream never falls back to the main output -- that one " + "carries every other session. `container.reconcile` once the " + "compositor is reachable, or start the session with containers on", + id=session.id, + container=session.container, + ) + + def _mint_phrase(self) -> str: + """A fresh phrase. + + From `hhstream_live.phrase`, which owns the wordlist, the shape and the + validator -- one definition, so a phrase the broker mints is one the + daemon will accept and a `--phrase` an operator types is held to the + same bar. Imported here rather than at module scope because this is the + only path that needs it and `hhstreamd` must stay importable without + the web app's wordlist being readable. + """ + + from hhstream_live import phrase as phrase_mod + + return phrase_mod.generate() + + def _checked_phrase(self, phrase: Any) -> str | None: + if phrase is None: + return None + from hhstream_live import phrase as phrase_mod + + if not isinstance(phrase, str): + raise ManagerError( + "invalid_argument", "phrase must be a string", id=None + ) + try: + return phrase_mod.validate(phrase) + except ValueError: + # The validator's own message names the value it refused, which is + # right for an operator reading their own terminal and wrong here: + # this reason reaches the broker's log and the audit trail, and the + # value is a credential (CONTRACT-sessions.md §3.5). So the shape + # is described and the phrase is NOT interpolated -- deliberately, + # and `from None` so the chained message cannot carry it either. + raise ManagerError( + "invalid_argument", + "the phrase given is not of the expected shape", + hint="four words of at least three letters and a five-character " + "tail, e.g. amber-otter-lantern-harbor-7k2qp; omit it to have " + "one minted", + ) from None + + def _checked_fps(self, fps: Any) -> int | None: + if fps is None: + return None + if isinstance(fps, bool) or not isinstance(fps, int): + raise ManagerError( + "invalid_argument", f"fps must be an int, got {type(fps).__name__}" + ) + if fps <= 0: + raise ManagerError("invalid_argument", f"fps must be positive, got {fps}") + # The ceiling is deliberately NOT checked here. It belongs to the + # daemon (`hhstream_live.publisher.FPS_CAP`), which is the process that + # has to keep up with it, and a second copy of the number here would be + # a second thing to raise when the box gets faster. + return fps + + async def _attach_stream( + self, session: Session, *, phrase: str, output: str, fps: int | None + ) -> Attachment: + live = self._live_streams() + if live is None: + raise ManagerError( + "stream_unavailable", + "no stream daemon is configured for this broker", + hint="the `hhstream-live` unit is what captures a stream; " + "`sessions.stream` in the module turns it on", + id=session.id, + ) + try: + return await asyncio.to_thread( + live.attach, + phrase=phrase, + output=output, + fps=fps, + session=session.id, + label=session.app, + ) + except LiveRejected as exc: + # The daemon's own vocabulary, mapped once: its `invalid_argument` + # is the caller's mistake (exit 2) and anything else is a stream + # that could not be opened (exit 10). The same split the driver's + # `kind` already gets. + error = "invalid_argument" if exc.kind == "invalid_argument" else "stream_unavailable" + raise ManagerError(error, str(exc), id=session.id) from exc + except LiveUnavailable as exc: + raise ManagerError( + "stream_unavailable", + f"the stream daemon could not be reached: {exc}", + hint="is hhstream-live running? the session itself is unaffected", + id=session.id, + ) from exc + + async def _detach_stream(self, session: Session) -> bool: + """Stop the capture for this session, if there is one. + + Never raises. Called from `disable`, which has a record to clear + regardless, and from teardown, where a daemon that cannot be reached + must not stop a session from being killed. + """ + + phrase = (session.stream or {}).get("phrase") + live = self._live_streams() + if not phrase or live is None: + return False + try: + return await asyncio.to_thread(live.detach, str(phrase)) + except (LiveRejected, LiveUnavailable) as exc: + log.warning( + "could not detach %s's stream on output %s: %s", + session.id, + (session.stream or {}).get("output"), + exc, + ) + return False + + def _attachments(self) -> tuple[dict[str, Attachment], bool]: + """The daemon's streams by phrase, and whether it answered. + + Synchronous and tolerant: the read verbs report what they can, and a + down daemon turns `attached` false rather than failing a `list` that + has records to show. + """ + + live = self._live_streams() + if live is None: + return {}, False + try: + return {a.phrase: a for a in live.list()}, True + except (LiveRejected, LiveUnavailable) as exc: + log.debug("the stream daemon did not answer `list`: %s", exc) + return {}, False + + @staticmethod + def _stream_public( + stored: dict[str, Any], attachment: Attachment | None + ) -> dict[str, Any]: + """The record, with the daemon's live view merged in. + + ``attached`` is its own field and is **not** folded into ``enabled``: a + record may say a session is enabled while the daemon is down, and "the + operator asked for this" and "a capture is running right now" are two + different facts. ``url`` and ``viewers`` are *absent* when it is false, + rather than stale -- a URL that no longer serves anything is worse than + no URL. + """ + + public = dict(stored) + public["attached"] = attachment is not None + if attachment is not None: + public["url"] = attachment.url + public["viewers"] = attachment.viewers + public["fps"] = attachment.fps + return public + + async def _sweep_streams(self) -> None: + """Stamp `touch` on every session somebody is watching. + + The one exception to "`touch` is the only mark of activity" + (CONTRACT-sessions.md §4), and the only thing in this daemon that + treats an observation as an intention. It exists for one failure: an + operator watching a long job on his phone, saying nothing to the + broker, and losing the session at three hours *because* watching was + all he was doing. + + It cannot hold a session open by itself. An enabled stream with no + viewer runs no `grim` and stamps nothing, so a forgotten `enable` is + not a leak -- which is why this is safe to run unconditionally. + """ + + while True: + await asyncio.sleep(self.config.stream_sweep_s) + try: + await self._touch_watched() + except asyncio.CancelledError: + raise + except Exception: # noqa: BLE001 -- a sweep must outlive one failure + log.exception("the stream viewer sweep failed; continuing") + + async def _touch_watched(self) -> list[str]: + """One pass of the sweep. Returns the ids it touched.""" + + live = self._live_streams() + if live is None: + return [] + try: + counts = await asyncio.to_thread(live.viewers) + except (LiveRejected, LiveUnavailable) as exc: + log.debug("the stream daemon did not answer `viewers`: %s", exc) + return [] + touched: list[str] = [] + for session in self.registry.live(): + phrase = (session.stream or {}).get("phrase") + if not phrase: + continue + viewers = counts.get(str(phrase), 0) + if viewers <= 0: + continue + self.registry.touch(session, force_flush=True) + touched.append(session.id) + self.audit.emit( + "stream_viewer_activity", + id=session.id, + output=(session.stream or {}).get("output"), + viewers=viewers, + ) + return touched + async def _sweep_pointer(self) -> None: """Expire a stale lease even when nobody asks. @@ -2152,6 +2600,22 @@ class SessionManager: self.driver = PointerDriver(self.config.pointer_socket) return self.driver + # -- the stream daemon ------------------------------------------------ + + def _live_streams(self) -> LiveStreams | None: + """The stream daemon's client, built on first use. + + ``None`` rather than raising, unlike :meth:`_pointer_driver`, and the + asymmetry is the point: a pointer verb with no driver is a verb that + cannot do its job, while a *read* of stream state with no daemon has a + true answer -- the records, with ``attached: false``. The write paths + (`_attach_stream`) raise `stream_unavailable` for themselves. + """ + + if self.live is None and self.config.live_socket is not None: + self.live = LiveStreams(self.config.live_socket) + return self.live + async def _drive(self, verb: str, *args: Any) -> Any: """One driver call, off the event loop. diff --git a/src/hhstreamd/protocol.py b/src/hhstreamd/protocol.py index 73d5579..cca2c2f 100644 --- a/src/hhstreamd/protocol.py +++ b/src/hhstreamd/protocol.py @@ -102,8 +102,17 @@ EXIT_CODE_FOR_ERROR: dict[str, ExitCode] = { "pointer_busy": ExitCode.BUSY, # 9 — the lease is required and not held "no_lease": ExitCode.NO_LEASE, + # 8 — a phrase that is not this session's (v4) + "stream_busy": ExitCode.BUSY, + # 6 — the stream daemon's own cap (v4) + "stream_budget": ExitCode.POLICY, # 10 — the compositor side failed; the session itself is fine "compositor": ExitCode.COMPOSITOR, + # The stream could not be reached or has no output to aim at (v4). Exit 10 + # for the reason `driver_refused` is: the *session* keeps running, and a + # caller that treated an unreachable stream as a dead session would kill an + # application over a web page. + "stream_unavailable": ExitCode.COMPOSITOR, "compositor_unavailable": ExitCode.COMPOSITOR, "no_container": ExitCode.COMPOSITOR, "driver_unavailable": ExitCode.COMPOSITOR, @@ -156,6 +165,12 @@ VERBS = ( # warp retargets the keys. So there is no `keyboard.acquire`. "keyboard.key", "keyboard.type", + # v4: the per-session live stream. `list` and `status` read; `enable` and + # `disable` are in the mutation set below. + "stream.enable", + "stream.disable", + "stream.list", + "stream.status", ) #: Verbs whose reply is remembered by ``request_id``, so a caller that died @@ -185,6 +200,12 @@ MUTATING_VERBS = frozenset( # nothing, and a cached one would be a stale one. "keyboard.key", "keyboard.type", + # v4. `enable` mints or rotates a phrase: a retry that minted a second + # one would invalidate the URL somebody is already holding, and + # `disable` is here for symmetry -- a retried disable after a daemon + # restart must report what the first one did. + "stream.enable", + "stream.disable", } ) diff --git a/src/hhstreamd/registry.py b/src/hhstreamd/registry.py index 41770b4..633b75e 100644 --- a/src/hhstreamd/registry.py +++ b/src/hhstreamd/registry.py @@ -23,11 +23,12 @@ from .clock import Clock, iso log = logging.getLogger("hhstreamd.registry") #: 3 adds the container, the pointer bookmark and the compositor signature -#: (PLAN §7). The number is read, never written by anyone but this module: a -#: record carrying an older one is **migrated on load** (:func:`migrate`), not -#: refused, because the processes it describes are still running and refusing -#: the record would strand them. -SCHEMA = 3 +#: (PLAN §7); 4 adds the live stream (CONTRACT-sessions.md §6, v4). The number +#: is read, never written by anyone but this module: a record carrying an older +#: one is **migrated on load** (:func:`migrate`), not refused, because the +#: processes it describes are still running and refusing the record would +#: strand them. +SCHEMA = 4 #: Tombstones survive this long so a ``status`` after an unexplained failure #: still answers the question. ``gc`` prunes them. @@ -138,6 +139,17 @@ class Session: #: ``{bookmark, last_released_at, released_by}``. Absent (``None``) means #: the pointer was never aimed here — not that its position is unknown. pointer: dict[str, Any] | None = None + #: The live stream, or ``None`` for a session nobody switched one on for + #: (CONTRACT-sessions.md §4). ``{enabled, phrase, output, fps, port, + #: enabled_at}``; the daemon's ``url`` and ``viewers`` are merged in by the + #: manager when it could be asked, and are deliberately not stored -- a + #: viewer count on disk is a number that is wrong the moment it is written. + #: + #: The phrase lives here because it has to survive a broker restart and a + #: re-enable: it is the stream's only credential, and an operator who read + #: four words out to somebody must not have them change underneath him + #: because the daemon bounced. + stream: dict[str, Any] | None = None #: ``HYPRLAND_INSTANCE_SIGNATURE`` of the compositor this session's #: container lives in, so a restarted broker re-attaches to the same #: instance instead of picking whichever one is there now. @@ -234,6 +246,38 @@ def container_record( } +def stream_record( + *, + phrase: str, + output: str, + fps: int, + enabled_at: str, +) -> dict[str, Any]: + """One ``stream`` sub-record (CONTRACT-sessions.md §4). + + Only what survives a restart. The daemon's ``url`` and ``viewers`` are + merged in by the manager when it could be asked and are deliberately absent + here: a viewer count on disk is wrong the moment it is written, and a URL + assembled from a remembered port is a URL that lies the first time the + listener moves. + + ``enabled`` is always ``True`` in a stored record. A session that was + never enabled has ``stream: None``, and `disable` clears the field rather + than setting ``enabled: False`` -- "nobody asked for a stream here" and + "somebody asked and then stopped" are the same state as far as anything + downstream is concerned, and two spellings for one state is how a + `if stream:` check starts being wrong. + """ + + return { + "enabled": True, + "phrase": phrase, + "output": output, + "fps": fps, + "enabled_at": enabled_at, + } + + def pointer_record( *, bookmark: dict[str, Any] | None, @@ -284,7 +328,14 @@ def migrate(data: dict[str, Any]) -> dict[str, Any]: upgraded = dict(data) # Only ever *adds* keys. A v2 record that somehow carries a container must # keep it: the field is new to the schema, not new to the session. - for key in ("container", "pointer", "signature"): + # + # `stream` (schema 4) is `None` for the same reason the others are: a + # session written by a daemon that knew nothing of streams had no stream, + # and "no stream" is a defined value rather than a guess. It must NOT be + # invented as `{"enabled": false}` either -- a record that says it was + # asked about and declined is a different claim from one that was never + # asked. + for key in ("container", "pointer", "signature", "stream"): upgraded.setdefault(key, None) upgraded["schema"] = SCHEMA log.info("migrated record %s from schema %s to %s", data.get("id"), schema, SCHEMA) diff --git a/src/hhstreamd/server.py b/src/hhstreamd/server.py index 0660994..7fa81d0 100644 --- a/src/hhstreamd/server.py +++ b/src/hhstreamd/server.py @@ -75,6 +75,10 @@ class ControlServer: "pointer.state": self._op_pointer_state, "keyboard.key": self._op_keyboard_key, "keyboard.type": self._op_keyboard_type, + "stream.enable": self._op_stream_enable, + "stream.disable": self._op_stream_disable, + "stream.list": self._op_stream_list, + "stream.status": self._op_stream_status, } # ------------------------------------------------------------------ @@ -264,6 +268,9 @@ class ControlServer: wait_ready_s=request.get("wait_ready_s"), home=request.get("home"), ephemeral_home=bool(request.get("ephemeral_home", False)), + stream=bool(request.get("stream", False)), + stream_phrase=request.get("stream_phrase"), + stream_fps=request.get("stream_fps"), ) return protocol.ok(id=record["id"], state=record["state"], session=record) @@ -297,6 +304,32 @@ class ControlServer: ) return protocol.ok(id=record["id"], state=record["state"], session=record) + async def _op_stream_enable( + self, request: dict[str, Any], caller: dict[str, Any] + ) -> dict[str, Any]: + return protocol.ok( + **await self.manager.stream_enable( + request.get("id"), + phrase=request.get("phrase"), + fps=request.get("fps"), + ) + ) + + async def _op_stream_disable( + self, request: dict[str, Any], caller: dict[str, Any] + ) -> dict[str, Any]: + return protocol.ok(**await self.manager.stream_disable(request.get("id"))) + + async def _op_stream_list( + self, request: dict[str, Any], caller: dict[str, Any] + ) -> dict[str, Any]: + return protocol.ok(**self.manager.stream_list()) + + async def _op_stream_status( + self, request: dict[str, Any], caller: dict[str, Any] + ) -> dict[str, Any]: + return protocol.ok(**self.manager.stream_status(request.get("id"))) + async def _op_container_list( self, request: dict[str, Any], caller: dict[str, Any] ) -> dict[str, Any]: From 850f9caa1d594cae02abe3fdd2cc7a13ca39d805 Mon Sep 17 00:00:00 2001 From: Hermenix Date: Sun, 11 Oct 2026 22:48:00 +0000 Subject: [PATCH 04/12] feat(cli): `session stream` and `start --stream` MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The operator's side of the four verbs: `hhstream session stream enable|disable|list|status`, and `start --stream [--stream-phrase] [--stream-fps]` for the one-call case. `stream enable` prints the URL on **stdout and nothing else**, with the detail on stderr, so `url=$(hhstream session stream enable ID)` works -- the same shape `start` already has for the session id. `start --stream` prints the id and then the URL, two lines, because a caller who asked for both should not have to run a second verb for the second one. The phrase is a credential, and the two renderers disagree on purpose: * `render_streams` (`stream list`) does **not** show it. A listing is what ends up in a shared terminal, a paste or a screenshot, so what it shows is enough to recognise and manage a stream -- session, output, rate, viewers, `LIVE` -- and not enough to watch one. It also prints the daemon's reachability, because "two records and no daemon" is a different situation from "two records and two captures" and the rows alone cannot say which. * `render_stream` (`stream status ID`) does show the URL. That question was asked about one session by name, which is what somebody about to watch it does, and making them reach for `--json` for the thing the verb is for would be the worse trade. When the daemon is not publishing, `status` says so and prints no URL -- the phrase is kept, so re-enabling restores the same one. `start --stream` that starts the session but cannot open the stream exits 0 and says `WARNING: the session is running but its stream was not enabled: …` on stderr. Loudly, because the exit code is the session's and a caller who asked for a stream must not read a 0 as one. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_011ZeH65zd6pFTgBSu52W9PE --- src/hhstream_session/__init__.py | 252 +++++++++++++++++++++++++++++++ src/hhstream_session/format.py | 81 ++++++++++ 2 files changed, 333 insertions(+) diff --git a/src/hhstream_session/__init__.py b/src/hhstream_session/__init__.py index db35def..cb18495 100644 --- a/src/hhstream_session/__init__.py +++ b/src/hhstream_session/__init__.py @@ -38,6 +38,9 @@ The verb tree, beyond the original eight: session keyboard type ID TEXT one argument, raw UTF-8, no escapes session keyboard key ID NAME tap: down then up session keyboard down/up ID NAME + session stream enable ID publish ID's screen behind a spoken phrase + session stream disable ID stop publishing, and forget the phrase + session stream list/status what is being published, and who is watching The pointer *and keyboard* verbs share one precondition — the pointer lease — and the lease id lives in :mod:`hhstream_session.lease` between invocations, @@ -187,6 +190,27 @@ def _build_namespace(parser: argparse.ArgumentParser) -> None: help="deduplication token: retrying a start with the same id returns the " "original reply instead of launching a second application", ) + start.add_argument( + "--stream", + action="store_true", + help="also enable a live stream for this session and print its URL. " + "Needs the session's container output, which this creates — which is " + "why it is a flag here rather than a second command you have to " + "sequence yourself.", + ) + start.add_argument( + "--stream-phrase", + metavar="PHRASE", + help="use this phrase instead of a minted one (four words and a " + "five-character tail). It is the stream's only credential, so it is " + "held to exactly that shape.", + ) + start.add_argument( + "--stream-fps", + type=int, + metavar="N", + help="frames per second for the stream (default: the broker's)", + ) start.add_argument( "--no-container", action="store_true", @@ -303,6 +327,9 @@ def _build_namespace(parser: argparse.ArgumentParser) -> None: # ---- keyboard ---------------------------------------------------- _build_keyboard(verbs, common) + # ---- stream ------------------------------------------------------ + _build_stream(verbs, common) + #: Button names the driver knows (``hhstream_pointer.buttons.CODES``), mirrored #: here rather than imported: this module's whole integration promise is that it @@ -627,6 +654,136 @@ def _build_keyboard(verbs: Any, common: argparse.ArgumentParser) -> None: sub.add_argument("name", help="XKB keysym name, e.g. Control_L, Shift_L") +#: Printed under `stream --help` and under `enable`. The two facts an operator +#: gets wrong about this group, and they pull in opposite directions: the +#: phrase is a password, and *watching* is the one thing besides `touch` that +#: keeps a session alive. +STREAM_NOTE = ( + "CREDENTIAL: the phrase in the URL is the stream's only protection — no " + "TLS, no second factor. Anyone who can reach the port and guess four " + "words can watch. It is never written to the audit log, and it is not " + "echoed by 'list' unless you ask for --json.\n\n" + "ACTIVITY: 'stream enable|disable|list|status' do NOT reset the idle " + "clock, like every pointer and keyboard verb. But A CONNECTED VIEWER " + "DOES: about once a minute the broker stamps 'touch' on any session " + "somebody is watching, so a session you are watching on a phone is not " + "reaped at three hours. An enabled stream with nobody watching captures " + "nothing and keeps nothing alive." +) + + +def _build_stream(verbs: Any, common: argparse.ArgumentParser) -> None: + """``session stream `` — a live view of one session, behind a phrase. + + A nested namespace for the reason ``pointer`` is one: these four verbs + share a precondition nothing else here has. It is the session's **container + output** — a stream never falls back to the main output, because that one + carries every other session. + """ + + namespace = verbs.add_parser( + "stream", + parents=[common], + help="publish one session's screen on the LAN, behind a spoken phrase", + description=( + "A live view of one session, in a browser, at a URL whose path is " + "four words and a five-character tail. Off by default and per " + "session: a session does not stream because it exists, it streams " + "because it was switched on.\n\n" + "What is published is the session's own container output, never " + "the main one — that output carries every other session, and " + "putting it behind one session's phrase is the mistake this " + "refuses to make (exit 10 with no container).\n\n" + "The transport is MJPEG over HTTP: a browser is the whole " + "client. Capture runs only while somebody is connected." + ), + epilog=f"{STREAM_NOTE}\n\n{ACTIVITY_NOTE}", + formatter_class=argparse.RawDescriptionHelpFormatter, + ) + streams = namespace.add_subparsers(dest="stream_verb", metavar="VERB") + namespace.set_defaults( + func=_cmd_stream_list, hhstream_session_handler=_cmd_stream_list, json=False + ) + + def add_stream_verb(name: str, handler, **kwargs: Any): + sub = streams.add_parser(name, parents=[common], **kwargs) + sub.set_defaults(func=handler, hhstream_session_handler=handler) + sub.add_argument("--json", action="store_true", help="emit the protocol object on stdout") + return sub + + enable = add_stream_verb( + "enable", + _cmd_stream_enable, + help="switch a session's stream on and print its URL", + description=( + "Mint a phrase, open a capture, and print the URL on stdout — so " + "$(hhstream session stream enable ID) is the URL and nothing " + "else. The phrase is kept in the session's record, so it survives " + "a broker restart and a re-enable: somebody you read it out to " + "keeps the URL you gave them.\n\n" + "Re-enabling an enabled session prints the same URL. A --phrase " + "that differs from the stored one is exit 8 rather than a silent " + "rotation, because rotating under a caller holding the old URL is " + "the one outcome neither answer can be mistaken for. 'disable' " + "then 'enable' rotates it." + ), + epilog=f"{STREAM_NOTE}\n\n{ACTIVITY_NOTE}", + formatter_class=argparse.RawDescriptionHelpFormatter, + ) + enable.add_argument("id", help="session id") + enable.add_argument( + "--phrase", + metavar="PHRASE", + help="use this phrase instead of a minted one. Held to exactly the " + "generated shape (four words of 3+ letters, then a five-character " + "tail): it is a password, and a looser check would accept 'a-b-c'.", + ) + enable.add_argument( + "--fps", + type=int, + metavar="N", + help="frames per second (default: the broker's; the daemon caps it)", + ) + + disable = add_stream_verb( + "disable", + _cmd_stream_disable, + help="switch a session's stream off and forget its phrase", + description=( + "Close the capture, drop the route, and clear the record. The " + "phrase is gone: a later 'enable' mints a new one, which is how a " + "phrase is rotated.\n\n" + "Disabling a stream the daemon has already forgotten is not an " + "error — that is the ordinary case after the daemon restarts, and " + "the record has to be cleared either way. 'detached' says which " + "happened." + ), + ) + disable.add_argument("id", help="session id") + + add_stream_verb( + "list", + _cmd_stream_list, + help="every session with a stream (does NOT count as activity)", + description=( + "Every session a stream was enabled on, with its output, rate and " + "connected viewer count. The phrase is shown only under --json: " + "this listing is the kind of thing that ends up in a terminal " + "somebody is sharing, and the phrase is the credential.\n\n" + "'available' is whether the stream daemon could be asked at all, " + "so 'no streams' is distinguishable from 'no daemon' without " + "parsing an error." + ), + ) + + status = add_stream_verb( + "status", + _cmd_stream_status, + help="one session's stream (does NOT count as activity)", + ) + status.add_argument("id", nargs="?", help="session id; omit for all of them") + + def _button_help() -> str: return f"one of: {', '.join(BUTTONS)} (default: left)" @@ -749,6 +906,9 @@ def _cmd_start(args: argparse.Namespace) -> int: home=args.home, ephemeral_home=args.ephemeral_home or None, request_id=args.request_id, + stream=args.stream or None, + stream_phrase=args.stream_phrase, + stream_fps=args.stream_fps, ) if not reply.get("ok"): # A failed start may still have created a session (it exited at once, or @@ -778,6 +938,27 @@ def _cmd_start(args: argparse.Namespace) -> int: "(the only mark of activity; list/status/logs/gc do not count)", file=sys.stderr, ) + if args.stream: + stream = session.get("stream") or {} + error = session.get("stream_error") + if stream.get("url"): + # stdout, beside the id: the URL is the other thing a caller asked + # for, and `$(... --stream)` giving the id and the URL on two lines + # is more useful than making them run a second verb for it. + print(stream["url"]) + print( + f"watching it counts as activity; stop it with: hhstream " + f"session stream disable {session_id}", + file=sys.stderr, + ) + else: + # The session started. Saying so loudly, because the exit code is + # 0 and a caller who asked for a stream must not read that as one. + print( + f"WARNING: the session is running but its stream was not " + f"enabled: {error or 'the broker reported no stream'}", + file=sys.stderr, + ) return int(ExitCode.OK) @@ -1312,6 +1493,77 @@ def _cmd_keyboard_hold(args: argparse.Namespace) -> int: return _finish_gesture(args, reply, render, args.id) +# -------------------------------------------------------------------------- +# the live stream +# -------------------------------------------------------------------------- + + +@_guard +def _cmd_stream_enable(args: argparse.Namespace) -> int: + reply = _request( + args, "stream.enable", id=args.id, phrase=args.phrase, fps=args.fps + ) + if not reply.get("ok"): + return _finish(args, reply, lambda r: "") + if args.json: + fmt.dump_json(reply) + return int(ExitCode.OK) + stream = reply.get("stream") or {} + # stdout is the URL and nothing else, the same shape `start` and `pointer + # acquire` already have: one token a caller passes on, detail on stderr. + print(stream.get("url") or "") + print( + f"{reply.get('id')}: streaming output {stream.get('output')} " + f"at {stream.get('fps')} fps", + file=sys.stderr, + ) + print( + "the phrase in that URL is its only protection, and watching it counts " + "as activity; stop it with: hhstream session stream disable " + f"{reply.get('id')}", + file=sys.stderr, + ) + if not stream.get("attached"): + # `enable` cannot normally get here -- it had to attach to succeed -- + # but a reply without it means the daemon answered and then went away, + # and a printed URL that serves nothing is worse than a warning. + print( + "WARNING: the broker recorded this stream but the daemon did not " + "confirm it is attached; the URL may not answer", + file=sys.stderr, + ) + return int(ExitCode.OK) + + +@_guard +def _cmd_stream_disable(args: argparse.Namespace) -> int: + reply = _request(args, "stream.disable", id=args.id) + + def render(r: dict[str, Any]) -> str: + detached = r.get("detached") + return ( + f"{r.get('id')}: stream disabled and its phrase forgotten" + + ("" if detached else "; the daemon had already dropped it") + ) + + return _finish(args, reply, render) + + +@_guard +def _cmd_stream_list(args: argparse.Namespace) -> int: + reply = _request(args, "stream.list") + return _finish(args, reply, fmt.render_streams) + + +@_guard +def _cmd_stream_status(args: argparse.Namespace) -> int: + session_id = getattr(args, "id", None) + reply = _request(args, "stream.status", id=session_id) + if session_id is None: + return _finish(args, reply, fmt.render_streams) + return _finish(args, reply, fmt.render_stream) + + # -------------------------------------------------------------------------- # standalone use # -------------------------------------------------------------------------- diff --git a/src/hhstream_session/format.py b/src/hhstream_session/format.py index c113e9c..6d70c39 100644 --- a/src/hhstream_session/format.py +++ b/src/hhstream_session/format.py @@ -120,6 +120,87 @@ def render_list( ) +def render_streams(reply: dict[str, Any]) -> str: + """``session stream list``: what is published, and who is watching. + + **The phrase is not in this table**, and that is the whole shape of it. A + listing is the kind of thing that ends up in a shared terminal, a paste or + a screenshot, and the phrase is the stream's only credential — so what is + shown is enough to recognise and manage a stream (the session, the output, + the rate, the viewers) and not enough to watch one. ``--json`` carries the + phrase and the URL, because a caller that asked for the object asked for + the credential too. + + ``available`` is printed even when there are streams: "two records and no + daemon" is a different situation from "two records and two captures", and + the difference is invisible from the rows alone. + """ + + rows = [] + for row in reply.get("streams") or []: + rows.append( + [ + _cell(row.get("id")), + _cell(row.get("app")), + _cell(row.get("output")), + _cell(row.get("fps")), + "yes" if row.get("attached") else "NO", + _cell(row.get("viewers")), + ] + ) + lines = [ + render_table(rows, ["ID", "APP", "OUTPUT", "FPS", "LIVE", "VIEWERS"]) + if rows + else "no session has a stream enabled" + ] + available = reply.get("available") + lines.append( + "daemon: " + + ( + "answering" + if available + else "NOT REACHABLE — the records below are what was asked for, " + "not what is running" + ) + ) + if rows: + lines.append( + "the URL (with its phrase) is in --json, or in `stream status `" + ) + return "\n".join(lines) + + +def render_stream(reply: dict[str, Any]) -> str: + """``session stream status ID``: one session's stream, URL included. + + The URL *is* here, unlike in the listing: this was asked about one session + by name, which is what somebody about to watch it does, and making them + reach for ``--json`` to get the thing the verb is for would be a worse + trade than the one the listing makes. + """ + + stream = reply.get("stream") + if not stream: + return f"{_cell(reply.get('id'))}: no stream enabled" + lines = [ + f"{_cell(reply.get('id'))}: output {_cell(stream.get('output'))} " + f"at {_cell(stream.get('fps'))} fps", + ] + if stream.get("attached"): + lines.append(f" url {_cell(stream.get('url'))}") + lines.append(f" viewers {_cell(stream.get('viewers'))}") + else: + # No URL, deliberately: the record's phrase is still there, but a URL + # that no longer serves anything is worse than none. See + # `SessionManager._stream_public`. + lines.append( + " url — (the daemon is not publishing this; the phrase is " + "kept, so re-enabling restores the same URL)" + ) + lines.append(f" enabled {_cell(stream.get('enabled_at'))}") + return "\n".join(lines) + + def render_containers(reply: dict[str, Any]) -> str: """``session container`` with no id: every container, and every orphan. From ca5dbfb2eafbd6aeed325968f274b06ec5f7a2a9 Mon Sep 17 00:00:00 2001 From: Hermenix Date: Sun, 11 Oct 2026 22:48:22 +0000 Subject: [PATCH 05/12] feat(nix): the `hhstream-live` unit, and the two things it has to be handed `services.hyprland-headless-stream.sessions.stream.enable` (default **false**) runs `hhstream-live` and hands the broker `/run/hhstream/live.sock`. Running it still turns nothing on: a stream exists only for the sessions `hhstream session stream enable` was run on, which is the plan's rule that a stream is never on for all sessions at all times. A fourth unit rather than a flag on an existing one, for the pointer driver's reason: `grim` blocks tens to hundreds of milliseconds, so the capture must not run where `touch` and `kill` are answered, and a LAN-facing HTTP parser must not run in the frame server whose one-socket contract cannot grow one. Two things the unit must be handed, both of them MEASURED failures rather than precautions: 1. **The capture command, spelled out, with grim's store path.** A unit's `PATH` has no `grim` (`docs/TODO/capture-binary-and-the-unit-path.md`), and the symptom is a stream that answers with an empty frame area and says why only in the daemon's journal. Every word is passed as `--capture-command=` and not `--capture-command `, because grim's last word is a bare `-` and argparse reads a lone dash as an option prefix rather than a value -- a unit crash-looping with "expected one argument", which is how that form was chosen. 2. **The compositor's session.** `XDG_RUNTIME_DIR` on the unit plus `execLive`, which discovers whichever `wayland-N` the compositor actually bound and exports it -- `execServer`'s trick, for `execServer`'s reason: the name is only known once the compositor is up, so it cannot be an `Environment=` line. Without it the unit was `active (running)`, the broker attached, `stream status` said `attached: true`, and every capture exited 1 with "XDG_RUNTIME_DIR is invalid or not set in the environment" until the publisher gave up after ten in a row. checks/vm-stream.nix found that on its first real run, which is the whole reason that check reads bytes instead of asserting a 200. `SupplementaryGroups` gets the compositor's group for the pointer driver's reason one step further -- this one has to *capture*, not merely find the socket -- and there is deliberately no `RuntimeDirectory=`: a second one on /run/hhstream would delete the broker's control socket when this unit stopped, which the plan names as the Stage 3 risk. `sessions.stream.fps` is the daemon's own default and nothing else sends one, so the option changes what actually happens; the port and bind address are the daemon's too, and the broker learns the URL from it. `nix/package.nix` names `hhstream_live.control` and `hhstreamd.live` in `pythonImportsCheck`: they are the one pair in this package that has to agree with each other across two processes, so an import error in either is worth finding at build time rather than the first time an operator enables a stream. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_011ZeH65zd6pFTgBSu52W9PE --- nix/module.nix | 177 ++++++++++++++++++++++++++++++++++++++++++++++-- nix/package.nix | 7 ++ 2 files changed, 177 insertions(+), 7 deletions(-) diff --git a/nix/module.nix b/nix/module.nix index 7a11c8f..2896664 100644 --- a/nix/module.nix +++ b/nix/module.nix @@ -46,6 +46,7 @@ let brokerRuntimeDir = "/run/${brokerRuntimeDirName}"; controlSocket = "${brokerRuntimeDir}/control.sock"; pointerSocket = "${brokerRuntimeDir}/pointer.sock"; + liveSocket = "${brokerRuntimeDir}/live.sock"; # Addresses that mean "this listener answers on loopback only". The spelling # `localhost` is included because it is what a person writes; it is not a @@ -164,6 +165,35 @@ let done exec ${lib.getExe' cfg.package "hhstream-server"} "$@" ''; + + # The same problem, for the stream daemon, one step worse. + # + # `grim` is an ordinary Wayland client: it needs XDG_RUNTIME_DIR *and* the + # name of the socket the compositor actually bound, and that name is only + # known once the compositor is up -- so, exactly as above, it cannot be an + # `Environment=` line. + # + # MEASURED, in checks/vm-stream.nix, which is the check that exists to find + # this: with the group but without the environment, the unit came up + # `active (running)`, the broker attached, `stream status` said + # `attached: true`, and every capture failed with + # + # grim: error: XDG_RUNTIME_DIR is invalid or not set in the environment. + # + # until the publisher gave up after ten in a row. A viewer with the right + # phrase got a page and no frames, and nothing in the session's own state + # said why -- the sibling failure to + # docs/TODO/capture-binary-and-the-unit-path.md, and the reason this unit + # spells out the whole capture command *and* hands it a session. + execLive = pkgs.writeShellScript "hhstream-exec-live" '' + for candidate in "$XDG_RUNTIME_DIR"/wayland-*; do + if [ -S "$candidate" ]; then + export WAYLAND_DISPLAY="''${candidate##*/}" + break + fi + done + exec ${lib.getExe' cfg.package "hhstream-live"} "$@" + ''; hyprlandConf = pkgs.writeText "hyprland.conf" (import ./hyprland-conf.nix { inherit lib; inherit (cfg) outputs extraConfig; @@ -701,15 +731,57 @@ in { # The session-stream web app — `hhstream-live`, the router that serves a # live view of one session per phrase, over one port - # (`docs/plans/2026-10-04-session-stream-webapp.md`). + # (`docs/plans/2026-10-04-session-stream-webapp.md`, Stage 2 and 3). # - # Only the *reach* of it is declared here so far: the port every stream - # shares, the address it will bind, and whether that port is opened in - # the firewall. The unit that runs the app, the phrases the broker keeps - # in its records and the `session stream enable` verbs are later stages - # of that plan — and deliberately so: a stream is switched on per - # session, never for all of them, so nothing here turns one on. + # `enable` runs the unit and hands the broker `/run/hhstream/live.sock`, + # over which `hhstream session stream enable ` switches one stream + # on. Everything else here is the *reach* of it: the port every stream + # shares, the address it binds, and whether that port is opened in the + # firewall. + # + # Running it still turns nothing on. A stream exists only for the + # sessions it was enabled on — a stream is switched on per session, never + # for all of them — so `enable = true` is a listener serving one form + # until somebody asks for a session. stream = { + enable = lib.mkOption { + type = lib.types.bool; + default = false; + description = '' + Run `hhstream-live`, the web app that publishes one session's + screen at a URL whose path is a spoken phrase, and give the broker + the socket it switches streams on over + (`/run/hhstream/live.sock`). + + Off by default, and running it turns **nothing** on: a stream + exists only for the sessions `hhstream session stream enable` was + called on, and until then this is a listener serving one form. + That is deliberate — a live view of a desktop is readable by + anyone who can reach the port and guess the phrase, so switching + one on is a per-session act and never a consequence of the service + being up. + + Needs {option}`sessions.enable`: the phrase lives in the session + record, and there are no sessions without the broker. + ''; + }; + + fps = lib.mkOption { + type = lib.types.ints.positive; + default = 5; + description = '' + Frames a second a stream captures at when nobody says otherwise. + + Five is one frame per ~200 ms plus a ~46 ms `grim`, measured rather + than guessed; the daemon's own hard cap is 10, because this box is + also running the compositor and the main frame publisher. Capture + happens **only while somebody is connected**, so an enabled stream + nobody is watching costs nothing at any rate. + + `hhstream session stream enable --fps N` overrides it per stream. + ''; + }; + port = lib.mkOption { type = lib.types.port; default = 9988; @@ -1236,5 +1308,96 @@ in { ]; }; }; + + # The stream daemon. A third executor beside the pointer driver, and the + # split is the driver's: `grim` blocks tens to hundreds of milliseconds, so + # the capture must not run where `touch` and `kill` are answered, and a + # LAN-facing HTTP parser must not run in the frame server whose one-socket + # contract cannot grow one (docs/CONTRACT-sessions.md §3.5). + systemd.services.hhstream-live = lib.mkIf (cfg.sessions.enable && cfg.sessions.stream.enable) { + description = "Per-session live stream for the headless Hyprland desktop"; + wantedBy = ["multi-user.target"]; + after = ["hyprland-headless.service"]; + # `BindsTo` for the pointer driver's reason: this process captures from + # that compositor's outputs through `grim`, so when the compositor goes + # it has nothing left to capture. The broker reports such a stream as + # `attached: false` rather than pretending, and re-enabling re-attaches. + bindsTo = ["hyprland-headless.service"]; + + environment = { + PYTHONUNBUFFERED = "1"; + # The compositor's session, not this unit's own: `grim` has to find the + # Wayland socket in here. `WAYLAND_DISPLAY` is set by `execLive` at + # start, because the socket's name is only known once the compositor + # has bound it. HOME is deliberately NOT redirected here the way the + # frame server's is -- this unit runs as the operator's account, and + # the compositor's runtime directory is not writable by it. + XDG_RUNTIME_DIR = runtimeDir; + }; + + serviceConfig = { + Type = "simple"; + User = cfg.sessions.user; + # The compositor's runtime directory is 0750 and owned by this group, + # and `grim` is a Wayland client of it -- same reason the pointer + # driver needs the group, one step further: this one has to *capture*, + # not merely find the socket. + SupplementaryGroups = [cfg.sessions.group]; + + # No `RuntimeDirectory`: /run/hhstream belongs to the broker's unit and + # to the tmpfiles entry above, and a second RuntimeDirectory= on the + # same path would delete the broker's control socket when this unit + # stopped. The plan calls this out by name as the Stage 3 risk. + WorkingDirectory = brokerRuntimeDir; + UMask = "0007"; + + Restart = "on-failure"; + RestartSec = 1; + NoNewPrivileges = true; + KillMode = "process"; + + # `grim` is NOT on a unit's PATH, and the symptom of leaving it off is + # a stream that serves an empty frame area without saying why -- + # docs/TODO/capture-binary-and-the-unit-path.md, found the first time + # the capture ran as a unit rather than from a login shell. So the + # whole capture command is spelled out here, with grim's store path. + # + # `--capture-command=` and not `--capture-command `: + # MEASURED, as a unit crash-looping with + # + # hhstream-live serve: error: argument --capture-command: expected + # one argument + # + # because grim's last word is a bare `-` (its "write to stdout"), and + # argparse reads a lone dash as an option prefix rather than a value. + # The `=` form is unambiguous for every word, so it is used for every + # word rather than only the one that needs it. + # Same second line of defence as the frame server's: `BindsTo=` stops + # this unit when the compositor goes, but the compositor is only + # *ready* once its socket is bound, and `execLive` has to find that + # socket to export its name. + ExecStartPre = "${findSession} >/dev/null"; + + ExecStart = lib.escapeShellArgs ( + [ + execLive + "serve" + "--control-socket=${liveSocket}" + "--bind=${cfg.sessions.stream.bind}" + "--port=${toString cfg.sessions.stream.port}" + "--fps=${toString cfg.sessions.stream.fps}" + ] + ++ map (word: "--capture-command=${word}") [ + (lib.getExe pkgs.grim) + "-c" # include the cursor + "-t" + "jpeg" + "-q" + "80" + "-" # write to stdout + ] + ); + }; + }; }; } diff --git a/nix/package.nix b/nix/package.nix index fcdaa13..e5078c0 100644 --- a/nix/package.nix +++ b/nix/package.nix @@ -38,6 +38,13 @@ python3.pkgs.buildPythonApplication { # The web app: importing the server pulls in the router, the publisher and # the pages. "hhstream_live.server" + # The two halves of `live.sock` (CONTRACT-sessions.md §3.5). Both are + # stdlib-only, and they are named here because they are the one pair in + # this package that has to agree with each other across two processes -- + # so an import error in either is worth finding at build time rather than + # the first time an operator enables a stream. + "hhstream_live.control" + "hhstreamd.live" ]; # `pythonImportsCheck` only *imports*, and `hhstream_live.phrase` reads its From 5f5726c02efad6b1f1e1ed750d27915635e6df82 Mon Sep 17 00:00:00 2001 From: Hermenix Date: Sun, 11 Oct 2026 22:48:45 +0000 Subject: [PATCH 06/12] feat(mcp): the stream, as two tools and two resources MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The split mcp-v2 froze, applied to the stream: switching one on is an act, so it is a tool; what is being published is state, so it is a resource. 23 tools and 10 resources. Tools: `session_stream_enable` (returns the URL, takes an optional `phrase` and `fps`), `session_stream_disable`, and `session_start` gains `stream`, `stream_phrase` and `stream_fps` -- the last because a stream needs the container output that starting creates, so one call is the shape an agent actually wants. Resources: `hhstream://streams` (every stream, with `available` so "no streams" is not confused with "no daemon") and `hhstream://sessions/{id}/stream`. What the descriptions carry is the part that matters here, because an agent reads them and nothing else: * **CREDENTIAL.** The phrase in the URL is the stream's only protection. The resource descriptions say outright not to put the URL where a session's own screen can see it -- an agent that pastes it into the browser it is driving has published the desktop. * **ACTIVITY.** The stream tools do *not* reset the idle clock, but a connected viewer does, which is the first exception to that rule this surface has had to state. `session_touch` is still the only thing an agent can do about it. * `session_start`'s summary prints `stream url:` beside the id, and a stream that could not be opened is reported loudly as `stream: NOT enabled -- …`, because the call succeeded and an agent must not read that as a stream. `nix/mcp.nix` counts the new surface in `entryPointCheckPhase` (23 and 10), which is the only place those numbers are asserted against the *wrapped* binary. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_011ZeH65zd6pFTgBSu52W9PE --- nix/mcp.nix | 6 +- src/hhstream_mcp/resources.py | 38 ++++++++ src/hhstream_mcp/tools.py | 163 +++++++++++++++++++++++++++++++++- 3 files changed, 200 insertions(+), 7 deletions(-) diff --git a/nix/mcp.nix b/nix/mcp.nix index a876b06..fd7f6e6 100644 --- a/nix/mcp.nix +++ b/nix/mcp.nix @@ -14,7 +14,7 @@ # `buildPythonApplication` output is not. # # `grim` and `ffmpeg` do go on PATH, because the CLI itself spawns them -- -# `shot --output` and `record` are two of the twenty-one tools, and the server +# `shot --output` and `record` are two of the twenty-three tools, and the server # is launched by an MCP client whose PATH is not ours to predict. `--prefix` # and not `--set`, so a deployment that wants its own ahead of these puts them # on PATH and wins. @@ -89,11 +89,11 @@ python3.pkgs.buildPythonApplication { echo "checking the wrapped entry point..." tools=$($out/bin/hhstream-mcp --list-tools) echo "$tools" - test "$(echo "$tools" | wc -l)" -eq 21 + test "$(echo "$tools" | wc -l)" -eq 23 resources=$($out/bin/hhstream-mcp --list-resources) echo "$resources" - test "$(echo "$resources" | wc -l)" -eq 8 + test "$(echo "$resources" | wc -l)" -eq 10 narrowed=$($out/bin/hhstream-mcp --list-tools --toolsets keyboard | wc -l) echo "--toolsets keyboard offers $narrowed tools" diff --git a/src/hhstream_mcp/resources.py b/src/hhstream_mcp/resources.py index 43ce2f9..40a9654 100644 --- a/src/hhstream_mcp/resources.py +++ b/src/hhstream_mcp/resources.py @@ -359,6 +359,24 @@ ENTRIES: tuple[Entry, ...] = ( media_type=JSON, read=_json_read(["session", "container"], "the container list"), ), + Entry( + route=("streams",), + name="streams", + title="What is being published, and who is watching", + description=( + "Every session a live stream was enabled on, with its output, " + "rate, URL and connected viewer count -- and `available`, which " + "says whether the stream daemon could be asked at all, so 'no " + "streams' is distinguishable from 'no daemon'.\n\n" + "THE URLS HERE CONTAIN THEIR PHRASES, which are the only thing " + "protecting those streams. Do not put this where a session's own " + "screen can see it." + _NOT_ACTIVITY + " A connected viewer, " + "however, does: that is the one exception, and it is why a " + "session somebody is watching is not reaped." + ), + media_type=JSON, + read=_json_read(["session", "stream", "list"], "the stream list"), + ), Entry( route=("screen",), name="screen", @@ -405,6 +423,26 @@ ENTRIES: tuple[Entry, ...] = ( read=_json_read(["session", "logs"], "that session's logs"), parameters=("lines",), ), + Entry( + route=("sessions", "{id}", "stream"), + name="session-stream", + title="One session's live stream", + description=( + "Whether this session is being published, and if so the URL to " + "watch it at, the output it shows, its frame rate and how many " + "people are connected right now. `null` when no stream was " + "enabled.\n\n" + "`attached` is separate from `enabled` on purpose: a record can " + "say the operator asked for a stream while the daemon is down, " + "and those are two different facts. The URL is absent rather than " + "stale when `attached` is false.\n\n" + "THE URL CONTAINS THE PHRASE that protects it. Switch a stream on " + "with the `session_stream_enable` tool -- reading this never " + "enables one." + _NOT_ACTIVITY + ), + media_type=JSON, + read=_json_read(["session", "stream", "status"], "that session's stream"), + ), Entry( route=("outputs", "{name}"), name="output", diff --git a/src/hhstream_mcp/tools.py b/src/hhstream_mcp/tools.py index 1eed94c..8c83df0 100644 --- a/src/hhstream_mcp/tools.py +++ b/src/hhstream_mcp/tools.py @@ -104,6 +104,19 @@ HOLD_NOTE = ( "`keyboard_up Control_L`." ) +STREAM_NOTE = ( + "CREDENTIAL: the phrase in the stream's URL is its only protection -- no " + "TLS, no second factor. Anyone who can reach the port and guess four " + "words can watch this session's screen. Treat the URL as a password: do " + "not put it anywhere the session's own output goes. It is never written " + "to the broker's audit log.\n\n" + "ACTIVITY: the stream tools do NOT reset the idle clock -- but A " + "CONNECTED VIEWER DOES. About once a minute the broker stamps `touch` on " + "any session somebody is watching, which is the one exception to " + "`session_touch` being the only mark of activity. An enabled stream with " + "nobody watching captures nothing and keeps nothing alive." +) + COORDINATES_NOTE = ( "Coordinates are OUTPUT-NORMALISED 0..1 inside the session's own " "container: (0,0) is its top-left corner and (1,1) its bottom-right. They " @@ -516,15 +529,58 @@ def _acquire_summary(outcome: Outcome, payload: dict[str, Any] | None) -> str: ) -def _start_summary(outcome: Outcome, payload: dict[str, Any] | None) -> str: - session_id = (payload or {}).get("id") or "(not reported)" +def _stream_summary(outcome: Outcome, payload: dict[str, Any] | None) -> str: + """The URL on the first line: it is the whole point of the call.""" + + stream = (payload or {}).get("stream") or {} + url = stream.get("url") + if not url: + return ( + "the broker recorded this stream but the daemon did not confirm " + "it is attached, so there is no URL to hand out yet; read " + "`hhstream://streams` and retry if it stays that way." + f"\n\n{outcome.stdout.strip()}" + ) return ( - f"session id: {session_id}\n" - f"Keep it alive with `session_touch` before its idle timeout expires.\n\n" + f"{url}\n" + "Hand that URL to whoever is watching. Its phrase is the only thing " + "protecting it, so do not put it anywhere this session's own screen " + "can see. Somebody watching counts as activity; `session_stream_" + "disable` stops it.\n\n" f"{outcome.stdout.strip()}" ) +def _start_summary(outcome: Outcome, payload: dict[str, Any] | None) -> str: + """The id first, and the stream's URL beside it when one was asked for. + + Both on the first lines, because both are what the next step needs: the id + to drive the session, the URL to hand to a person. + """ + + session = payload or {} + session_id = session.get("id") or "(not reported)" + lines = [f"session id: {session_id}"] + stream = session.get("stream") or {} + if stream.get("url"): + lines.append(f"stream url: {stream['url']}") + lines.append( + "That URL's phrase is its only protection -- treat it as a " + "password. Somebody watching it counts as activity." + ) + elif session.get("stream_error"): + # Loudly, because the call succeeded: a caller that asked for a stream + # must not read a session id as one. + lines.append( + f"stream: NOT enabled -- {session['stream_error']}. The session is " + "running; retry with `session_stream_enable`." + ) + lines.append( + "Keep it alive with `session_touch` before its idle timeout expires." + ) + return "\n".join(lines) + f"\n\n{outcome.stdout.strip()}" + + def build() -> tuple[Tool, ...]: """The whole surface. A function, so the table is built once per process.""" @@ -712,6 +768,33 @@ def _session_tools() -> list[Tool]: "the session ends. Mutually exclusive with `home`." ), }, + "stream": { + "type": "boolean", + "description": ( + "also enable a live stream for this session, and " + "return its URL in the reply's `stream.url`. One " + "call rather than start-then-enable, because a " + "stream needs the container output that starting " + "creates. A stream that cannot be opened does NOT " + "fail the start -- the application is on the " + "screen, and `stream_error` says what went wrong." + ), + }, + "stream_phrase": { + "type": "string", + "description": ( + "use this phrase instead of a minted one: four " + "words of at least three letters and a " + "five-character tail, e.g. " + "`amber-otter-lantern-harbor-7k2qp`. Held to " + "exactly that shape, because it is a password." + ), + }, + "stream_fps": { + "type": "integer", + "minimum": 1, + "description": "frames per second for the stream (default: the broker's)", + }, "request_id": REQUEST_ID, }, ["app"], @@ -728,6 +811,9 @@ def _session_tools() -> list[Tool]: *_optional(a, "wait_ready", "wait-ready"), *_optional(a, "home"), *_switch(a, "ephemeral_home", "ephemeral-home"), + *_switch(a, "stream"), + *_optional(a, "stream_phrase", "stream-phrase"), + *_optional(a, "stream_fps", "stream-fps"), *_optional(a, "request_id", "request-id"), ], positionals=lambda a: [a["app"], *[str(x) for x in a.get("argv") or []]], @@ -872,6 +958,73 @@ def _session_tools() -> list[Tool]: positionals=lambda a: [a["id"]] if a.get("id") else [], ), ), + Tool( + name="session_stream_enable", + title="Publish a session's screen", + description=( + "Switch on a live view of this session's screen, in a browser, " + "at a URL whose path is a spoken phrase -- and return that " + "URL. This is how a person watches what an agent is doing.\n\n" + "It publishes the session's own container output, never the " + "main one: that output carries every other session, and " + "putting it behind one session's phrase is the mistake this " + "refuses to make (exit 10 with no container).\n\n" + "Re-enabling an already-enabled session returns the same URL. " + "A `phrase` that differs from the stored one is exit 8 rather " + "than a silent rotation, because rotating under somebody who " + "is holding the old URL is the one outcome neither answer can " + "be mistaken for. Disable then enable to rotate it.\n\n" + "Capture runs only while somebody is connected, so an enabled " + "stream nobody is watching costs nothing.\n\n" + f"{STREAM_NOTE}" + ), + schema=obj( + { + "id": SESSION_ID, + "phrase": { + "type": "string", + "description": ( + "use this phrase instead of a minted one: four " + "words of at least three letters and a " + "five-character tail. Held to exactly that shape, " + "because it is this stream's only credential." + ), + }, + "fps": { + "type": "integer", + "minimum": 1, + "description": "frames per second (default: the broker's; the daemon caps it)", + }, + }, + ["id"], + ), + invoke=json_verb( + ["session", "stream", "enable"], + options=lambda a: [*_optional(a, "phrase"), *_optional(a, "fps")], + positionals=lambda a: [a["id"]], + summary=_stream_summary, + ), + ), + Tool( + name="session_stream_disable", + title="Stop publishing a session's screen", + description=( + "Close the capture, drop the route, and forget the phrase. " + "Anyone holding the old URL loses it, which is the point: " + "this is also how a phrase is rotated, since the next enable " + "mints a new one.\n\n" + "Disabling a stream the daemon has already forgotten is not an " + "error -- that is the ordinary case after the daemon restarts, " + "and the record is cleared either way. `detached` says which " + "happened.\n\n" + f"{ACTIVITY_NOTE}" + ), + schema=obj({"id": SESSION_ID}, ["id"]), + invoke=json_verb( + ["session", "stream", "disable"], positionals=lambda a: [a["id"]] + ), + destructive=True, + ), ] @@ -1214,6 +1367,8 @@ TOOLSETS: dict[str, tuple[str, ...]] = { "session_adopt", "session_container", "session_reconcile", + "session_stream_enable", + "session_stream_disable", ), "pointer": ( "pointer_acquire", From b4abd085ccda6113f69c2b584dee6b4987616bec Mon Sep 17 00:00:00 2001 From: Hermenix Date: Sun, 11 Oct 2026 22:49:18 +0000 Subject: [PATCH 07/12] test(stream): both ends of live.sock, the broker's verbs, and a real viewer in a guest Three layers, because the stream has three and each one can be wrong on its own. `tests/live/test_live_control.py` (20 tests) runs `Control` against a fake capture: the four ops, `attach` idempotent on the phrase, the same phrase aimed at another output refused, a malformed phrase refused *without the value in the message*, and `fps` defaulting to the daemon's when the broker sends none. `tests/sessions/unit/test_stream_verbs.py` (24 tests) runs the broker against a fake daemon, and its subjects are the decisions rather than the plumbing: no container output is a refusal and never a fallback to the main output; a re-`enable` returns the same phrase and a different one is exit 8; a down daemon turns `attached` false rather than failing a read; the sweep touches a watched session and leaves an unwatched one alone; teardown detaches before the container is destroyed. `tests/unit/test_mcp_*` grow the two tools, the two resources and the three `session_start` arguments, with the read-only half asserted *absent* as a tool by name -- re-adding one should be a failure, not a silent widening. `checks/vm-stream.nix` is the new guest check, and it exists for the two things none of the above can say: 1. **That `grim` captures a real compositor from inside a unit.** It reads actual JPEGs off `//stream.mjpg` with `checks/lib/mjpeg_read.py`, a hand-rolled multipart reader that shares no code with the server -- a parser that did could not tell you the boundary was wrong. It framed on SOI/EOI rather than `Content-Length` for the same reason. This is what caught the unit's missing `XDG_RUNTIME_DIR`: `frames: [25827, 25827, 25827]` now, ten capture failures and a stream that gave up before. 2. **That a connected viewer moves the idle clock.** That needs the broker's real sweep, on a real timer, with a client genuinely still there a minute later -- hence `--hold`, and hence a check that spends ~90 seconds twice. The negative control runs first, because if the sweep touched an unwatched stream the positive case would pass for the wrong reason. Measured in the guest: with nobody watching `580s -> 502s` remaining, with a viewer `499s -> 596s`. It also asserts the phrase is in neither the audit log nor a 404 body, that the old URL stops serving after `disable`, that `disable` then `enable` rotates it, that `kill` takes the stream down and leaves the daemon up, and that `start --stream` yields an id and a URL that serves frames. No clock is injected anywhere in this check. An injected one would prove the arithmetic, which the unit tests already do, and not the daemon. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_011ZeH65zd6pFTgBSu52W9PE --- checks/default.nix | 6 + checks/lib/mjpeg_read.py | 148 +++++++ checks/vm-stream.nix | 372 ++++++++++++++++ tests/live/test_live_control.py | 327 ++++++++++++++ tests/sessions/unit/test_exit_codes.py | 2 + tests/sessions/unit/test_registry.py | 7 +- tests/sessions/unit/test_stream_verbs.py | 519 +++++++++++++++++++++++ tests/unit/test_mcp_resources.py | 48 ++- tests/unit/test_mcp_server.py | 49 ++- tests/unit/test_mcp_tools.py | 155 ++++++- 10 files changed, 1626 insertions(+), 7 deletions(-) create mode 100755 checks/lib/mjpeg_read.py create mode 100644 checks/vm-stream.nix create mode 100644 tests/live/test_live_control.py create mode 100644 tests/sessions/unit/test_stream_verbs.py diff --git a/checks/default.nix b/checks/default.nix index ae71224..cab711d 100644 --- a/checks/default.nix +++ b/checks/default.nix @@ -2,6 +2,7 @@ # # unit -- pytest over tests/, against the nix/package.nix build # vm-mcp -- the MCP server over stdio, against a live broker +# vm-stream -- the per-session live stream, with a real HTTP viewer # vm-basic -- nixosTest with a fake capture backend: module + wire contract # vm-hyprland -- nixosTest with the real compositor and grim # vm-sessions -- nixosTest with the real compositor and the session broker @@ -80,4 +81,9 @@ in # JSON-RPC client the check writes itself. `sessionArgs` plus the one package # the others have no reason to carry. vm-mcp = pkgs.callPackage ./vm-mcp.nix (sessionArgs // { inherit hhstream-mcp; }); + + # The live stream: `stream enable`, a real HTTP viewer inside the guest, and + # the one thing no in-process test can say -- that a connected viewer moves + # the idle clock. Spends real time on purpose; see the file's header. + vm-stream = pkgs.callPackage ./vm-stream.nix sessionArgs; } diff --git a/checks/lib/mjpeg_read.py b/checks/lib/mjpeg_read.py new file mode 100755 index 0000000..c4056c0 --- /dev/null +++ b/checks/lib/mjpeg_read.py @@ -0,0 +1,148 @@ +#!/usr/bin/env python3 +"""``hhs-mjpeg-read`` -- read a real MJPEG stream, and optionally keep watching. + +What `checks/vm-stream.nix` uses as the *person with a phone*. Two jobs, and the +second is the one that matters: + +1. Read N frames off ``multipart/x-mixed-replace`` and report their sizes, so + "the stream works" is a statement about bytes a client actually received + rather than about a 200. +2. **Hold the connection open** for a while afterwards. The broker counts + connected viewers and treats one as activity, so proving that needs a viewer + that is still there a minute later -- a client that read three frames and + exited would make the sweep correct and untested. + +Standard library only, and a hand-rolled multipart reader rather than anything +that understands HTTP for us: the framing is this repository's own +(`hhstream_live.server`), and a parser that shared code with it could not tell +you the boundary was wrong. +""" + +from __future__ import annotations + +import argparse +import json +import socket +import sys +import time +import urllib.parse + +SOI = b"\xff\xd8" +EOI = b"\xff\xd9" + + +def connect(url: str, timeout: float) -> socket.socket: + parsed = urllib.parse.urlsplit(url) + host = parsed.hostname or "127.0.0.1" + port = parsed.port or 80 + path = parsed.path or "/" + sock = socket.create_connection((host, port), timeout=timeout) + request = ( + f"GET {path} HTTP/1.1\r\n" + f"Host: {host}:{port}\r\n" + "Accept: */*\r\n" + "Connection: close\r\n\r\n" + ) + sock.sendall(request.encode("ascii")) + return sock + + +def read_headers(sock: socket.socket) -> tuple[int, dict[str, str], bytes]: + buffer = b"" + while b"\r\n\r\n" not in buffer: + chunk = sock.recv(4096) + if not chunk: + raise SystemExit("the server closed the connection before any headers") + buffer += chunk + head, rest = buffer.split(b"\r\n\r\n", 1) + lines = head.decode("latin-1").split("\r\n") + status = int(lines[0].split()[1]) + headers = {} + for line in lines[1:]: + if ":" in line: + name, _, value = line.partition(":") + headers[name.strip().lower()] = value.strip() + return status, headers, rest + + +def frames(sock: socket.socket, rest: bytes, wanted: int, deadline: float) -> list[int]: + """Sizes of the JPEGs read off the multipart body, by SOI/EOI. + + Framed on the markers rather than on ``Content-Length``: a stream that sent + the right lengths and the wrong bytes would pass a length-based reader, and + what the check is about is whether a browser would paint this. + """ + + sizes: list[int] = [] + buffer = rest + while len(sizes) < wanted: + if time.monotonic() > deadline: + raise SystemExit( + f"only {len(sizes)} of {wanted} frames arrived before the deadline" + ) + start = buffer.find(SOI) + end = buffer.find(EOI, start + 2) if start != -1 else -1 + if start != -1 and end != -1: + sizes.append(end + 2 - start) + buffer = buffer[end + 2 :] + continue + try: + chunk = sock.recv(65536) + except TimeoutError: + raise SystemExit("the stream went quiet") from None + if not chunk: + raise SystemExit("the stream ended before enough frames arrived") + buffer += chunk + return sizes + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(prog="hhs-mjpeg-read", description=__doc__) + parser.add_argument("url", help="the stream.mjpg URL") + parser.add_argument("--frames", type=int, default=2, help="how many to read") + parser.add_argument( + "--hold", + type=float, + default=0.0, + metavar="SECONDS", + help="keep the connection open this long after reading, so the broker " + "counts a viewer for that whole time", + ) + parser.add_argument("--timeout", type=float, default=30.0) + args = parser.parse_args(argv) + + sock = connect(args.url, args.timeout) + sock.settimeout(args.timeout) + try: + status, headers, rest = read_headers(sock) + content_type = headers.get("content-type", "") + if status != 200: + print(json.dumps({"status": status}), flush=True) + raise SystemExit(f"the stream answered {status}, not 200") + if "multipart/x-mixed-replace" not in content_type: + raise SystemExit(f"content-type is {content_type!r}, not multipart") + sizes = frames(sock, rest, args.frames, time.monotonic() + args.timeout) + print( + json.dumps( + {"status": status, "content_type": content_type, "frames": sizes} + ), + flush=True, + ) + if args.hold > 0: + # Still connected, and still reading: a viewer that stopped reading + # would have its socket buffer fill and the server would see it go. + deadline = time.monotonic() + args.hold + while time.monotonic() < deadline: + try: + if not sock.recv(65536): + break + except TimeoutError: + continue + print(json.dumps({"held_for_s": args.hold}), flush=True) + finally: + sock.close() + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/checks/vm-stream.nix b/checks/vm-stream.nix new file mode 100644 index 0000000..42b1204 --- /dev/null +++ b/checks/vm-stream.nix @@ -0,0 +1,372 @@ +# checks..vm-stream -- the per-session live stream, end to end, with a +# real HTTP viewer inside the guest. +# +# The session machine plus `sessions.stream.enable = true`, which runs +# `hhstream-live` and hands the broker `/run/hhstream/live.sock`. Then: +# `hhstream session stream enable` on a live session, and a client that reads +# actual JPEGs off `//stream.mjpg`. +# +# WHAT THIS PROVES THAT THE UNIT TESTS CANNOT. +# +# `tests/live/test_live_control.py` and `tests/sessions/unit/test_stream_verbs.py` +# already run both ends of `live.sock` against each other with a fake capture. +# What they cannot say is that `grim` captures a real compositor from inside a +# *unit* -- which is the one failure this feature has already been bitten by +# once elsewhere in this tree (`docs/TODO/capture-binary-and-the-unit-path.md`: +# a unit's PATH has no `grim`, and the symptom is an empty frame area with no +# error). So the capture command is passed explicitly by the module and this +# check reads the bytes that come out of it. +# +# It also proves the one thing no in-process test can: that a *connected +# viewer* moves the idle clock. That needs the broker's real sweep, on a real +# timer, with a client that is genuinely still there a minute later -- which is +# why `hhs-mjpeg-read --hold` exists and why this check spends real time. +# +# THE TWO REAL-TIME WAITS ARE THE POINT OF THEIR SUBTESTS. The sweep runs once +# a minute (`hhstreamd.manager.STREAM_SWEEP_S`), and both the positive case and +# its negative control have to outlast one pass. No clock is injected anywhere: +# an injected one would prove the arithmetic, which the unit tests already do, +# and not the daemon. +# +# Habits copied from the other session checks: every wait is bounded, each +# subtest asserts once, and a failure dumps the compositor's diagnostics and +# the broker's journal before it propagates. +{ + testers, + hhstream, + hhsModule, + testTools, + hyprland, + foot, + jq, + grim, + procps, + coreutils, + dejavu_fonts, + width ? 1280, + height ? 720, + sessionUser ? "hermes", + port ? 9988, +}: +testers.runNixOSTest ( + { pkgs, ... }: + let + tools = pkgs.callPackage ./lib/session-tools.nix { inherit hhstream; }; + guest = import ./lib/session-machine.nix { + inherit + hhsModule + hhstream + hyprland + testTools + foot + jq + procps + grim + coreutils + dejavu_fonts + ; + sessionTools = tools; + }; + prelude = import ./lib/session-prelude.nix { + inherit hhstream coreutils sessionUser; + }; + + # The viewer. Plain python3 and a hand-rolled multipart reader, so the + # framing this repository serves is read by something that does not share + # code with it -- the same choice `tests/assert_stream.py` makes about the + # frame socket, for the same reason. + viewer = pkgs.runCommandLocal "hhs-mjpeg-read" { } '' + install -Dm755 ${./lib/mjpeg_read.py} $out/bin/hhs-mjpeg-read + sed -i '1s|.*|#!${pkgs.python3}/bin/python3|' $out/bin/hhs-mjpeg-read + $out/bin/hhs-mjpeg-read --help > /dev/null + ''; + in + { + name = "hhstream-vm-stream"; + + # Boot, plus two sweep windows of real waiting (~70 s each) and a session + # launch. Capped so a hang is a failure rather than a wedged builder. + globalTimeout = 2400; + + nodes.machine = { + imports = [ + (guest { + inherit width height sessionUser; + apps.foot = { + exec = "${foot}/bin/foot"; + argsAllow = [ ]; + }; + }) + ]; + + services.hyprland-headless-stream.sessions.stream = { + enable = true; + # Loopback: the viewer runs inside the guest. Reaching it from outside + # is `vm-stream-firewall`'s subject, and opening a port here would make + # two checks assert the same thing badly. + bind = "127.0.0.1"; + inherit port; + }; + + environment.systemPackages = [ viewer ]; + }; + + testScript = + prelude + + '' + # `machine.sleep` wants a `timedelta`: a bare number is deprecated as + # `secs=` and refused outright by the driver's type check as + # `duration=`. MEASURED, as `nix build` failing before the VM even + # booted: "Expected `timedelta | None`, found `Literal[75]`". + import datetime as dt + + READ = "${viewer}/bin/hhs-mjpeg-read" + PORT = ${toString port} + LIVE_SOCKET = "/run/hhstream/live.sock" + #: `hhstreamd.manager.STREAM_SWEEP_S`, plus room for one round trip. + SWEEP_WINDOW_S = 75 + + + def expires_in(session_id): + """The session's own idle deadline, as the broker reports it.""" + + return cli_json(f"status {session_id} --json")["session"]["expires_in_s"] + + + def stream_json(args=""): + return cli_json(f"stream {args} --json".strip()) + + + start_all() + bring_up("the compositor and broker") + + with subtest("the as-user wrapper propagates exit status"): + machine.succeed(as_user(TRUE), timeout=120) + machine.fail(as_user(FALSE), timeout=120) + + with subtest("the stream daemon came up and owns its socket, not the broker's"): + try: + machine.wait_for_unit("hhstream-live.service", timeout=180) + machine.wait_for_file(LIVE_SOCKET, timeout=120) + restarts = machine.succeed( + "systemctl show -p NRestarts --value hhstream-live.service" + ).strip() + assert restarts == "0", ( + f"hhstream-live has restarted {restarts} time(s): it is " + "crash-looping, not running" + ) + # The risk the plan names by name: a second RuntimeDirectory on + # /run/hhstream would have deleted the broker's socket. + machine.succeed(f"test -S {CONTROL}") + machine.succeed(f"test -S {LIVE_SOCKET}") + # And it is listening on the port it was given, with no stream + # on it: running the daemon turns nothing on. + machine.wait_until_succeeds( + f"ss -ltn | grep -q 127.0.0.1:{PORT}", timeout=60 + ) + assert stream_json("list")["streams"] == [] + except Exception: + dump("the stream daemon did not come up") + raise + + with subtest("a session with no stream reports none"): + session_id = machine.succeed(cli("start foot"), timeout=300).strip() + print(f"session {session_id}") + assert stream_json(f"status {session_id}")["stream"] is None + + with subtest("enabling mints a phrase and the daemon publishes the output"): + try: + url = machine.succeed( + cli(f"stream enable {session_id}"), timeout=180 + ).strip() + print(f"stream url: {url}") + assert url.startswith(f"http://127.0.0.1:{PORT}/"), url + + status = stream_json(f"status {session_id}")["stream"] + assert status["attached"] is True, status + assert status["output"] == f"hh-{session_id}", status + assert status["url"] == url, status + # The phrase is in the URL and in the record, and nowhere else. + assert status["phrase"] in url + except Exception: + dump("enabling the stream", session_id) + raise + + with subtest("the phrase is not in the audit log"): + # It is the stream's only credential, and the log is read by + # whoever can read logs (CONTRACT-sessions.md §3.5). + phrase = stream_json(f"status {session_id}")["stream"]["phrase"] + events = machine.succeed( + "cat /var/lib/hhstream/events.jsonl || true", timeout=60 + ) + assert "stream_enable" in events, "the enable was not audited at all" + assert phrase not in events, "the phrase reached the audit log" + + with subtest("a real client reads JPEGs off the stream"): + # The subtest the unit tests cannot write: `grim` capturing a real + # compositor, from inside a unit, through the module's own capture + # command. + try: + out = machine.succeed( + as_user(f"{READ} {url}stream.mjpg --frames 3"), timeout=180 + ) + first = json.loads(out.strip().splitlines()[0]) + print(f"viewer read: {first}") + assert first["status"] == 200, first + assert "multipart/x-mixed-replace" in first["content_type"], first + assert len(first["frames"]) == 3, first + # Real frames of a real desktop, not a 1x1 placeholder. + assert all(size > 1024 for size in first["frames"]), first["frames"] + except Exception: + dump("reading the stream", session_id) + raise + + with subtest("a wrong phrase is a 404 and is never echoed back"): + wrong = "wrong-words-entirely-here-00000" + status, body = machine.execute( + as_user(f"{READ} http://127.0.0.1:{PORT}/{wrong}/stream.mjpg --frames 1"), + timeout=120, + ) + assert status != 0, f"a wrong phrase was served: {body}" + assert wrong not in body, "the wrong phrase was echoed back" + + with subtest("an enabled stream nobody watches keeps nothing alive"): + # The negative control, and it has to come first: if the sweep + # touched an unwatched stream, the positive case below would pass + # for the wrong reason. + try: + assert stream_json("list")["streams"][0]["viewers"] == 0 + before = expires_in(session_id) + machine.sleep(duration=dt.timedelta(seconds=SWEEP_WINDOW_S)) + after = expires_in(session_id) + print(f"with no viewer: {before:.0f}s -> {after:.0f}s remaining") + assert after < before, ( + "the idle deadline did not move closer over a whole sweep " + "window with nobody watching, so something is touching this " + "session" + ) + except Exception: + dump("the unwatched stream", session_id) + raise + + with subtest("a connected viewer is activity and moves the idle deadline"): + # The one exception to "touch is the only mark of activity" + # (CONTRACT-sessions.md §4). The viewer holds its connection open + # across a sweep, because a client that read and left would make + # the sweep correct and untested. + try: + before = expires_in(session_id) + machine.succeed( + background( + "hhs-viewer", + f"{READ} {url}stream.mjpg --frames 2 " + f"--hold {SWEEP_WINDOW_S} --timeout {SWEEP_WINDOW_S + 60}", + user=USER, + ), + timeout=120, + ) + watching = wait_for( + lambda: stream_json("list")["streams"][0]["viewers"] > 0, + timeout=60, + what="the broker to see a connected viewer", + ) + assert watching + after = wait_for( + lambda: expires_in(session_id) > before, + timeout=SWEEP_WINDOW_S + 60, + what="the viewer sweep to reset the idle clock", + ) + print(f"with a viewer: {before:.0f}s -> {expires_in(session_id):.0f}s") + + events = machine.succeed("cat /var/lib/hhstream/events.jsonl") + assert "stream_viewer_activity" in events + assert stream_json(f"status {session_id}")["stream"]["phrase"] not in events + except Exception: + dump("the watched stream", session_id) + raise + + with subtest("re-enabling returns the same url and starts no second capture"): + again = machine.succeed(cli(f"stream enable {session_id}"), timeout=180).strip() + assert again == url, f"the phrase rotated: {again} != {url}" + assert len(stream_json("list")["streams"]) == 1 + + with subtest("a phrase that is not this session's is refused"): + # Of the generated SHAPE and not this session's phrase -- the two + # refusals are different exits, and a fixture that is merely + # malformed would take the shape path (exit 2) and never reach the + # broker. MEASURED: the first fixture here was + # `other-words-go-here-99999`, which fails on `go` being two + # letters -- the CLI refused it on shape and this subtest read + # exit 2 where it wanted 8. + status, out = machine.execute( + cli(f"stream enable {session_id} --phrase other-words-going-here-99999"), + timeout=120, + ) + # Exit 8 (BUSY): rotating under somebody holding the old URL is the + # one outcome neither answer can be mistaken for. + assert status == 8, f"expected exit 8, got {status}: {out}" + assert stream_json(f"status {session_id}")["stream"]["url"] == url + + with subtest("a malformed phrase is refused without echoing it"): + status, out = machine.execute( + cli(f"stream enable {session_id} --phrase hunter2"), timeout=120 + ) + assert status == 2, f"expected exit 2, got {status}: {out}" + assert "hunter2" not in out, "the refusal echoed the phrase back" + + with subtest("disabling stops the capture and forgets the phrase"): + machine.succeed(cli(f"stream disable {session_id}"), timeout=180) + assert stream_json(f"status {session_id}")["stream"] is None + assert stream_json("list")["streams"] == [] + status, _ = machine.execute( + as_user(f"{READ} {url}stream.mjpg --frames 1"), timeout=120 + ) + assert status != 0, "the old URL still serves after disable" + + with subtest("disable then enable rotates the phrase"): + rotated = machine.succeed(cli(f"stream enable {session_id}"), timeout=180).strip() + assert rotated != url, "the phrase did not rotate" + url = rotated + + with subtest("kill takes the stream down with the session"): + # The teardown *order* is unit-tested (a capture aimed at a removed + # output fails every time); what is asserted here is the outcome + # both halves have to agree on. + try: + machine.succeed(cli(f"kill {session_id}"), timeout=300) + assert stream_json("list")["streams"] == [] + status, _ = machine.execute( + as_user(f"{READ} {url}stream.mjpg --frames 1"), timeout=120 + ) + assert status != 0, "the stream outlived its session" + # And the daemon is still up: a session ending is not its + # problem. + machine.succeed("systemctl is-active hhstream-live.service") + except Exception: + dump("killing a streaming session", session_id) + raise + + with subtest("start --stream does it in one call"): + # The usual way an agent sets one up, and the reason `start` takes + # the flag at all: a stream needs the container output that + # starting creates. + try: + out = machine.succeed(cli("start foot --stream"), timeout=300) + lines = [line for line in out.strip().splitlines() if line] + streamed_id, streamed_url = lines[0], lines[1] + print(f"{streamed_id} -> {streamed_url}") + assert streamed_url.startswith(f"http://127.0.0.1:{PORT}/") + read = json.loads( + machine.succeed( + as_user(f"{READ} {streamed_url}stream.mjpg --frames 2"), + timeout=180, + ).strip().splitlines()[0] + ) + assert len(read["frames"]) == 2, read + machine.succeed(cli(f"kill {streamed_id}"), timeout=300) + except Exception: + dump("start --stream") + raise + ''; + } +) diff --git a/tests/live/test_live_control.py b/tests/live/test_live_control.py new file mode 100644 index 0000000..db0304f --- /dev/null +++ b/tests/live/test_live_control.py @@ -0,0 +1,327 @@ +"""``live.sock``: the four ops, over a real socket, against the shipped client. + +The server is `hhstream_live.control` and the client is +`hhstreamd.live.LiveStreams` — both shipped, talking over a real ``AF_UNIX`` +socket, so this is the one place the two halves of CONTRACT-sessions.md §3.5 +are read against each other. Only the capture is substituted, because this +machine has no compositor. + +The opposite choice from `tests/assert_stream.py`, and for the stated reason: +what is being asserted here is that *this* client reaches *this* daemon, which +a second implementation could say nothing about. The *wire format* is held +honest separately, by the raw-JSON tests at the bottom that bypass the client +entirely. +""" + +from __future__ import annotations + +import contextlib +import json +import socket +import threading +from pathlib import Path + +import pytest +from fake_jpeg import jpeg_bytes + +from hhstream_live import phrase as phrase_mod +from hhstream_live.control import Control, ControlServer +from hhstream_live.publisher import FPS_CAP, StreamRegistry +from hhstreamd.live import Attachment, LiveRejected, LiveStreams, LiveUnavailable + +PORT = 9988 + + +class FakeCapture: + def grab(self) -> bytes: + return jpeg_bytes() + + +class Daemon: + """The shipped control server, with a poll-interval accept loop.""" + + def __init__(self, socket_path: Path) -> None: + self.registry = StreamRegistry() + self.control = Control( + self.registry, + capture_for=lambda output: FakeCapture(), + url_for=lambda phrase: f"http://127.0.0.1:{PORT}/{phrase}/", + ) + self.server = ControlServer(socket_path, self.control) + self.server.listen() + self.connections: list[socket.socket] = [] + self._stop = threading.Event() + self._listener = self.server._sock + self._listener.settimeout(0.05) + self._thread = threading.Thread(target=self._serve, daemon=True) + self._thread.start() + + def _serve(self) -> None: + while not self._stop.is_set(): + try: + conn, _ = self._listener.accept() + except (TimeoutError, socket.timeout): + continue + except OSError: + return + self.connections.append(conn) + with conn: + self.server._serve_connection(conn) + + def stop(self) -> None: + self._stop.set() + for conn in self.connections: + with contextlib.suppress(OSError): + conn.shutdown(socket.SHUT_RDWR) + self._thread.join(timeout=2) + self.registry.stop_all() + self.server.close() + + +@pytest.fixture +def short_tmpdir(): + """A directory with a short path. + + Its own fixture rather than `tmp_path`, and the reason is the same one + `tests/sessions/conftest.py` has: an ``AF_UNIX`` address is capped near 108 + bytes and pytest's own paths are longer than that. Not imported from there + -- these two suites have separate conftests, and a cross-import between + them is how the collection collision in this tree started. + """ + + import shutil + import tempfile + + path = tempfile.mkdtemp(prefix="hhs") + try: + yield Path(path) + finally: + shutil.rmtree(path, ignore_errors=True) + + +@pytest.fixture +def daemon(short_tmpdir: Path): + daemon = Daemon(short_tmpdir / "live.sock") + try: + yield daemon + finally: + daemon.stop() + + +@pytest.fixture +def client(daemon): + client = LiveStreams(daemon.server.socket_path) + try: + yield client + finally: + client.close() + + +# ── attach ───────────────────────────────────────────────────────────────── + + +def test_attaching_publishes_an_output_and_returns_the_daemon_s_url(client, daemon): + phrase = phrase_mod.generate() + attachment = client.attach( + phrase=phrase, output="hh-x", fps=5, session="x", label="chromium" + ) + + assert isinstance(attachment, Attachment) + assert (attachment.output, attachment.fps, attachment.session) == ("hh-x", 5, "x") + assert attachment.viewers == 0 + # The URL is the DAEMON's: the broker never assembles one, because it does + # not own the port. + assert attachment.url == f"http://127.0.0.1:{PORT}/{phrase}/" + assert daemon.registry.get(phrase) is not None + + +def test_attaching_the_same_phrase_twice_starts_no_second_capture(client, daemon): + phrase = phrase_mod.generate() + first = client.attach(phrase=phrase, output="hh-x", fps=5, session="x") + again = client.attach(phrase=phrase, output="hh-x", fps=5, session="x") + + assert again.url == first.url + assert len(daemon.registry.all()) == 1 + + +def test_the_same_phrase_aimed_somewhere_else_is_refused(client, daemon): + """A phrase is a credential for one view; re-pointing it silently would + show somebody a desktop they were not given.""" + + phrase = phrase_mod.generate() + client.attach(phrase=phrase, output="hh-x", fps=5) + + with pytest.raises(LiveRejected) as caught: + client.attach(phrase=phrase, output="hh-other", fps=5) + assert caught.value.kind == "invalid_argument" + assert daemon.registry.get(phrase).spec.output == "hh-x" + + +def test_a_phrase_that_is_not_of_the_shape_is_refused_without_echoing_it(client): + """The refusal crosses a socket into the broker's log. It is a password.""" + + with pytest.raises(LiveRejected) as caught: + client.attach(phrase="hunter2", output="hh-x", fps=5) + + assert caught.value.kind == "invalid_argument" + assert "hunter2" not in str(caught.value) + + +def test_a_rate_over_the_cap_is_refused_by_the_process_that_must_keep_up(client): + with pytest.raises(LiveRejected) as caught: + client.attach(phrase=phrase_mod.generate(), output="hh-x", fps=FPS_CAP + 1) + assert caught.value.kind == "invalid_argument" + assert str(FPS_CAP) in str(caught.value) + + +@pytest.mark.parametrize("fps", [0, -1]) +def test_a_nonsensical_rate_is_refused(client, fps): + with pytest.raises(LiveRejected): + client.attach(phrase=phrase_mod.generate(), output="hh-x", fps=fps) + + +def test_an_empty_output_is_refused(client): + with pytest.raises(LiveRejected) as caught: + client.attach(phrase=phrase_mod.generate(), output=" ", fps=5) + assert caught.value.kind == "invalid_argument" + + +# ── detach, list, viewers ────────────────────────────────────────────────── + + +def test_detaching_stops_the_capture_and_says_it_did(client, daemon): + phrase = phrase_mod.generate() + client.attach(phrase=phrase, output="hh-x", fps=5) + + assert client.detach(phrase) is True + assert daemon.registry.get(phrase) is None + + +def test_detaching_something_unattached_is_false_and_not_an_error(client): + """`disable` after the daemon restarted is the ordinary case.""" + + assert client.detach(phrase_mod.generate()) is False + + +def test_listing_returns_every_stream(client): + phrases = [phrase_mod.generate() for _ in range(3)] + for index, phrase in enumerate(phrases): + client.attach(phrase=phrase, output=f"hh-{index}", fps=5, session=f"s-{index}") + + listed = client.list() + assert {a.output for a in listed} == {"hh-0", "hh-1", "hh-2"} + assert all(isinstance(a, Attachment) for a in listed) + + +def test_viewers_reports_a_count_per_phrase(client, daemon): + phrase = phrase_mod.generate() + client.attach(phrase=phrase, output="hh-x", fps=5) + assert client.viewers() == {phrase: 0} + + daemon.registry.get(phrase).add_viewer() + daemon.registry.get(phrase).add_viewer() + assert client.viewers() == {phrase: 2} + + +def test_an_empty_daemon_lists_nothing_rather_than_failing(client): + assert client.list() == [] + assert client.viewers() == {} + + +# ── failure ──────────────────────────────────────────────────────────────── + + +def test_a_socket_that_is_not_there_is_unavailable(short_tmpdir: Path): + client = LiveStreams(short_tmpdir / "nothing-here.sock") + with pytest.raises(LiveUnavailable): + client.list() + + +def test_a_daemon_that_went_away_is_unavailable_and_not_a_stale_answer(client, daemon): + client.attach(phrase=phrase_mod.generate(), output="hh-x", fps=5) + assert client.list() # the connection is now held + daemon.stop() + + with pytest.raises(LiveUnavailable): + client.list() + + +def test_the_client_reconnects_after_the_daemon_restarts(short_tmpdir: Path): + """The held connection is checked for EOF before each write, as the + driver's is -- so a restart costs one reconnect and not a dead client.""" + + path = short_tmpdir / "live.sock" + first = Daemon(path) + client = LiveStreams(path) + try: + client.attach(phrase=phrase_mod.generate(), output="hh-x", fps=5) + first.stop() + second = Daemon(path) + try: + # The new daemon knows nothing, which is the point: the client + # reached it at all. + assert client.list() == [] + finally: + second.stop() + finally: + client.close() + + +# ── the wire format itself ───────────────────────────────────────────────── +# +# These bypass `LiveStreams` on purpose. The tests above prove the two shipped +# halves agree with each other; these prove the *format* §3.5 freezes, which a +# pair that agreed on something else would also pass. + + +def raw(path: Path, message: dict) -> dict: + with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as sock: + sock.settimeout(5) + sock.connect(str(path)) + with sock.makefile("rwb") as stream: + stream.write(json.dumps(message).encode() + b"\n") + stream.flush() + return json.loads(stream.readline()) + + +def test_the_frozen_op_names_and_reply_shape(daemon): + path = daemon.server.socket_path + phrase = phrase_mod.generate() + + attached = raw(path, {"op": "attach", "phrase": phrase, "output": "hh-x", "fps": 5}) + assert attached["ok"] is True + stream = attached["stream"] + assert set(stream) >= {"phrase", "output", "fps", "session", "url", "viewers"} + + assert raw(path, {"op": "list"})["streams"][0]["output"] == "hh-x" + assert raw(path, {"op": "viewers"})["viewers"] == {phrase: 0} + assert raw(path, {"op": "detach", "phrase": phrase}) == {"ok": True, "detached": True} + + +def test_an_unknown_op_is_refused_with_a_kind(daemon): + reply = raw(daemon.server.socket_path, {"op": "levitate"}) + assert reply["ok"] is False + assert reply["kind"] == "invalid_argument" + + +def test_malformed_json_is_refused_without_closing_the_connection(daemon): + path = daemon.server.socket_path + with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as sock: + sock.settimeout(5) + sock.connect(str(path)) + with sock.makefile("rwb") as stream: + stream.write(b"{not json\n") + stream.flush() + first = json.loads(stream.readline()) + assert first["ok"] is False + # Still answering: one bad line must not cost the broker its + # connection, because the next op may be a `detach` in a teardown. + stream.write(json.dumps({"op": "list"}).encode() + b"\n") + stream.flush() + assert json.loads(stream.readline())["ok"] is True + + +def test_a_request_that_is_not_an_object_is_refused(daemon): + reply = raw(daemon.server.socket_path, ["attach"]) # type: ignore[arg-type] + assert reply["ok"] is False + assert reply["kind"] == "invalid_argument" diff --git a/tests/sessions/unit/test_exit_codes.py b/tests/sessions/unit/test_exit_codes.py index fd576f7..93c4061 100644 --- a/tests/sessions/unit/test_exit_codes.py +++ b/tests/sessions/unit/test_exit_codes.py @@ -93,6 +93,8 @@ def test_the_table_has_no_entries_this_test_does_not_know_about(): "readiness_timeout", "busy", "already_managed", "pointer_busy", "container_budget", "no_lease", "compositor", "compositor_unavailable", "no_container", "driver_unavailable", "driver_refused", + # v4, the stream. No new exit *numbers*: 8, 6 and 10 respectively. + "stream_busy", "stream_budget", "stream_unavailable", } assert set(EXIT_CODE_FOR_ERROR) == documented diff --git a/tests/sessions/unit/test_registry.py b/tests/sessions/unit/test_registry.py index 0faa63d..e21c51c 100644 --- a/tests/sessions/unit/test_registry.py +++ b/tests/sessions/unit/test_registry.py @@ -11,6 +11,7 @@ import pytest from hhstreamd.clock import ManualClock from hhstreamd.registry import ( + SCHEMA, ContainerState, Registry, Session, @@ -271,7 +272,7 @@ def test_derived_fields_reported_to_callers(registry: Registry): assert public["idle_s"] == 600.0 assert public["expires_in_s"] == 10800.0 - 600.0 assert public["live"] is True - assert public["schema"] == 3 + assert public["schema"] == SCHEMA # --------------------------------------------------------------------------- @@ -350,7 +351,7 @@ def test_a_schema_two_record_is_adopted_and_not_rejected(tmp_path: Path): assert quarantined == [] assert restored.id == "foot-aaaaaa" - assert restored.schema == 3 + assert restored.schema == SCHEMA # The honest value for everything schema 3 added: the daemon that wrote # this record knew nothing of containers, so there is nothing to claim. assert (restored.container, restored.pointer, restored.signature) == (None, None, None) @@ -362,7 +363,7 @@ def test_migrating_never_overwrites_a_field_that_was_already_there(): migrated = migrate({"schema": 2, "id": "a", "container": {"state": "owned"}}) assert migrated["container"] == {"state": "owned"} - assert migrated["schema"] == 3 + assert migrated["schema"] == SCHEMA def test_a_record_from_a_newer_schema_is_read_not_guessed_at(caplog): diff --git a/tests/sessions/unit/test_stream_verbs.py b/tests/sessions/unit/test_stream_verbs.py new file mode 100644 index 0000000..ad70784 --- /dev/null +++ b/tests/sessions/unit/test_stream_verbs.py @@ -0,0 +1,519 @@ +"""The stream verbs: the phrase, the record, the teardown order and the sweep. + +Both ends are real. The broker end is the manager, its `LiveStreams` client +and its registry; the daemon end is the shipped `Control` and `ControlServer` +over a real ``AF_UNIX`` socket. Only two things are substituted: the clock, +because the viewer sweep's effect on an idle deadline has to be testable +without waiting a minute, and the *capture*, because this machine has no +compositor — a `grim` here would fail for a reason none of these tests are +about. + +What that leaves real is everything the feature actually turns on: that a +phrase minted by the broker is one the daemon accepts, that it survives a +re-enable, that a stream is detached before its output is removed, and that a +connected viewer moves the idle clock when nothing else does. +""" + +from __future__ import annotations + +import asyncio +import contextlib +import socket +import threading +from pathlib import Path + +import pytest +from fakes import check_path + +from hhstream_live import phrase as phrase_mod +from hhstream_live.control import Control, ControlServer +from hhstream_live.publisher import StreamRegistry +from hhstreamd.clock import ManualClock +from hhstreamd.manager import ManagerConfig, ManagerError, SessionManager +from hhstreamd.policy import Policy +from hhstreamd.protocol import ExitCode, exit_code_for +from hhstreamd.registry import ContainerState, Session, State, container_record + +#: A JPEG the publisher will accept without a compositor. One pixel, enough +#: for `Stream` to have captured something. +ONE_PIXEL_JPEG = b"\xff\xd8\xff\xd9" + + +def make_session(session_id: str = "foot-aaaaaa", *, container: bool = True) -> Session: + return Session( + id=session_id, + app="foot", + argv=["/usr/bin/foot"], + pid=4242, + pgid=4242, + sid=4242, + pid_starttime=918342, + cmdline_sha256="9f2c", + home="/home/someone", + runtime_dir=f"/run/hhstream/sessions/{session_id}", + created_at="2026-10-01T09:14:02Z", + created_boottime=1000.0, + last_activity_at="2026-10-01T09:14:02Z", + last_activity_boottime=1000.0, + idle_timeout_s=10800.0, + state=State.READY, + container=( + container_record( + output=f"hh-{session_id}", + workspace=1000, + mode=(1280, 720), + created_by="broker", + state=ContainerState.OWNED, + ) + if container + else None + ), + ) + + +class FakeCapture: + """A capture with no compositor. Counts grabs so a test can see one.""" + + def __init__(self) -> None: + self.grabs = 0 + + def grab(self) -> bytes: + self.grabs += 1 + return ONE_PIXEL_JPEG + + +class FakeLiveDaemon: + """The shipped `Control` and protocol loop, with a poll-interval accept. + + The accept loop is the one thing not shipped, for `FakeDriver`'s reason: + ``serve_forever`` blocks in ``accept()`` by design and closing a listening + socket does not reliably wake a thread already inside it, so a fake that + used it would cost every test the stop timeout. + """ + + PORT = 9988 + + def __init__(self, socket_path: Path) -> None: + check_path(Path(socket_path)) + self.registry = StreamRegistry() + self.captures: dict[str, FakeCapture] = {} + self.control = Control( + self.registry, + capture_for=self._capture_for, + url_for=lambda phrase: f"http://127.0.0.1:{self.PORT}/{phrase}/", + ) + self.server = ControlServer(Path(socket_path), self.control) + self.server.listen() + self.socket_path = Path(socket_path) + self.connections: list[socket.socket] = [] + self._stop = threading.Event() + self._listener = self.server._sock + self._listener.settimeout(0.05) + self._thread = threading.Thread(target=self._serve, daemon=True) + self._thread.start() + + def _capture_for(self, output: str) -> FakeCapture: + capture = FakeCapture() + self.captures[output] = capture + return capture + + def _serve(self) -> None: + while not self._stop.is_set(): + try: + conn, _ = self._listener.accept() + except (TimeoutError, socket.timeout): + continue + except OSError: + return + self.connections.append(conn) + with conn: + self.server._serve_connection(conn) + + def watch(self, phrase: str, viewers: int = 1) -> None: + """Pretend ``viewers`` browsers are connected to ``phrase``.""" + + stream = self.registry.get(phrase) + assert stream is not None, "nothing is published under that phrase" + for _ in range(viewers): + stream.add_viewer() + + def unwatch(self, phrase: str) -> None: + stream = self.registry.get(phrase) + assert stream is not None + while stream.viewers: + stream.remove_viewer() + + def outputs(self) -> list[str]: + return [s.spec.output for s in self.registry.all()] + + def stop(self) -> None: + """Stop answering, for real. + + Shutting the *accepted* connections down matters and is easy to get + wrong: the broker holds one open between ops, so a stop that only + closed the listener would leave a thread still sitting in `readline` + and still answering `list` -- and the "the daemon is down" tests would + pass against a daemon that was up. + """ + + self._stop.set() + for conn in self.connections: + with contextlib.suppress(OSError): + conn.shutdown(socket.SHUT_RDWR) + self._thread.join(timeout=2) + self.registry.stop_all() + self.server.close() + + +class Harness: + def __init__(self, manager: SessionManager, daemon: FakeLiveDaemon, clock: ManualClock): + self.manager = manager + self.daemon = daemon + self.clock = clock + + def add(self, session_id: str = "foot-aaaaaa", **kwargs) -> Session: + return self.manager.registry.add(make_session(session_id, **kwargs)) + + def run(self, coro): + return asyncio.run(coro) + + +@pytest.fixture +def harness(short_tmpdir: Path): + daemon = FakeLiveDaemon(short_tmpdir / "live.sock") + clock = ManualClock(boottime=1000.0) + manager = SessionManager( + ManagerConfig( + state_dir=short_tmpdir / "state", + runtime_dir=short_tmpdir / "run", + policy=Policy(), + live_socket=daemon.socket_path, + ), + clock=clock, + ) + try: + yield Harness(manager, daemon, clock) + finally: + if manager.live is not None: + manager.live.close() + daemon.stop() + + +# ── enable ───────────────────────────────────────────────────────────────── + + +def test_enabling_mints_a_phrase_the_daemon_accepts_and_publishes(harness): + harness.add() + reply = harness.run(harness.manager.stream_enable("foot-aaaaaa")) + + stream = reply["stream"] + assert stream["enabled"] is True + assert stream["attached"] is True + assert stream["output"] == "hh-foot-aaaaaa" + # The phrase the broker minted is one the daemon's own validator took. + assert phrase_mod.validate(stream["phrase"]) == stream["phrase"] + assert stream["url"].endswith(f"/{stream['phrase']}/") + # ...and the daemon really is publishing that output. + assert harness.daemon.outputs() == ["hh-foot-aaaaaa"] + + +def test_the_phrase_is_in_the_record_so_it_survives_a_restart(harness): + harness.add() + first = harness.run(harness.manager.stream_enable("foot-aaaaaa"))["stream"] + + stored = harness.manager.registry.get("foot-aaaaaa").stream + assert stored["phrase"] == first["phrase"] + # And it is on disk, not just in memory: a URL somebody was read out to + # must not change because the broker bounced. + records, broken = harness.manager.registry.load_records() + assert broken == [] + on_disk = {record.id: record for record in records} + assert on_disk["foot-aaaaaa"].stream["phrase"] == first["phrase"] + + +def test_re_enabling_returns_the_same_phrase_and_starts_no_second_capture(harness): + harness.add() + first = harness.run(harness.manager.stream_enable("foot-aaaaaa"))["stream"] + again = harness.run(harness.manager.stream_enable("foot-aaaaaa"))["stream"] + + assert again["phrase"] == first["phrase"] + assert again["url"] == first["url"] + # One stream on the daemon, not two: a re-enable after a restart must not + # double the grim rate on one output. + assert len(harness.daemon.registry.all()) == 1 + + +def test_a_phrase_that_is_not_this_session_s_is_refused_not_rotated(harness): + harness.add() + harness.run(harness.manager.stream_enable("foot-aaaaaa")) + mine = harness.manager.registry.get("foot-aaaaaa").stream["phrase"] + + with pytest.raises(ManagerError) as caught: + harness.run( + harness.manager.stream_enable("foot-aaaaaa", phrase=phrase_mod.generate()) + ) + assert caught.value.error == "stream_busy" + assert exit_code_for(caught.value.error) == ExitCode.BUSY + # The stored one is untouched: somebody may be holding its URL. + assert harness.manager.registry.get("foot-aaaaaa").stream["phrase"] == mine + + +def test_disable_then_enable_rotates_the_phrase(harness): + harness.add() + first = harness.run(harness.manager.stream_enable("foot-aaaaaa"))["stream"]["phrase"] + harness.run(harness.manager.stream_disable("foot-aaaaaa")) + second = harness.run(harness.manager.stream_enable("foot-aaaaaa"))["stream"]["phrase"] + + assert second != first + assert harness.daemon.registry.get(first) is None + + +def test_a_caller_s_own_phrase_is_held_to_the_generated_shape(harness): + harness.add() + with pytest.raises(ManagerError) as caught: + harness.run(harness.manager.stream_enable("foot-aaaaaa", phrase="a-b-c")) + + assert caught.value.error == "invalid_argument" + assert exit_code_for(caught.value.error) == ExitCode.USAGE + # The refusal must not echo the phrase: this reason is logged. + assert "a-b-c" not in caught.value.reason + + +def test_a_session_with_no_container_is_refused_and_never_falls_back(harness): + """The main output carries every other session. This is the one that matters.""" + + harness.add(container=False) + with pytest.raises(ManagerError) as caught: + harness.run(harness.manager.stream_enable("foot-aaaaaa")) + + assert caught.value.error == "stream_unavailable" + assert exit_code_for(caught.value.error) == ExitCode.COMPOSITOR + assert harness.daemon.outputs() == [] + + +def test_a_bad_rate_is_a_usage_error_from_the_broker(harness): + harness.add() + for bad in (0, -1): + with pytest.raises(ManagerError) as caught: + harness.run(harness.manager.stream_enable("foot-aaaaaa", fps=bad)) + assert caught.value.error == "invalid_argument" + + +def test_a_rate_over_the_daemon_s_cap_is_refused_by_the_daemon(harness): + """The cap belongs to the process that has to keep up with it.""" + + harness.add() + with pytest.raises(ManagerError) as caught: + harness.run(harness.manager.stream_enable("foot-aaaaaa", fps=1000)) + + # The daemon's `invalid_argument` kind, mapped to exit 2 by the broker. + assert caught.value.error == "invalid_argument" + assert harness.daemon.outputs() == [] + + +def test_no_daemon_configured_is_unavailable_and_not_a_dead_session(harness, short_tmpdir): + manager = SessionManager( + ManagerConfig( + state_dir=short_tmpdir / "s2", + runtime_dir=short_tmpdir / "r2", + policy=Policy(), + live_socket=short_tmpdir / "nothing-here.sock", + ), + clock=ManualClock(boottime=1000.0), + ) + manager.registry.add(make_session()) + + with pytest.raises(ManagerError) as caught: + asyncio.run(manager.stream_enable("foot-aaaaaa")) + assert caught.value.error == "stream_unavailable" + # Exit 10, which the table defines as "the session is NOT lost". + assert exit_code_for(caught.value.error) == ExitCode.COMPOSITOR + assert manager.registry.get("foot-aaaaaa").live + + +# ── disable ──────────────────────────────────────────────────────────────── + + +def test_disabling_stops_the_capture_and_clears_the_record(harness): + harness.add() + harness.run(harness.manager.stream_enable("foot-aaaaaa")) + reply = harness.run(harness.manager.stream_disable("foot-aaaaaa")) + + assert reply["detached"] is True + assert reply["stream"] is None + assert harness.manager.registry.get("foot-aaaaaa").stream is None + assert harness.daemon.outputs() == [] + + +def test_disabling_a_stream_the_daemon_forgot_is_not_an_error(harness): + """The ordinary case after the daemon restarts; the record still clears.""" + + harness.add() + stream = harness.run(harness.manager.stream_enable("foot-aaaaaa"))["stream"] + harness.daemon.registry.remove(stream["phrase"]) # as a restart would + + reply = harness.run(harness.manager.stream_disable("foot-aaaaaa")) + assert reply["detached"] is False + assert harness.manager.registry.get("foot-aaaaaa").stream is None + + +def test_disabling_a_session_with_no_stream_says_so(harness): + harness.add() + with pytest.raises(ManagerError) as caught: + harness.run(harness.manager.stream_disable("foot-aaaaaa")) + assert caught.value.error == "stream_unavailable" + + +# ── reading ──────────────────────────────────────────────────────────────── + + +def test_the_listing_merges_the_daemon_s_view_into_the_records(harness): + harness.add("foot-aaaaaa") + harness.add("foot-bbbbbb") + harness.run(harness.manager.stream_enable("foot-aaaaaa")) + phrase = harness.manager.registry.get("foot-aaaaaa").stream["phrase"] + harness.daemon.watch(phrase, viewers=3) + + reply = harness.manager.stream_list() + assert reply["available"] is True + assert [row["id"] for row in reply["streams"]] == ["foot-aaaaaa"] + assert reply["streams"][0]["viewers"] == 3 + assert reply["streams"][0]["attached"] is True + + +def test_a_down_daemon_turns_attached_false_rather_than_failing_the_read(harness): + harness.add() + harness.run(harness.manager.stream_enable("foot-aaaaaa")) + harness.daemon.stop() + + reply = harness.manager.stream_list() + assert reply["available"] is False + row = reply["streams"][0] + assert row["attached"] is False + # No stale URL and no stale count: absent is the honest answer. + assert "url" not in row + assert "viewers" not in row + # The phrase is still there, which is what makes a re-enable the same URL. + assert row["phrase"] + + +def test_status_of_a_session_with_no_stream_is_a_null_not_an_error(harness): + harness.add() + assert harness.manager.stream_status("foot-aaaaaa")["stream"] is None + + +def test_status_with_no_id_is_the_listing(harness): + harness.add() + assert "streams" in harness.manager.stream_status(None) + + +# ── the idle clock ───────────────────────────────────────────────────────── + + +def test_enabling_and_reading_do_not_touch_the_idle_clock(harness): + session = harness.add() + before = session.last_activity_boottime + harness.clock.advance(300.0) + + harness.run(harness.manager.stream_enable("foot-aaaaaa")) + harness.manager.stream_list() + harness.manager.stream_status("foot-aaaaaa") + + assert harness.manager.registry.get("foot-aaaaaa").last_activity_boottime == before + + +def test_a_connected_viewer_is_activity_and_moves_the_deadline(harness): + """The one exception to "`touch` is the only mark of activity".""" + + session = harness.add() + before = session.last_activity_boottime + harness.run(harness.manager.stream_enable("foot-aaaaaa")) + phrase = harness.manager.registry.get("foot-aaaaaa").stream["phrase"] + harness.daemon.watch(phrase) + harness.clock.advance(600.0) + + touched = harness.run(harness.manager._touch_watched()) + + assert touched == ["foot-aaaaaa"] + after = harness.manager.registry.get("foot-aaaaaa").last_activity_boottime + assert after == before + 600.0 + + +def test_an_enabled_stream_nobody_watches_keeps_nothing_alive(harness): + """Otherwise a forgotten `enable` would be an immortal session.""" + + session = harness.add() + before = session.last_activity_boottime + harness.run(harness.manager.stream_enable("foot-aaaaaa")) + harness.clock.advance(600.0) + + assert harness.run(harness.manager._touch_watched()) == [] + assert harness.manager.registry.get("foot-aaaaaa").last_activity_boottime == before + + +def test_the_sweep_touches_only_the_sessions_being_watched(harness): + harness.add("foot-aaaaaa") + harness.add("foot-bbbbbb") + for session_id in ("foot-aaaaaa", "foot-bbbbbb"): + harness.run(harness.manager.stream_enable(session_id)) + watched = harness.manager.registry.get("foot-bbbbbb").stream["phrase"] + harness.daemon.watch(watched) + + assert harness.run(harness.manager._touch_watched()) == ["foot-bbbbbb"] + + +def test_the_sweep_survives_a_daemon_that_is_not_there(harness): + harness.add() + harness.run(harness.manager.stream_enable("foot-aaaaaa")) + harness.daemon.stop() + + assert harness.run(harness.manager._touch_watched()) == [] + + +def test_the_viewer_audit_entry_never_carries_the_phrase(harness): + """The audit log is read by whoever can read logs, and this is a password.""" + + harness.add() + harness.run(harness.manager.stream_enable("foot-aaaaaa")) + phrase = harness.manager.registry.get("foot-aaaaaa").stream["phrase"] + harness.daemon.watch(phrase) + harness.run(harness.manager._touch_watched()) + + written = (Path(harness.manager.config.state_dir) / "events.jsonl").read_text() + assert "stream_viewer_activity" in written + assert "stream_enable" in written + assert phrase not in written + + +# ── teardown order ───────────────────────────────────────────────────────── + + +def test_a_kill_detaches_the_stream_before_removing_the_output(harness): + """Order is load-bearing: a capture aimed at a removed output fails every + time, and the daemon's failure budget would close the viewers saying + "unknown output" instead of the session simply ending.""" + + harness.add() + harness.run(harness.manager.stream_enable("foot-aaaaaa")) + order: list[str] = [] + real_detach = harness.manager._detach_stream + real_destroy = harness.manager._destroy_container + + async def detach(session): + order.append("detach") + return await real_detach(session) + + def destroy(session): + order.append("output remove") + return real_destroy(session) + + harness.manager._detach_stream = detach + harness.manager._destroy_container = destroy + harness.run( + harness.manager.teardown( + harness.manager.registry.get("foot-aaaaaa"), reason="kill", grace_s=0.1 + ) + ) + + assert order == ["detach", "output remove"] + assert harness.daemon.outputs() == [] diff --git a/tests/unit/test_mcp_resources.py b/tests/unit/test_mcp_resources.py index a9b0559..8b0e540 100644 --- a/tests/unit/test_mcp_resources.py +++ b/tests/unit/test_mcp_resources.py @@ -42,11 +42,13 @@ EXPECTED_FIXED = { "hhstream://sessions", "hhstream://pointer", "hhstream://containers", + "hhstream://streams", "hhstream://screen", } EXPECTED_TEMPLATES = { "hhstream://sessions/{id}{?events}", "hhstream://sessions/{id}/logs{?lines}", + "hhstream://sessions/{id}/stream", "hhstream://outputs/{name}", } @@ -76,7 +78,8 @@ def test_a_template_is_exactly_an_entry_with_a_capture(): def test_the_three_reads_the_activity_rule_names_say_so_themselves(): """The rule has to be where it is read, and these are read, not called.""" - for name in ("sessions", "session", "session-logs", "pointer", "containers"): + for name in ("sessions", "session", "session-logs", "pointer", "containers", + "streams", "session-stream"): description = resources.BY_NAME[name].description assert "does NOT reset" in description, name assert "`session_touch`" in description, name @@ -109,6 +112,7 @@ def test_each_fixed_resource_runs_the_verb_it_names(read): "hhstream://sessions": ["session", "list", "--json"], "hhstream://pointer": ["session", "pointer", "state", "--json"], "hhstream://containers": ["session", "container", "--json"], + "hhstream://streams": ["session", "stream", "list", "--json"], } for uri, argv in cases.items(): _, calls = read(uri, stdout=mcp_fake.reply(ok=True)) @@ -290,6 +294,48 @@ def test_a_refused_capture_carries_the_cli_s_own_words(read): assert "hh-gone" in str(caught.value) +# ── the stream ───────────────────────────────────────────────────────────── + + +def test_one_session_s_stream_reads_through_its_own_route(read): + _, calls = read("hhstream://sessions/foot-abc/stream", stdout=mcp_fake.reply(ok=True)) + assert calls == [["session", "stream", "status", "--json", "--", "foot-abc"]] + + +def test_the_stream_routes_do_not_collide_with_the_logs_route(): + """Three templates under `sessions/{id}`; each must resolve to its own.""" + + assert resources.resolve("hhstream://sessions/x")[0].name == "session" + assert resources.resolve("hhstream://sessions/x/logs")[0].name == "session-logs" + assert resources.resolve("hhstream://sessions/x/stream")[0].name == "session-stream" + + +def test_reading_a_stream_never_enables_one(read): + """The read is `status`, not `enable` -- switching one on is a tool.""" + + _, calls = read("hhstream://sessions/x/stream", stdout=mcp_fake.reply(ok=True)) + assert "enable" not in calls[0] + _, calls = read("hhstream://streams", stdout=mcp_fake.reply(ok=True)) + assert "enable" not in calls[0] + + +def test_both_stream_reads_warn_that_the_url_carries_its_credential(): + """The URL is in these bodies, so the warning has to be on them.""" + + for name in ("streams", "session-stream"): + description = resources.BY_NAME[name].description + assert "PHRASE" in description, name + assert "only thing protecting" in resources.BY_NAME["streams"].description + + +def test_the_streams_listing_says_a_viewer_is_the_activity_exception(): + """The one exception to `session_touch`, where somebody will read it.""" + + description = resources.BY_NAME["streams"].description + assert "does NOT reset" in description + assert "connected viewer" in description + + # ── the one read that must not be a resource ─────────────────────────────── diff --git a/tests/unit/test_mcp_server.py b/tests/unit/test_mcp_server.py index 5b04d24..56c4348 100644 --- a/tests/unit/test_mcp_server.py +++ b/tests/unit/test_mcp_server.py @@ -117,7 +117,7 @@ def test_tools_list_returns_every_tool_with_a_title_and_a_description(session): listed = run(go).tools assert {tool.name for tool in listed} == set(tools.BY_NAME) - assert len(listed) == 21 + assert len(listed) == 23 for tool in listed: assert tool.title, tool.name assert tool.description and len(tool.description) > 40, tool.name @@ -524,6 +524,53 @@ def test_a_tool_outside_the_selection_cannot_be_called_by_name(session): assert fake.calls == [] +def test_the_stream_url_round_trips_through_a_resource_read(session): + """The phrase reaching an agent is the point of the whole feature.""" + + reply = { + "ok": True, + "id": "s-1", + "stream": { + "enabled": True, + "attached": True, + "output": "hh-s-1", + "fps": 5, + "viewers": 1, + "url": "http://127.0.0.1:9988/amber-otter-lantern-harbor-7k2qp/", + }, + } + fake, connected = session(stdout=json.dumps(reply, indent=2)) + + async def go(): + async with connected as client: + await client.initialize() + return await client.read_resource(AnyUrl("hhstream://sessions/s-1/stream")) + + content = run(go).contents[0] + assert json.loads(content.text)["stream"]["url"] == reply["stream"]["url"] + assert fake.calls == [["session", "stream", "status", "--json", "--", "s-1"]] + + +def test_enabling_a_stream_through_a_tool_call_returns_its_url(session): + reply = { + "ok": True, + "id": "s-1", + "stream": {"attached": True, "url": "http://127.0.0.1:9988/a-b-c-d-12345/"}, + } + fake, connected = session(stdout=json.dumps(reply, indent=2)) + + async def go(): + async with connected as client: + await client.initialize() + return await client.call_tool("session_stream_enable", {"id": "s-1"}) + + result = run(go) + assert not result.isError + assert result.content[0].text.startswith(reply["stream"]["url"]) + assert result.structuredContent == reply + assert fake.calls == [["session", "stream", "enable", "--json", "--", "s-1"]] + + def test_content_for_a_text_only_result_has_no_image_block(): blocks = server.content_for(tools.Result(text="hello")) assert [block.type for block in blocks] == ["text"] diff --git a/tests/unit/test_mcp_tools.py b/tests/unit/test_mcp_tools.py index 6733d8c..eb7920e 100644 --- a/tests/unit/test_mcp_tools.py +++ b/tests/unit/test_mcp_tools.py @@ -51,6 +51,7 @@ EXPECTED = { # lifecycle. The read-only verbs are resources; see MOVED_TO_RESOURCES. "session_start", "session_touch", "session_kill", "session_gc", "session_adopt", "session_container", "session_reconcile", + "session_stream_enable", "session_stream_disable", # pointer "pointer_acquire", "pointer_move", "pointer_click", "pointer_scroll", "pointer_down", "pointer_up", "pointer_release", @@ -72,7 +73,7 @@ MOVED_TO_RESOURCES = { def test_the_acting_verbs_are_tools_and_nothing_else_is(): assert set(tools.BY_NAME) == EXPECTED - assert len(tools.TOOLS) == 21 + assert len(tools.TOOLS) == 23 def test_the_read_only_verbs_are_not_tools(): @@ -108,7 +109,12 @@ def test_only_the_verbs_that_read_are_marked_read_only(): # `record` writes a file, so it is not read-only either. assert not tools.BY_NAME["record"].read_only assert tools.BY_NAME["session_kill"].destructive - assert [t.name for t in tools.TOOLS if t.destructive] == ["session_kill"] + # `stream_disable` is destructive too, and for a reason worth naming: it + # forgets the phrase, so anybody holding the URL loses it. + assert [t.name for t in tools.TOOLS if t.destructive] == [ + "session_kill", + "session_stream_disable", + ] # ── the two rules an agent gets wrong ────────────────────────────────────── @@ -448,6 +454,151 @@ def test_a_broker_that_is_down_is_reported_as_the_operator_s_problem(call): assert "frame tools" in result.text +# ── the live stream ──────────────────────────────────────────────────────── + + +def test_enabling_a_stream_puts_the_url_where_a_caller_will_look(call): + reply = { + "ok": True, + "id": "s-1", + "stream": { + "enabled": True, + "output": "hh-s-1", + "fps": 5, + "attached": True, + "url": "http://127.0.0.1:9988/amber-otter-lantern-harbor-7k2qp/", + "viewers": 0, + }, + } + result, calls = call( + "session_stream_enable", {"id": "s-1"}, stdout=json.dumps(reply, indent=2) + ) + + assert calls == [["session", "stream", "enable", "--json", "--", "s-1"]] + # The URL on the first line: it is the whole point of the call. + assert result.text.startswith(reply["stream"]["url"]) + assert result.data == reply + # ...and the one thing a caller must know about it. + assert "only thing protecting it" in result.text + + +def test_enabling_a_stream_passes_a_phrase_and_a_rate_through(call): + _, calls = call( + "session_stream_enable", + {"id": "s-1", "phrase": "amber-otter-lantern-harbor-7k2qp", "fps": 3}, + stdout=mcp_fake.reply(ok=True, id="s-1", stream={"url": "http://x/"}), + ) + assert calls == [ + [ + "session", "stream", "enable", "--json", + "--phrase=amber-otter-lantern-harbor-7k2qp", + "--fps=3", + "--", "s-1", + ] + ] + + +def test_an_enable_the_daemon_did_not_confirm_does_not_invent_a_url(call): + """A printed URL that serves nothing is worse than saying there is none.""" + + result, _ = call( + "session_stream_enable", + {"id": "s-1"}, + stdout=mcp_fake.reply(ok=True, id="s-1", stream={"attached": False}), + ) + assert "did not confirm" in result.text + assert "http" not in result.text.split("\n\n")[0] + + +def test_a_stream_on_a_session_with_no_container_is_exit_ten(call): + """It must never fall back to the main output -- that one carries everyone.""" + + result, _ = call( + "session_stream_enable", + {"id": "s-1"}, + stderr=mcp_fake.reply( + ok=False, + error="stream_unavailable", + reason="session s-1 has no container output to publish", + ), + exit_code=int(ExitCode.COMPOSITOR), + ) + assert result.is_error + assert result.data["exit_code"] == int(ExitCode.COMPOSITOR) + assert "not lost" in result.text # the exit-10 remedy: degraded, not dead + + +def test_a_phrase_that_is_not_this_session_s_is_exit_eight(call): + result, _ = call( + "session_stream_enable", + {"id": "s-1", "phrase": "other-words-go-here-99999"}, + stderr=mcp_fake.reply(ok=False, error="stream_busy", reason="already has a stream"), + exit_code=int(ExitCode.BUSY), + ) + assert result.is_error + assert result.text.startswith("exit 8 (BUSY)") + + +def test_disabling_a_stream_is_marked_destructive_and_forgets_the_phrase(call): + _, calls = call( + "session_stream_disable", {"id": "s-1"}, stdout=mcp_fake.reply(ok=True, detached=True) + ) + assert calls == [["session", "stream", "disable", "--json", "--", "s-1"]] + assert tools.BY_NAME["session_stream_disable"].destructive + assert "forget" in tools.BY_NAME["session_stream_disable"].description + + +def test_start_can_ask_for_a_stream_and_gets_its_url_back(call): + """One call, because a stream needs the container output that start makes.""" + + reply = { + "ok": True, + "id": "s-9", + "stream": {"url": "http://127.0.0.1:9988/amber-otter-lantern-harbor-7k2qp/"}, + } + result, calls = call( + "session_start", + {"app": "chromium", "stream": True, "stream_fps": 4}, + stdout=json.dumps(reply, indent=2), + ) + assert calls == [ + ["session", "start", "--json", "--stream", "--stream-fps=4", "--", "chromium"] + ] + assert "session id: s-9" in result.text + assert reply["stream"]["url"] in result.text + + +def test_a_start_whose_stream_failed_says_so_although_it_succeeded(call): + """Exit 0 with a running session; a caller must not read that as a stream.""" + + result, _ = call( + "session_start", + {"app": "chromium", "stream": True}, + stdout=mcp_fake.reply( + ok=True, id="s-9", stream_error="session s-9 has no container output to publish" + ), + ) + assert not result.is_error + assert "stream: NOT enabled" in result.text + assert "session_stream_enable" in result.text + + +def test_a_start_that_asked_for_no_stream_says_nothing_about_one(call): + result, calls = call( + "session_start", {"app": "foot"}, stdout=mcp_fake.reply(ok=True, id="s-1") + ) + assert "--stream" not in calls[0] + assert "stream" not in result.text.lower().split("\n\n")[0] + + +def test_the_stream_note_carries_both_facts_it_exists_for(): + assert "CREDENTIAL" in tools.STREAM_NOTE + assert "only protection" in tools.STREAM_NOTE + # The exception to the activity rule has to be where the stream is. + assert "A CONNECTED VIEWER DOES" in tools.STREAM_NOTE + assert tools.STREAM_NOTE in tools.BY_NAME["session_stream_enable"].description + + # ── shot and record ──────────────────────────────────────────────────────── From 9d33639c04edb3009f99bbdf80adc204c1187011 Mon Sep 17 00:00:00 2001 From: Hermenix Date: Sun, 11 Oct 2026 22:53:50 +0000 Subject: [PATCH 08/12] fix(sessions): `start --stream` answers with the daemon's stream, not the record's MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `start`'s reply was `Session.public()`, which reports what was *stored* -- and a `url` is never stored, because naming one is the daemon's (CONTRACT-sessions.md §3.5) and a URL on disk is a URL that lies the first time the listener moves. So a successful `start --stream` handed back a stream with a phrase and no URL, which no caller could tell from a stream that failed to open: the CLI printed WARNING: the session is running but its stream was not enabled: the broker reported no stream over a stream the daemon had attached a second earlier, and the MCP tool's summary said `stream: NOT enabled` for the same reason. `stream enable` was always right, because it answers with `_stream_public` -- only the one-call path was wrong. Found by checks/vm-stream.nix's last subtest, which is the only place `start --stream` runs against a real daemon. The unit test added beside it pins the seam rather than the symptom: the reply carries a `url`, the record does not, so anything answering from the record alone has to merge. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_011ZeH65zd6pFTgBSu52W9PE --- src/hhstreamd/manager.py | 19 ++++++++++++++++++- tests/sessions/unit/test_stream_verbs.py | 22 ++++++++++++++++++++++ 2 files changed, 40 insertions(+), 1 deletion(-) diff --git a/src/hhstreamd/manager.py b/src/hhstreamd/manager.py index d37a6f8..ce8c9fc 100644 --- a/src/hhstreamd/manager.py +++ b/src/hhstreamd/manager.py @@ -1018,6 +1018,9 @@ class SessionManager: if session.live: await self._read_back(session, app_id) stream_error: str | None = None + #: The daemon's view of the stream, which is the only place a `url` + #: exists. Locals and not attributes, for the reason below. + stream_public: dict[str, Any] | None = None if stream and session.live: # After the read-back, because that is what settles the container's # output, and a stream has to have one to publish @@ -1032,9 +1035,10 @@ class SessionManager: # be the wrong trade; the refusal is reported on the session's # `stream_error` and the caller can retry `stream.enable`. try: - await self.stream_enable( + enabled = await self.stream_enable( session.id, phrase=stream_phrase, fps=stream_fps ) + stream_public = enabled.get("stream") except ManagerError as exc: log.warning("could not enable %s's stream: %s", session.id, exc.reason) session.last_error = session.last_error or exc.reason @@ -1049,6 +1053,19 @@ class SessionManager: # a key that appears only on failure is a key every caller has to # remember to look for. record["stream_error"] = stream_error + if stream_public is not None: + # The daemon's view, not the record's. `Session.public()` + # reports what was stored, and a `url` is never stored -- + # naming one is the daemon's (§3.5), and a URL on disk is a URL + # that lies after the listener moves. So a successful + # `start --stream` that answered with `session.public()` alone + # handed back a stream with no URL, which a caller cannot tell + # from one that failed to open. + # + # MEASURED in checks/vm-stream.nix: the daemon attached, the + # broker recorded it, and the CLI printed "the session is + # running but its stream was not enabled". + record["stream"] = stream_public if session.state == State.EXITED: # The app exited during the settle: surface it rather than handing # back a healthy-looking id (DESIGN.md §7 #3). diff --git a/tests/sessions/unit/test_stream_verbs.py b/tests/sessions/unit/test_stream_verbs.py index ad70784..76484fd 100644 --- a/tests/sessions/unit/test_stream_verbs.py +++ b/tests/sessions/unit/test_stream_verbs.py @@ -217,6 +217,28 @@ def test_enabling_mints_a_phrase_the_daemon_accepts_and_publishes(harness): assert harness.daemon.outputs() == ["hh-foot-aaaaaa"] +def test_the_url_is_the_daemon_s_and_is_not_in_the_record(harness): + """So anything answering with the record alone must merge the reply in. + + REGRESSION. `start --stream` built its answer from `Session.public()`, + which reports what was *stored* -- and a `url` is never stored, because + naming one is the daemon's (CONTRACT-sessions.md §3.5). The result was a + successful start whose stream had no URL, which the CLI could not tell + from a stream that failed to open: it printed "the session is running but + its stream was not enabled" over a stream the daemon had just attached. + Found by checks/vm-stream.nix, which is also where the `start --stream` + path is proven end to end -- there is no launcher here. + """ + + harness.add() + reply = harness.run(harness.manager.stream_enable("foot-aaaaaa")) + + assert reply["stream"]["url"] + stored = harness.manager.registry.get("foot-aaaaaa").public(harness.clock.boottime()) + assert stored["stream"]["phrase"] == reply["stream"]["phrase"] + assert "url" not in stored["stream"] + + def test_the_phrase_is_in_the_record_so_it_survives_a_restart(harness): harness.add() first = harness.run(harness.manager.stream_enable("foot-aaaaaa"))["stream"] From 887ed17be2e4fb34c44a23b116da30e813df3e9e Mon Sep 17 00:00:00 2001 From: Hermenix Date: Sun, 11 Oct 2026 22:54:26 +0000 Subject: [PATCH 09/12] docs(stream): mcp-v3, the seventh box, and two TODOs the check wrote MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `docs/CONTRACT-mcp.md` goes to mcp-v3 with a change table: two tools, two resources, three `session_start` arguments, and the one rule that is not a name at all -- a connected viewer is the single exception to the idle-clock rule, and it cannot be invoked, so it is documented in §6 rather than offered in §3. v3 is additive; every v2 name still means what it meant. README.md leads the stream with the shape of it rather than the options: one port for the machine, a phrase per session, a browser `` as the entire client, and the four things worth knowing before switching one on -- that it is per session and off by default, that the phrase survives a restart, that capture runs only while somebody is connected, and that watching counts as activity. The phrase is called what it is: the only protection, with no TLS and no second factor. ARCHITECTURE.md gains the fourth unit as a seventh box -- `hhstream-live`, hanging off the broker by `live.sock` with one port and a browser at the end of it -- and the counts move to 23 and 10. Two TODOs, both written by `checks/vm-stream.nix`: * `a-stream-that-gave-up-still-reports-attached.md`. The publisher stops a stream after ten consecutive capture failures and the stop is sticky on purpose, but `Control._describe` does not pass `stopped` on, so `_stream_public` reports `attached: true` for a capture that has permanently given up. That is the second place `attached` answers "is a capture running right now" wrongly, and a field that is right only when nothing is wrong is not worth reading. * `capture-binary-and-the-unit-path.md` is now half fixed, and says so. The module passes grim's store path and this check reads real frames -- which immediately found the sibling fault, the unit's missing `XDG_RUNTIME_DIR`. What is still open is the other half: the app resolves its capture command by name and never checks it, so any deployment that is not this module still discovers a broken capture ten failures later. The plan's status section records Stage 2 and Stage 3 as built, the two decisions §5 left open (that `start` takes `--stream`, and that the default rate is the daemon's), and what Stage 2 deliberately did not do -- §6's restart re-attachment, which has its own TODO. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_011ZeH65zd6pFTgBSu52W9PE --- README.dev.md | 8 ++- README.md | 58 ++++++++++++++-- docs/ARCHITECTURE.md | 35 +++++++--- docs/CONTRACT-mcp.md | 67 ++++++++++++++++--- docs/TODO/README.md | 4 +- ...eam-that-gave-up-still-reports-attached.md | 58 ++++++++++++++++ docs/TODO/capture-binary-and-the-unit-path.md | 13 ++++ ...oes-not-reattach-after-a-broker-restart.md | 66 ++++++++++++++++++ .../plans/2026-10-04-session-stream-webapp.md | 65 +++++++++++++++--- 9 files changed, 341 insertions(+), 33 deletions(-) create mode 100644 docs/TODO/a-stream-that-gave-up-still-reports-attached.md create mode 100644 docs/TODO/stream-does-not-reattach-after-a-broker-restart.md diff --git a/README.dev.md b/README.dev.md index 75dbdb4..7b3170f 100644 --- a/README.dev.md +++ b/README.dev.md @@ -52,7 +52,7 @@ import the SDK: ```bash nix develop hhstream-mcp --list-tools # the 21 acting verbs -hhstream-mcp --list-resources # the 8 read-only URIs +hhstream-mcp --list-resources # the 10 read-only URIs ``` ## The checks @@ -68,8 +68,9 @@ hhstream-mcp --list-resources # the 8 read-only URIs | `vm-pointer` | the **lease and the driver**: a version-2 virtual pointer, a warp that lands where it was aimed *as the application reports it*, a click delivered as one press and one release, a contended acquire refused with BUSY, an expired lease swept with its held button lifted, and a broker restart that leaves the lock free, the bookmark intact and the driver reachable | yes | | `vm-keyboard` | the **keymap, the focus and a decoded symbol**: the keymap the driver uploads reaching a real compositor, `pointer acquire` leaving the aimed window holding the *keyboard* and not merely the cursor, and `hhs-keyboard-target` decoding `hello` from the keymap it was **forwarded** — so a driver that stopped uploading one arrives as `sym=?` and goes red | yes | | `vm-keyboard-de` | the same file with `sessions.keyboard.layout = "de"`, because that option reaches the uploaded keymap through exactly one path — `--xkb-layout` on the driver unit — and a single-layout check cannot tell that path from a hardcoded `us`: keysym `y`'s evdev code swaps (21 against 44), the forwarded `symbols=` name with it, and `ü` goes from exit 2 to `udiaeresis` | yes | +| `vm-stream` | the **live stream end to end**: `stream enable` on a live session mints a phrase the daemon accepts, a real HTTP viewer reads JPEGs off `//stream.mjpg` from *inside* the guest, `stream list` reports that viewer, the broker's idle-clock sweep stamps `touch` for it (and does not when nobody is watching), a wrong phrase 404s, and `kill` detaches the stream before the output is removed | yes | | `vm-stream-firewall` | the stream's **port**: the guest's own running firewall accepts the configured port and not the module's default, a peer outside the guest reaches it, and is refused on a port beside it that was never opened | yes | -| `vm-mcp` | the **MCP server over stdio**: `mcp.enable` puts the binary on PATH and starts no unit, the `initialize` handshake completes over a real pipe, all 21 tools and 8 resources are listed with their annotations and their two rules, both a tool call and a resource read reach the live broker and return its own protocol object, two reads of a session do **not** move its idle deadline while `session_touch` does, a frame arrives as a blob at exactly its output's configured size, the pointer lease is granted/used/returned against the real driver, `--toolsets` narrows the tools and leaves the resources alone, and a refusal arrives as a tool error (or, for a read, a protocol error) carrying the frozen exit code | yes | +| `vm-mcp` | the **MCP server over stdio**: `mcp.enable` puts the binary on PATH and starts no unit, the `initialize` handshake completes over a real pipe, all 23 tools and 10 resources are listed with their annotations and their two rules, both a tool call and a resource read reach the live broker and return its own protocol object, two reads of a session do **not** move its idle deadline while `session_touch` does, a frame arrives as a blob at exactly its output's configured size, the pointer lease is granted/used/returned against the real driver, `--toolsets` narrows the tools and leaves the resources alone, and a refusal arrives as a tool error (or, for a read, a protocol error) carrying the frozen exit code | yes | All of them at once: @@ -95,6 +96,7 @@ nix build -L .#checks.x86_64-linux.vm-keyboard nix build -L .#checks.x86_64-linux.vm-keyboard-de nix build -L .#checks.x86_64-linux.vm-stream-firewall nix build -L .#checks.x86_64-linux.vm-mcp +nix build -L .#checks.x86_64-linux.vm-stream ``` `-L` is not optional in practice: without it, Nix hides the build log, and the @@ -262,6 +264,8 @@ checks/ vm-stream-firewall.nix nixosTest, the stream's port in the guest's firewall vm-mcp.nix nixosTest, hhstream-mcp driven over stdio by a raw JSON-RPC client against the live broker + vm-stream.nix nixosTest, `session stream enable` and a real HTTP + viewer reading MJPEG from inside the guest lib/mcp_client.py that client: hhs-mcp-call, stdlib only and deliberately NOT the MCP SDK's -- the SDK on both ends would agree with itself about a framing neither side had to get right diff --git a/README.md b/README.md index 5bdb2a1..f0b4442 100644 --- a/README.md +++ b/README.md @@ -35,6 +35,52 @@ hhstream-client --socket /run/hypr-stream/stream.sock --count 1 --out frame.png The socket is a plain file descriptor: the server serves every client that connects, and keeps running when one disconnects. +## Watching a session: one port, a phrase per session + +A session can publish its own screen on a web page, so a person can watch what +an agent is doing from a phone. One port for the whole machine, and a path that +is four words and a short tail: + +```bash +hhstream session stream enable chromium-k3f9d2 +# http://127.0.0.1:9988/amber-otter-lantern-harbor-7k2qp/ +``` + +That URL is the whole interface — a browser `` is the entire client, over +MJPEG — and the phrase in it is its **only** protection. No TLS, no second +factor: anyone who can reach the port and guess four words can watch. It is +never written to the audit log, and `stream list` does not print it. + +Four things worth knowing before switching one on: + +- **It is per session and off by default.** A session does not stream because + it exists; it streams because somebody enabled it. What is published is that + session's own container output, never the main one — the main output carries + every other session, and putting it behind one session's phrase is the + mistake this refuses to make. +- **The phrase survives a restart.** It lives in the session's record, so + somebody you read it out to keeps the URL you gave them. `disable` then + `enable` rotates it. +- **Capture runs only while somebody is connected.** An enabled stream nobody + is watching costs no `grim` and no CPU. +- **Watching counts as activity.** This is the one exception to "`touch` is the + only mark of activity": about once a minute the broker notices a connected + viewer and resets that session's idle clock, so a long job you are watching + on a phone is not reaped at three hours. An enabled stream with nobody + watching keeps nothing alive. + +```nix +services.hyprland-headless-stream.sessions.stream = { + enable = true; # runs hhstream-live; turns no stream on + # bind = "0.0.0.0"; # and openFirewall, to reach it from another device + # openFirewall = true; +}; +``` + +Loopback by default, where the path to a phone is `ssh -L 9988:127.0.0.1:9988`. +`hhstream session start --stream` starts an application and prints its +URL in one go, which is the usual way an agent sets one up. + ## Driving it from an agent: the MCP server `hhstream-mcp` is the `hhstream` CLI behind a [Model Context @@ -45,14 +91,16 @@ application without writing command lines. The interface is frozen in It is split the way the protocol splits things: -- **Resources are the state** — eight of them, read by URI, and reading one +- **Resources are the state** — ten of them, read by URI, and reading one changes nothing. `hhstream://status` is whether the desktop is up, `hhstream://sessions` is what is running, `hhstream://sessions//logs` is an application's own output, `hhstream://pointer` is who holds the input - lease, `hhstream://screen` is a frame of the screen. -- **Tools are the acts** — twenty-one, one per CLI verb that does something. + lease, `hhstream://screen` is a frame of the screen, `hhstream://streams` is + what is being published and who is watching. +- **Tools are the acts** — twenty-three, one per CLI verb that does something. `session_start` puts an application on the screen, `pointer_acquire` takes the - input lease, `keyboard_type` types, `session_kill` ends a session. + input lease, `keyboard_type` types, `session_stream_enable` publishes the + screen for a person to watch, `session_kill` ends a session. A host decides what state to put in front of its model; the model decides what to do. That is the whole of the split, and it is why `shot` is still a tool @@ -104,7 +152,7 @@ account in the group. `hhstream-mcp --list-tools` and `--list-resources` print the two tables and exit, which is the quickest way to check an install. -If twenty-one tools is more than a client has room for, `--toolsets` narrows +If twenty-three tools is more than a client has room for, `--toolsets` narrows what is offered to whichever families are wanted — `frames`, `sessions`, `pointer`, `keyboard`: diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index c26f7c1..0c05cb1 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -26,9 +26,17 @@ │ hhstream-sessions.service │─────▶│ hhstream-pointer.service │ │ hhstreamd │ │ one Wayland connection │ │ registry, containers, │ JSON │ one virtual pointer │ - │ the pointer lease │ over │ per output │ - └────────────┬──────────────┘ a └──────────────────────────┘ - │ private socket + │ the pointer lease, │ over │ per output │ + │ the stream phrases │ a └──────────────────────────┘ + └────────────┬────────┬─────┘ private socket + │ │ ┌──────────────────────────┐ + │ └─────────────▶│ hhstream-live.service │ + │ live.sock │ one port (9988) │ + │ │ grim ─▶ MJPEG per │ + │ │ phrase, while watched │ + │ └───────────┬──────────────┘ + │ │ HTTP + │ a browser ┘ // │ AF_UNIX control.sock, mode 0660 ▼ hhstream session ... (the CLI an agent drives) @@ -37,12 +45,12 @@ │ ┌────────────┴──────────────┐ │ hhstream-mcp │ spawned per client, stdio only - │ 21 tools (the acts) │◀─── an MCP client's JSON-RPC - │ 8 resources (the state)│ + │ 23 tools (the acts) │◀─── an MCP client's JSON-RPC + │ 10 resources (the state)│ └───────────────────────────┘ ``` -Five pieces, deliberately separate, and a sixth that is deliberately not a +Six pieces, deliberately separate, and a seventh that is deliberately not a piece at all: 1. **The compositor** (`Hyprland`, unchanged upstream) renders into virtual @@ -62,9 +70,20 @@ piece at all: a separate process because a Wayland client needs its own connection and a driver blocked on the compositor must not stall session lifecycle. -And the sixth, which is a *shim* and not a layer: +6. **The stream daemon** (`hhstream-live`) publishes one session's output as + MJPEG on one port, behind a four-word phrase, while somebody is connected. + A third executor on the pointer driver's pattern and for its reasons: `grim` + blocks tens to hundreds of milliseconds so it must not run where `touch` and + `kill` are answered, and a LAN-facing HTTP parser must not run in the frame + server whose one-socket contract cannot grow one. The broker owns the + *decision* (which session, under which phrase, in the record); the daemon + owns the port, the capture and the URL. It knows nothing about sessions or + idle clocks — it publishes what it is told to publish, and reports a viewer + count the broker chooses to read as activity. -6. **The MCP server** (`hhstream-mcp`, `docs/CONTRACT-mcp.md`) is the CLI with a +And the seventh, which is a *shim* and not a layer: + +7. **The MCP server** (`hhstream-mcp`, `docs/CONTRACT-mcp.md`) is the CLI with a Model Context Protocol surface on it, so an agent reads state by URI and acts by calling tools instead of writing command lines. It is drawn above the CLI rather than beside the broker because that is where it sits: every diff --git a/docs/CONTRACT-mcp.md b/docs/CONTRACT-mcp.md index f4b6ff8..f647164 100644 --- a/docs/CONTRACT-mcp.md +++ b/docs/CONTRACT-mcp.md @@ -1,4 +1,4 @@ -# CONTRACT — the MCP server (mcp-v2) +# CONTRACT — the MCP server (mcp-v3) This file is the interface of `hhstream-mcp`: the resource URIs, the tool names, the arguments each takes, what comes back, and which of the CLI's exit codes @@ -10,7 +10,19 @@ containers and the input lease). these argument shapes is a consumer like any other, and renaming a tool breaks it exactly as renaming a wire verb would. -Version: mcp-v2 (2026-10-11). v1 was the first freeze and lasted one day. +Version: mcp-v3 (2026-10-11). v1 was the first freeze and lasted one day; v2 +split state out into resources; v3 adds the live stream. v3 is **additive** — +every v2 name still means what it meant. + +v3 changes, all of them the per-session live stream +(`docs/CONTRACT-sessions.md` §4, v4): + +| § | Change | Why | +| --- | --- | --- | +| §3 | Tools `session_stream_enable` and `session_stream_disable`. 21 tools became 23. | Switching a stream on is an *act*, and a consequential one: it publishes a desktop on the LAN behind a phrase. Nothing about it is a read. | +| §3 | `session_start` gains `stream`, `stream_phrase` and `stream_fps`, and its reply carries the URL. | A stream needs the container output that starting creates, so sequencing start-then-enable in the caller means every caller repeats the ordering and the half-done case. One call, one answer, and the phrase comes back with the session. | +| §4 | Resources `hhstream://streams` and `hhstream://sessions/{id}/stream`. 8 resources became 10. | What is being published, and who is watching, is state — so it is read, not called. | +| §6 | A connected viewer is the **one exception** to the idle-clock rule. | About once a minute the broker stamps `touch` on any session somebody is watching. It is not a verb and cannot be invoked; it is the broker noticing that a person is looking. | **v2 is a breaking change, and the only one: the verbs that merely read are resources now, not tools.** @@ -120,7 +132,7 @@ to an argv list. | family | tools | | --- | --- | | `frames` | `shot`, `record`, `run` | -| `sessions` | `session_start`, `session_touch`, `session_kill`, `session_gc`, `session_adopt`, `session_container`, `session_reconcile` | +| `sessions` | `session_start`, `session_touch`, `session_kill`, `session_gc`, `session_adopt`, `session_container`, `session_reconcile`, `session_stream_enable`, `session_stream_disable` | | `pointer` | `pointer_acquire`, `pointer_move`, `pointer_click`, `pointer_scroll`, `pointer_down`, `pointer_up`, `pointer_release` | | `keyboard` | `keyboard_type`, `keyboard_key`, `keyboard_down`, `keyboard_up` | @@ -154,13 +166,15 @@ running. | Tool | Arguments | | --- | --- | -| `session_start` | `app`, `argv?`, `idle_timeout?`, `wait_ready?`, `home?`, `ephemeral_home?`, `request_id?` | +| `session_start` | `app`, `argv?`, `idle_timeout?`, `wait_ready?`, `home?`, `ephemeral_home?`, `stream?`, `stream_phrase?`, `stream_fps?`, `request_id?` | | `session_touch` | `id` | | `session_kill` | `id`, `grace?`, `signal?`, `request_id?` | | `session_gc` | `request_id?` | | `session_adopt` | `pid`, `app?`, `idle_timeout?`, `request_id?` | | `session_container` | `id` | | `session_reconcile` | `id?`, `request_id?` | +| `session_stream_enable` | `id`, `phrase?`, `fps?` | +| `session_stream_disable` | `id` | Every one of these runs with `--json`, so what comes back is the broker's own protocol object: in the text part verbatim, and in `structuredContent` parsed. @@ -170,6 +184,21 @@ second wire format. `home` and `ephemeral_home` are mutually exclusive, and the schema says so — the CLI's argparse group, moved forward to where the refusal is free. +`session_stream_enable` returns the **URL** on the first line of its text part, +because that is the whole point of the call: it is what gets handed to a person. +It publishes the session's own container output and never the main one — that +output carries every other session, and putting it behind one session's phrase +is the mistake the broker refuses to make (exit 10 with no container). Re-enabling +returns the same URL; a different `phrase` is exit 8, not a silent rotation, +because rotating under somebody holding the old URL is the one outcome neither +answer can be mistaken for. `session_stream_disable` is marked **destructive**: +it forgets the phrase, so anyone holding that URL loses it. + +`session_start` with `stream: true` is one call rather than two, and a stream +that cannot be opened **does not fail the start** — the application is on the +screen, and killing it over a web page would be the wrong trade. The reply +carries `stream_error` whenever `stream` was asked for, `null` when it worked. + `session_container` is **not** marked read-only, and that is not an oversight: with an `id` the CLI runs `container.reconcile`, which writes the record back. That is also why it is a tool rather than a resource. The no-id listing, which @@ -232,6 +261,13 @@ verb a tool would have, with `--json` where the verb has it. | `hhstream://sessions/{id}` | `application/json` — one session with its recent audit events | `?events=N` | | `hhstream://sessions/{id}/logs` | `application/json` — the application's own captured output | `?lines=N` | | `hhstream://outputs/{name}` | one output captured alone, as an image blob. Exactly that output's configured size, which is the coordinate space the pointer tools aim in. | — | +| `hhstream://streams` | `application/json` — every session a stream was enabled on, with its URL, rate and connected viewer count, plus `available` (whether the stream daemon answered) | — | +| `hhstream://sessions/{id}/stream` | `application/json` — one session's stream, or `null`. `attached` is separate from `enabled`, and the URL is absent rather than stale when it is false. | — | + +**The URLs in those two bodies contain their phrases**, which are the only thing +protecting those streams. Both descriptions say so, because a host that puts +`hhstream://streams` into a model's context has put a credential there — and +the one place it must never go is anywhere the streamed session can see. Rules this surface keeps: @@ -349,13 +385,22 @@ in terms of this surface. They are on the individual tools and resources and not only in the server instructions, because a client shows a thing's own description at the point of use: -1. **`session_touch` is the only thing that resets a session's idle clock.** - Reading `hhstream://sessions`, `hhstream://sessions/{id}` or its logs does - not, `session_gc` does not, and neither does any pointer or keyboard tool. A +1. **`session_touch` is the only thing a *caller* can do that resets a + session's idle clock.** Reading `hhstream://sessions`, + `hhstream://sessions/{id}` or its logs does not, `session_gc` does not, the + stream tools do not, and neither does any pointer or keyboard tool. A session being driven is still a session nobody has said they want kept. The - three resources it applies to say so in their own descriptions, because a + resources it applies to say so in their own descriptions, because a description is what a host shows beside the thing it describes. + **The one exception is a connected viewer** (`docs/CONTRACT-sessions.md` §4, + v4): about once a minute the broker stamps `touch` on any session somebody + is watching through its live stream. It is not a tool and cannot be + invoked — it is the broker noticing that a person is looking, which is the + one form of "still wanted" that does not come down the control socket. It + cannot hold a session open by itself: an enabled stream with no viewer + captures nothing and stamps nothing. + 2. **Every keyboard tool needs the pointer lease, and holds only while it holds.** There is no keyboard lease: one compositor seat means one keyboard focus. A held non-modifier key floods the window until it is lifted or the @@ -376,6 +421,12 @@ socket, the CLI and the broker are stdlib-only by policy. `packages.hhstream` must not grow that dependency; the root `pyproject.toml` excludes `hhstream_mcp*` from its package discovery so it cannot acquire it by accident. +The stream tools and the two stream resources additionally need +`services.hyprland-headless-stream.sessions.stream.enable`, which runs the +`hhstream-live` unit and gives the broker the socket it switches streams on +over. Without it they answer exit 10 (`stream_unavailable`) — the session is +unaffected, which is what that code means. + `services.hyprland-headless-stream.mcp.enable` (default `false`) puts the binary on the system PATH. That is the whole of the option: there is no unit, no socket and no state directory, because the server is spawned per client over diff --git a/docs/TODO/README.md b/docs/TODO/README.md index dae4329..099bc10 100644 --- a/docs/TODO/README.md +++ b/docs/TODO/README.md @@ -14,9 +14,11 @@ Each file answers three questions and stops: | [scroll-has-no-compositor-proof](scroll-has-no-compositor-proof.md) | medium | works live; unproven in a gate | | [reconcile-while-containers-are-off](reconcile-while-containers-are-off.md) | medium | partly fixed: discovery now survives a restart; teardown still lives inside the feature | | [kill-can-leave-the-app-alive](kill-can-leave-the-app-alive.md) | medium | found by counting windows, not sessions | -| [capture-binary-and-the-unit-path](capture-binary-and-the-unit-path.md) | medium | found the first time the app was run as a unit: a unit's `PATH` has no `grim` | +| [capture-binary-and-the-unit-path](capture-binary-and-the-unit-path.md) | medium | partly fixed: the module passes grim's store path and a check reads real frames; the app still never checks its capture binary | | [mcp-in-process-has-no-call-deadline](mcp-in-process-has-no-call-deadline.md) | medium | two costs of running the CLI in process: no killable deadline, and no `sg` recovery | | [mcp-resources-cannot-be-subscribed-to](mcp-resources-cannot-be-subscribed-to.md) | medium | the SDK hardcodes `subscribe=False`, so a log resource can only be re-read | +| [stream-does-not-reattach-after-a-broker-restart](stream-does-not-reattach-after-a-broker-restart.md) | low | the phrase survives and the sweep works; `attached` reports false until re-enabled | +| [a-stream-that-gave-up-still-reports-attached](a-stream-that-gave-up-still-reports-attached.md) | low | the other direction: a capture that permanently gave up still reads `LIVE: YES` | Closed since this folder was written: diff --git a/docs/TODO/a-stream-that-gave-up-still-reports-attached.md b/docs/TODO/a-stream-that-gave-up-still-reports-attached.md new file mode 100644 index 0000000..7d4d64f --- /dev/null +++ b/docs/TODO/a-stream-that-gave-up-still-reports-attached.md @@ -0,0 +1,58 @@ +# A stream whose capture gave up still reports `attached: true` + +## The issue + +`hhstream_live.publisher` stops a stream after `FAILURE_LIMIT` (10) consecutive +capture failures, and the stop is sticky on purpose — the case it guards is an +output that was removed, and that does not come back. The stream object stays in +`StreamRegistry` with `stopped = True`; the HTTP handler returns as soon as it +notices, so viewers are dropped and new ones get a page that ends immediately. + +Nothing on the broker side can see that. `Control._describe` returns +`spec.public()` plus `url` and `viewers` — not `stopped`, `captured`, +`failures` or `last_frame_age_s`, all of which `Stream.status()` already +computes. So `LiveStreams.list()` reports the phrase, `_attachments()` finds it +and `_stream_public` sets `attached = True`. `hhstream session stream status` +says `LIVE: YES` for a capture that has permanently given up. + +MEASURED, in checks/vm-stream.nix, by the bug that check found: the +`hhstream-live` unit had the compositor's group but not its environment, so +every `grim` exited 1 with + + error: XDG_RUNTIME_DIR is invalid or not set in the environment. + +The unit was `active (running)`, the broker's record was intact, `stream status` +said `attached: true` with a URL, and the only place the truth appeared was a +single `WARNING` in the daemon's journal. The environment bug is fixed +(`execLive` in nix/module.nix); the reporting gap it exposed is not. + +## Ideas + +1. **Carry the publisher's health through `list`.** *(recommended)* Add + `stopped` and `last_frame_age_s` to `Control._describe` — both already exist + on `Stream.status()` — parse them in `hhstreamd.live.Attachment`, and have + `_stream_public` report `attached: false` (or a third state) when the daemon + says the capture is dead. The cost is one more field pair in + CONTRACT-sessions.md §3.5 and a `render_stream` line; the gain is that + `attached` means what it was added to mean. +2. **Drop a stopped stream from the registry.** Then `list` stops naming it and + `attached` goes false for free, with no new fields. Rejected as the first + move because it destroys the evidence: `failures` and the phrase are what an + operator needs in order to understand what happened, and a 404 on the URL + reads as "wrong phrase". +3. **Leave it.** What ships. `stream disable` then `stream enable` revives it + (a fresh `Stream`, same phrase, same URL), which is what an operator reaches + for anyway, and the daemon's journal does say what happened. + +## Why it matters + +Low severity because the capture only gives up for a reason that is itself a +failure — a removed output, a compositor that went away, a misconfigured unit — +and in two of those three the session is in trouble in ways `status` does show. + +It matters because `attached` exists specifically to separate "the operator +asked for this" from "a capture is running right now" (CONTRACT-sessions.md +§3.5), and this is the second case where it answers the second question wrongly; +see [stream-does-not-reattach-after-a-broker-restart](stream-does-not-reattach-after-a-broker-restart.md) +for the first. A field that is right only when nothing is wrong is not worth +reading. diff --git a/docs/TODO/capture-binary-and-the-unit-path.md b/docs/TODO/capture-binary-and-the-unit-path.md index c2a0a76..b3ca37a 100644 --- a/docs/TODO/capture-binary-and-the-unit-path.md +++ b/docs/TODO/capture-binary-and-the-unit-path.md @@ -27,6 +27,19 @@ worst way. The unit starts, the port opens, the landing page answers, and `"failures": 10`. A viewer sees an empty frame area and nothing else says why. Only the unit's log names the reason, one line above the giving-up entry. +**Half of this is now fixed.** The module passes grim's absolute store path +(idea 1), checks/vm-stream.nix starts the unit and reads real JPEGs off the +stream — the check this file asked for — and that check immediately found the +sibling fault: the unit had the compositor's *group* but not its *environment*, +so grim exited 1 with "XDG_RUNTIME_DIR is invalid or not set", ten times, with +the same silence. Fixed too (`execLive`). What is still open is idea 2: the app +resolves its capture command by name and never checks it, so a standalone +`hhstream-live serve` and any deployment shape that is not this module still +discover a missing or unusable capture binary ten failures later. The quiet +half — a stream that gave up while the broker still reports it attached — has +its own file now: +[a-stream-that-gave-up-still-reports-attached](a-stream-that-gave-up-still-reports-attached.md). + ## Ideas 1. **An absolute binary, as the module already does elsewhere** *(recommended)*. diff --git a/docs/TODO/stream-does-not-reattach-after-a-broker-restart.md b/docs/TODO/stream-does-not-reattach-after-a-broker-restart.md new file mode 100644 index 0000000..4a82de6 --- /dev/null +++ b/docs/TODO/stream-does-not-reattach-after-a-broker-restart.md @@ -0,0 +1,66 @@ +# A broker restart leaves an enabled stream recorded but not attached + +## The issue + +`docs/plans/2026-10-04-session-stream-webapp.md` §6 says what a restart should +do: + +> If `hhstream-live` is still up, the broker asks it for the list and +> re-attaches them to the session records, so `status` and teardown still see +> them. + +Stage 2 did not build that half. What happens today, measured in +`tests/sessions/unit/test_stream_verbs.py` +(`test_a_down_daemon_turns_attached_false_rather_than_failing_the_read`) and by +reading `SessionManager.startup`: + +* the record keeps its phrase, so nothing is lost and a later `enable` restores + the *same* URL — which is the property §4 exists for, and it works; +* but `startup` does not call `LiveStreams.list()`, so the broker does not + learn about streams the daemon is still publishing; +* so `stream list` and `stream status` report `attached: false` for a stream + that is, in fact, up and serving. The viewer sweep then skips it, because it + only touches sessions whose phrase the daemon reports a viewer for — and it + asks `viewers`, which *would* name it. So the sweep actually still works; + what is wrong is the reporting. + +The symptom is therefore narrow and confusing rather than broad: after +`systemctl restart hhstream-sessions`, somebody watching a stream keeps +watching it, the session keeps being touched, and `hhstream session stream +list` says `LIVE: NO`. An operator reading that would reasonably `enable` +again — which is harmless (`attach` is idempotent on the phrase) but should not +be necessary. + +The other half of §6 *is* implemented: if the daemon restarted too, the records +keep their phrases and the streams stay off until re-enabled, because an open +port and a running capture are explicit acts. + +## Ideas + +1. **Ask the daemon in `startup` and merge.** *(recommended)* One + `LiveStreams.list()` where `reconcile_containers()` already runs, matching + attachments to records by phrase. Small and local: `_stream_public` already + merges an attachment into a record, so this is about *having* the + attachments at start-up rather than only at read time. The one thing to get + right is a phrase the daemon is publishing that no live record claims — + which is the stream equivalent of an orphan `hh-*` output, and should + probably be detached rather than adopted. +2. **Ask on every read instead.** `_attachments()` already does, which is why + the *reads* are nearly right. The reason it is not enough: there is no + record-side state to update, so a restart's first `stream list` is correct + and `attached` is still computed per call — this is really just (1) done + lazily, and it hides the orphan case entirely. +3. **Leave it.** What ships. The phrase survives, re-enabling is idempotent and + costs one record write, and the sweep keeps working. The cost is a status + field that lies after a restart. + +## Why it matters + +Low severity because nothing breaks and nothing is lost: the stream keeps +serving, the viewer keeps being counted, and the fix an operator will reach for +by instinct (`enable` again) is exactly the right one and is idempotent. + +It matters at all because `attached` was added specifically so that "the +operator asked for this" and "a capture is running right now" could be told +apart — and after a restart it reports the second one wrongly, which is the one +job that field has. diff --git a/docs/plans/2026-10-04-session-stream-webapp.md b/docs/plans/2026-10-04-session-stream-webapp.md index e423e67..4d98517 100644 --- a/docs/plans/2026-10-04-session-stream-webapp.md +++ b/docs/plans/2026-10-04-session-stream-webapp.md @@ -175,12 +175,54 @@ one at a time; no browser in a VM check. 3. **Port 9988**, fixed and single, no range fallback. 4. **Capture pauses at zero viewers** (v1's choice, kept). -**Stage 1 is built** (`src/hhstream_live/`, `tests/live/`, 51 unit tests): the -router, the pages, the phrase and the publisher, runnable standalone via -`hhstream-live serve --stream PHRASE=OUTPUT`. The broker verbs and -`/run/hhstream/live.sock` are Stage 2. +**Stage 1 is built** (`src/hhstream_live/`, `tests/live/`): the router, the +pages, the phrase and the publisher, runnable standalone via `hhstream-live +serve --stream PHRASE=OUTPUT`. -**Stage 3 has started, out of order and on purpose** (2026-10-11). A machine +**Stage 2 is built** (2026-10-11), and §5's surface is what shipped: +`stream.enable|disable|list|status` on the control socket, `hhstream session +stream …` on the CLI, the record field at **registry schema 4**, +`/run/hhstream/live.sock` with `hhstreamd/live.py` on the broker's end and +`hhstream_live/control.py` on the daemon's, the teardown order, and the viewer +heartbeat. `docs/CONTRACT-sessions.md` went to **sessions-v4** for it. + +Two decisions §5 left open, settled in the building: + +1. **`start` takes `--stream`.** A stream needs the container output that + starting creates, so "start, then enable" made every caller repeat the + ordering and the half-done case in between. One call, and the URL comes + back with the session id. A stream that cannot be opened does **not** fail + the start — the application is on the screen, and `stream_error` says what + went wrong. +2. **The default rate is the daemon's, not the broker's.** §5 did not say who + owns it, and the first cut had the broker send its own default on every + attach — which would have made `sessions.stream.fps` a setting that changed + nothing, the exact failure `--xkb-layout` exists to avoid. `fps` is now + omitted from `attach` unless a caller named one. + +What Stage 2 did **not** do: `stream.enable` carries no `--port`, and §6's +restart re-attachment is not implemented. A broker restart leaves the records' +phrases intact and their streams off until re-enabled, which is §6's own +fallback ("an open port and a running capture are explicit acts") — but it does +*not* ask a surviving daemon for its list and re-attach, so `attached` reads +false until somebody enables again. Recorded in `docs/TODO/`. + +**Stage 3 is built** (2026-10-11), the unit included — `hhstream-live.service` +runs the app with `--control-socket`, the module's own capture command (`grim` +is not on a unit's PATH), and `sessions.stream.{enable,fps,port,bind,openFirewall}`. +What follows is why the firewall half landed first. + +The unit has to be handed two things, and both were found by +`checks/vm-stream.nix` rather than reasoned about: the capture command with +grim's store path, because a unit's `PATH` has no `grim`; and the compositor's +session — `XDG_RUNTIME_DIR` plus the `wayland-N` the compositor actually bound, +discovered at start by `execLive`. Without the second the unit was +`active (running)`, the broker attached, `stream status` said `attached: true`, +and every capture exited 1 with "XDG_RUNTIME_DIR is invalid or not set" until +the publisher gave up. Both failures are silent in the same way, which is why +the check reads JPEG bytes instead of asserting a 200. + +**Stage 3 started out of order and on purpose.** A machine had to be able to *reach* a stream before the rest of the app is settled: the operator asked for the port to be open on the deployment machine, where the app runs by hand, and a hand-written firewall line in one machine's config is the @@ -189,7 +231,12 @@ openFirewall }` and the single rule it stands for are in `nix/module.nix`, with `checks/vm-stream-firewall.nix` asking the *running* firewall — and a peer outside the guest — whether the configured port is the one that opened. -What that does **not** do: no unit, no phrases, no `session stream -enable|disable|list|status`, and nothing in the module turns a stream on. The -app is still started by hand, and the option's whole job is to describe where a -stream will be reachable from. \ No newline at end of file +At the time that did **not** include a unit, phrases or the verbs; those are +above. What is still true of it is the last sentence: nothing in the module +turns a stream *on*. `sessions.stream.enable` runs a listener serving one form, +and a stream exists only for the sessions somebody enabled it on. + +**Stage 4 — the host pass — has not been done.** Nothing here has had a real +browser pointed at it; `checks/vm-stream.nix` reads the MJPEG with a +hand-rolled client inside the guest, which proves the framing and the capture +but not that a phone paints it. \ No newline at end of file From 0c06de4ea75a00025637c3e5f0dc31967c5c6533 Mon Sep 17 00:00:00 2001 From: Hermenix Date: Sun, 11 Oct 2026 22:56:57 +0000 Subject: [PATCH 10/12] test(mcp): the stream through the MCP layer, in the guest `vm-mcp` asserted 21 tools and the eight resource URIs by name, which the stream made stale -- and by name is the point, so the lists move with the surface: 23 tools, `hhstream://streams` among the fixed resources, `hhstream://sessions/{id}/stream` among the templates. Its machine now enables `sessions.stream`, because two of those tools and two of those resources are about streams and with no daemon they could only be shown to *refuse*. Two subtests, and the first is the one worth having: a session started over MCP with `stream: true` has to hand back the URL in `structuredContent`, in the text a model actually reads, and again from `hhstream://sessions/{id}/stream` -- three places that must agree, because the URL is the daemon's to name, is never stored, and a reply assembled from the broker's record alone carries a phrase and no URL. That is exactly how it went wrong once. The second takes the stream away over MCP and reads both resources again. What this does not do is read a frame: bytes off the port are `vm-stream`'s subject. What it owns is that the phrase reaches an agent at all. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_011ZeH65zd6pFTgBSu52W9PE --- checks/vm-mcp.nix | 65 +++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 63 insertions(+), 2 deletions(-) diff --git a/checks/vm-mcp.nix b/checks/vm-mcp.nix index 30ec763..a33d75d 100644 --- a/checks/vm-mcp.nix +++ b/checks/vm-mcp.nix @@ -11,7 +11,7 @@ # WHAT THIS PROVES, AND WHAT IT DOES NOT. # # It proves the *surface*, both halves of it: the handshake completes over a -# real pipe, all twenty-one tools and all eight resources are listed, a tool +# real pipe, all twenty-three tools and all ten resources are listed, a tool # call reaches the broker and comes back with the broker's own protocol object, # a resource read reaches it the same way, an image arrives as a blob with the # geometry the module configured, `--toolsets` actually narrows what is @@ -119,6 +119,17 @@ testers.runNixOSTest ( package = hhstream-mcp; }; + # Enabled because two tools and two resources here are about streams, + # and with no daemon they could only be shown to *refuse*. What a stream + # actually serves is `vm-stream`'s subject -- it reads JPEGs off the + # port; what this check owns is that the URL (with its phrase) comes back + # through the MCP layer at all. + services.hyprland-headless-stream.sessions.stream = { + enable = true; + bind = "127.0.0.1"; + port = 9988; + }; + environment.systemPackages = [ client ]; }; @@ -227,10 +238,11 @@ testers.runNixOSTest ( listed = mcp("--list")["tools"] names = sorted(tool["name"] for tool in listed) print(f"{len(names)} tools: {', '.join(names)}") - assert len(names) == 21, names + assert len(names) == 23, names for expected in ( "shot", "record", "run", "session_start", "session_touch", "session_kill", + "session_stream_enable", "session_stream_disable", "pointer_acquire", "pointer_click", "keyboard_type", ): assert expected in names, f"{expected} is missing from the listing" @@ -257,10 +269,12 @@ testers.runNixOSTest ( "hhstream://screen", "hhstream://sessions", "hhstream://status", + "hhstream://streams", ], uris assert templates == [ "hhstream://outputs/{name}", "hhstream://sessions/{id}/logs{?lines}", + "hhstream://sessions/{id}/stream", "hhstream://sessions/{id}{?events}", ], templates by_uri = {r["uri"]: r for r in both["resources"]} @@ -532,6 +546,53 @@ testers.runNixOSTest ( except Exception: dump("session_kill over mcp", session_id) raise + + with subtest("a stream asked for at start comes back as a url, twice"): + # The two places an agent can learn the phrase, and they have to + # agree: the `session_start` reply, and the resource for that + # session's stream. A URL is the daemon's to name and is never + # stored, so a reply built from the broker's record alone would + # carry a phrase and no URL -- which is how this went wrong once + # (see `SessionManager.start`). + streamed_id = None + try: + streamed = succeeded( + "session_start", {"app": "foot", "stream": True}, timeout=300 + ) + streamed_id = streamed["structuredContent"]["id"] + stream = streamed["structuredContent"]["stream"] + print(f"{streamed_id} -> {stream['url']}") + assert stream["url"].startswith("http://127.0.0.1:9988/"), stream + # The phrase is IN the URL and is the whole of its protection, + # so the text a model reads has to carry it too. + assert stream["url"] in text_of(streamed) + assert stream["phrase"] in stream["url"] + assert streamed["structuredContent"]["stream_error"] is None + + body = json.loads( + read(f"hhstream://sessions/{streamed_id}/stream")[0]["text"] + ) + assert body["stream"]["url"] == stream["url"], body + assert body["stream"]["attached"] is True, body + + # And the listing resource names it, with the daemon reachable. + listed = json.loads(read("hhstream://streams")[0]["text"]) + assert listed["available"] is True, listed + assert [row["id"] for row in listed["streams"]] == [streamed_id], listed + except Exception: + dump("session_start --stream over mcp", streamed_id) + raise + + with subtest("disabling over mcp takes the stream off both resources"): + try: + succeeded("session_stream_disable", {"id": streamed_id}, timeout=180) + body = json.loads( + read(f"hhstream://sessions/{streamed_id}/stream")[0]["text"] + ) + assert body["stream"] is None, body + assert json.loads(read("hhstream://streams")[0]["text"])["streams"] == [] + finally: + succeeded("session_kill", {"id": streamed_id}, timeout=300) ''; } ) From a61558a0ade9281599ada8e665c2a538625749ef Mon Sep 17 00:00:00 2001 From: Hermenix Date: Sun, 11 Oct 2026 22:59:45 +0000 Subject: [PATCH 11/12] docs: the two counts that are prose rather than assertions `tools.py`'s module docstring and the module's `mcp.enable` warning both state the size of the surface in words, so neither moved when the stream added to it. Twenty-three tools, and twenty of them plus seven of the ten resources need the broker -- counted rather than carried over: the three that do not are `shot`, `record` and `run`, and `hhstream://status`, `hhstream://screen` and `hhstream://outputs/{name}` beside them. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_011ZeH65zd6pFTgBSu52W9PE --- nix/module.nix | 4 ++-- src/hhstream_mcp/tools.py | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/nix/module.nix b/nix/module.nix index 2896664..35ee3eb 100644 --- a/nix/module.nix +++ b/nix/module.nix @@ -989,8 +989,8 @@ in { # to declare. environment.systemPackages = lib.mkIf cfg.mcp.enable [ cfg.mcp.package ]; - # A warning and not an assertion: eighteen of the twenty-one tools and six - # of the eight resources need the broker, and an agent handed them against + # A warning and not an assertion: twenty of the twenty-three tools and + # seven of the ten resources need the broker, and an agent handed them against # a machine where it is off gets exit 5 from every one of them -- which is # a true answer, badly timed. The frame tools are genuinely useful on their # own, so this is a thing worth saying rather than a thing worth refusing. diff --git a/src/hhstream_mcp/tools.py b/src/hhstream_mcp/tools.py index 8c83df0..f8ea5eb 100644 --- a/src/hhstream_mcp/tools.py +++ b/src/hhstream_mcp/tools.py @@ -31,7 +31,7 @@ with an ``action`` enum. Three reasons, in order of weight: 3. ``readOnlyHint`` is per tool and is still a real distinction among what is left: ``shot`` reads, ``session_kill`` destroys, ``record`` writes a file. -The cost is twenty-one tools in one listing, and :data:`TOOLSETS` is the lever +The cost is twenty-three tools in one listing, and :data:`TOOLSETS` is the lever for a client that cannot afford them all. It is paid back by the names being the CLI's own: an agent that has read ``hhstream session --help``, or this repository's README, already knows what ``session_touch`` is, and a transcript From 9708154eea660118d229865532acb036dde0f33b Mon Sep 17 00:00:00 2001 From: Hermenix Date: Sun, 11 Oct 2026 23:08:13 +0000 Subject: [PATCH 12/12] test(mcp): count the unnarrowed resources against the listing, not a literal `--toolsets` has to leave the resources alone, and the subtest that says so asserted `5` and `3` -- so the stream's two resources turned a correct server into a red check, which is the second literal in this file the surface outgrew in one day. Counted against `uris` and `templates` from the listing subtest instead. That is the comparison the assertion was always about: not "six", but "as many as this server lists when nothing is narrowed". Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_011ZeH65zd6pFTgBSu52W9PE --- checks/vm-mcp.nix | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/checks/vm-mcp.nix b/checks/vm-mcp.nix index a33d75d..cdd0a7b 100644 --- a/checks/vm-mcp.nix +++ b/checks/vm-mcp.nix @@ -519,8 +519,12 @@ testers.runNixOSTest ( timeout=180, ) ) - assert len(still_there["resources"]) == 5, still_there - assert len(still_there["resourceTemplates"]) == 3, still_there + # Every one of them, unnarrowed: `--toolsets` is a tool budget and + # must not take the host's context away with it. Counted against + # the full listing above rather than against a literal, so the next + # resource added cannot make this pass by being forgotten. + assert len(still_there["resources"]) == len(uris), still_there + assert len(still_there["resourceTemplates"]) == len(templates), still_there # And a tool outside the selection must be unreachable, not merely # unlisted, or the flag is cosmetic.