Files
hyprland-headless-stream-pl…/docs/ARCHITECTURE.md
T
HermenixandClaude Opus 5 887ed17be2 docs(stream): mcp-v3, the seventh box, and two TODOs the check wrote
`docs/CONTRACT-mcp.md` goes to mcp-v3 with a change table: two tools, two
resources, three `session_start` arguments, and the one rule that is not a name
at all -- a connected viewer is the single exception to the idle-clock rule, and
it cannot be invoked, so it is documented in §6 rather than offered in §3. v3 is
additive; every v2 name still means what it meant.

README.md leads the stream with the shape of it rather than the options: one
port for the machine, a phrase per session, a browser `<img>` as the entire
client, and the four things worth knowing before switching one on -- that it is
per session and off by default, that the phrase survives a restart, that capture
runs only while somebody is connected, and that watching counts as activity.
The phrase is called what it is: the only protection, with no TLS and no second
factor.

ARCHITECTURE.md gains the fourth unit as a seventh box -- `hhstream-live`,
hanging off the broker by `live.sock` with one port and a browser at the end of
it -- and the counts move to 23 and 10.

Two TODOs, both written by `checks/vm-stream.nix`:

* `a-stream-that-gave-up-still-reports-attached.md`. The publisher stops a
  stream after ten consecutive capture failures and the stop is sticky on
  purpose, but `Control._describe` does not pass `stopped` on, so
  `_stream_public` reports `attached: true` for a capture that has permanently
  given up. That is the second place `attached` answers "is a capture running
  right now" wrongly, and a field that is right only when nothing is wrong is
  not worth reading.
* `capture-binary-and-the-unit-path.md` is now half fixed, and says so. The
  module passes grim's store path and this check reads real frames -- which
  immediately found the sibling fault, the unit's missing `XDG_RUNTIME_DIR`.
  What is still open is the other half: the app resolves its capture command by
  name and never checks it, so any deployment that is not this module still
  discovers a broken capture ten failures later.

The plan's status section records Stage 2 and Stage 3 as built, the two
decisions §5 left open (that `start` takes `--stream`, and that the default rate
is the daemon's), and what Stage 2 deliberately did not do -- §6's restart
re-attachment, which has its own TODO.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011ZeH65zd6pFTgBSu52W9PE
2026-10-11 22:54:26 +00:00

184 lines
12 KiB
Markdown

# Architecture
```
┌──────────────────────────────────────────┐
│ hyprland-headless.service │
│ Hyprland 0.56.2, no physical monitor │
│ private XDG_RUNTIME_DIR, as `user` │
│ outputs come from `outputs[]` │
│ + one headless output per session │
└───┬───────────┬───────────────┬──────────┘
│ │ │
IPC: request / │ │ Wayland │ Wayland
event sockets │ │ session │ (virtual pointer)
│ ▼ │
│ ┌────────────────────────┴─────────┐
│ │ hyprland-headless-stream.service │
│ │ hhstream-server │
│ │ capture ──▶ one frame slot │
│ │ accept ──▶ thread per client│
│ └───────────────┬──────────────────┘
│ │ AF_UNIX, mode 0660
│ ▼
│ many clients ── hhstream-client / any reader
│
┌────────────┴──────────────┐ ┌──────────────────────────┐
│ hhstream-sessions.service │─────▶│ hhstream-pointer.service │
│ hhstreamd │ │ one Wayland connection │
│ registry, containers, │ JSON │ one virtual pointer │
│ the pointer lease, │ over │ per output │
│ the stream phrases │ a └──────────────────────────┘
└────────────┬────────┬─────┘ private socket
│ │ ┌──────────────────────────┐
│ └─────────────▶│ hhstream-live.service │
│ live.sock │ one port (9988) │
│ │ grim ─▶ MJPEG per │
│ │ phrase, while watched │
│ └───────────┬──────────────┘
│ │ HTTP
│ a browser ┘ /<phrase>/
│ AF_UNIX control.sock, mode 0660
▼
hhstream session ... (the CLI an agent drives)
▲
│ in-process: argv in, exit code out
│
┌────────────┴──────────────┐
│ hhstream-mcp │ spawned per client, stdio only
│ 23 tools (the acts) │◀─── an MCP client's JSON-RPC
│ 10 resources (the state)│
└───────────────────────────┘
```
Six pieces, deliberately separate, and a seventh that is deliberately not a
piece at all:
1. **The compositor** (`Hyprland`, unchanged upstream) renders into virtual
outputs. It knows nothing about frames on a socket.
2. **The frame server** (`hhstream`) captures, holds exactly one frame, and
publishes it. It owns the socket, the framing and the drop policy.
3. **The client** is any process that can open a Unix socket and read the
contract's byte format. The shipped `hhstream-client` is a convenience, not a
requirement.
4. **The session broker** (`hhstreamd`) starts allowlisted applications as
tracked sessions, gives each one its own headless output and workspace (a
*container*), reads the compositor back over its own IPC, and owns the
pointer lease. It is the only part that keeps state across a restart.
5. **The pointer driver** (`hhstream-pointer`) is a Wayland client of the same
session that creates one `zwlr_virtual_pointer_v1` per output. It holds no
lease and no session state: the broker decides, the driver executes. It is
a separate process because a Wayland client needs its own connection and a
driver blocked on the compositor must not stall session lifecycle.
6. **The stream daemon** (`hhstream-live`) publishes one session's output as
MJPEG on one port, behind a four-word phrase, while somebody is connected.
A third executor on the pointer driver's pattern and for its reasons: `grim`
blocks tens to hundreds of milliseconds so it must not run where `touch` and
`kill` are answered, and a LAN-facing HTTP parser must not run in the frame
server whose one-socket contract cannot grow one. The broker owns the
*decision* (which session, under which phrase, in the record); the daemon
owns the port, the capture and the URL. It knows nothing about sessions or
idle clocks — it publishes what it is told to publish, and reports a viewer
count the broker chooses to read as activity.
And the seventh, which is a *shim* and not a layer:
7. **The MCP server** (`hhstream-mcp`, `docs/CONTRACT-mcp.md`) is the CLI with a
Model Context Protocol surface on it, so an agent reads state by URI and
acts by calling tools instead of writing command lines. It is drawn above
the CLI rather than beside the broker because that is where it sits: every
tool and every resource builds the argv list the CLI's parser takes and runs
the CLI's own handler **in the same process**, and it is the one component
here that owns no state at all — not even the pointer lease, which is an
explicit argument on every gesture tool. Add it to the diagram's list of
things that can go wrong and nothing new appears: every failure it reports
is the CLI's or the broker's, carrying their exit code.
The split between its two halves is *who decides*, not what it costs. A
resource is the host's to include (`hhstream://sessions` is state anyone may
want in context); a tool is the model's to call (`session_kill` is a
decision). That is why `shot` is both — a tool because deciding to look is
an act, and `hhstream://screen` because a frame is also just state.
It is also the only part with a third-party dependency (the `mcp` SDK),
which is why it is a separate derivation (`nix/mcp.nix`) and why
`packages.hhstream` — the compositor's own units — keeps the closure it had.
## Why containers are outputs
A session's container is a headless **output**, not just a workspace, and the
two consequences are the reason:
- `grim -o hh-<id>` captures exactly one session, so one agent's screenshot
cannot contain another's window;
- `zwlr_virtual_pointer_manager_v1` version 2 creates a pointer *bound to an
output*, so a warp to (0.5, 0.5) is the middle of that session's screen and
not the middle of the desktop. Version 1 is refused rather than used,
because its coordinates are global and every click would land somewhere
plausible and wrong.
The cursor itself is singular — there is one seat — so pointer access is
serialised by a broker-wide **lease**: one holder, a TTL the broker enforces
with its own sweeper, and buttons lifted on release or expiry. The lease
disciplines these sessions; it does not enforce against a hostile client,
which can create its own virtual pointer (that is what sandboxing, not the
lease, would fix).
## Why one frame, not a queue
The server keeps a single slot holding the most recent frame. Each client thread
sends whatever is in that slot when it is ready. A client that cannot keep up
therefore misses frames instead of forcing the server to buffer them: memory use
does not grow with the number of stalled clients, and a stalled client cannot
delay the others. See `docs/CONTRACT.md` for exactly what a client observes.
## Capture
Capturing is a separate program (by default `grim`, a wlr-screencopy client) run
inside the compositor's session and pointed at stdout. That choice is what keeps
`hhstream` free of Wayland and GPU code — the awkward part lives in a tool that
already exists and is already tested — and it is why the capture command is a
module option: tests inject a fake, and a future backend (a native screencopy
client, a hardware encoder) is a command-line swap rather than a rewrite.
## The hardware requirement, stated plainly
`Hyprland` 0.56.2 asks aquamarine for its **headless backend as mandatory**
(`src/Compositor.cpp`) and does not accept the DRM backend if it is
unavailable. Aquamarine's headless backend still needs an EGL/GBM **render
node**; there is no software-only path that works without one. Consequently a
host with no graphics device at all (`/dev/dri` missing) cannot run this, and
Pyprland fails with `CBackend::create() failed!` before any of its managers come
up.
So the requirements are:
- a render node, i.e. `/dev/dri/renderD*` — from a real GPU, a virtual GPU
(`virtio-gpu`, `virtio-gpu-gl`, QXL) or a NixOS test VM's own QEMU display;
- the `video`/`render` groups for whichever user runs it (the module sets them).
## Testing, and what each layer proves
| Layer | Proves |
|---|---|
| `tests/unit` | the framing, the capture wrapper and the server's client handling, with no compositor and no GPU |
| `checks.vm-basic` | the module actually starts both units, the socket appears with the right owner/mode, the Hello parses and two frames arrive — with a fake capture program |
| `checks.vm-hyprland` | the real thing: headless Hyprland comes up in the VM and `grim` captures a frame of the configured size |
| `checks.vm-sessions` | the session broker side: a real application runs as a tracked session on the compositor, the compositor's IPC sees its window, the idle reaper fires for real (and does not fire when the session is touched), and a broker restart adopts its sessions back |
| `checks.vm-ipc` | both IPC sockets exist at the names the contract freezes and fit in `sun_path`; `hyprctl` answers; and a subscriber on the **event** socket receives an `openwindow` for a window the test creates — the one assertion that can tell a working event socket from a silently truncated one |
| `checks.vm-containers` | one `hh-*` output per session at the container mode, the application's window on it, a second window of the same application beside the first, an unmatched window quarantined instead of landing on whoever had focus, `grim -o` isolating one container, and teardown removing the output and releasing its workspace slot |
| `checks.vm-pointer` | the driver binds a version-2 virtual pointer; acquire warps into the container and restores from the bookmark; a click arrives at the application, which reports the coordinates it was delivered at; a second acquire is BUSY naming the holder; an expired lease is swept and its held button lifted; a broker restart leaves the lock free, the bookmark intact and the driver reachable |
The three session checks share one guest (`checks/lib/session-machine.nix`) and
one set of test-script helpers (`checks/lib/session-prelude.nix`); the helpers
contain no assertions, so each check's script is still a complete account of
what it proves.
`vm-hyprland` is the only check that can fail for reasons outside this repo, so
it is kept separate from `vm-basic` on purpose: when it fails, the compositor's
log is the answer and the other two layers are already known good.
**Run the VM checks one at a time.** Each boots a guest with a
software-rendered compositor; two at once on a two-core machine make the
timing-sensitive assertions flake, and a flaky green is worth less than a slow
one.