Files
HermenixandClaude Opus 5 de408e4b7f the surface: CLI verbs, the module's options, and three checks the compositor votes in
Everything below the CLI was merged and green against fakes; none of it was
reachable and none of it had ever met a compositor. This wires it up and then
puts the compositor in a position to disagree.

CLI (src/hhstream/cli.py, src/hhstream_session/**)

  session container [ID], session reconcile [ID], and the seven pointer verbs
  from PLAN §9. `start` still prints the bare session id and nothing else.
  `list` gains WS / OUTPUT / POINTER, with the gap marker where a value is
  genuinely unknown rather than a blank that reads as zero -- the POINTER
  column needs a second, read-only `pointer.state` call because the `list`
  reply carries no lease state.

  The lease id is cached in `hhstream_session.lease` (under
  $XDG_RUNTIME_DIR), because PLAN §9's gesture verbs take no --lease and the
  wire requires a lease_id on every one of them; --lease and $HHSTREAM_LEASE
  override it, and a `no_lease` refusal drops the stale entry.

  Exit codes are the contract's: a gesture with no lease is 9 and never
  reaches the broker, and a compositor failure is 10 with a message that says
  the session is still running. `kill` now warns about
  `session.container_survivors` -- windows the compositor still shows -- and
  distinguishes `null` ("could not ask") from `[]`.

  `shot`/`record --output NAME` capture one container through `grim -o`; the
  frame socket publishes the main output only, so it cannot do it.

Module (nix/module.nix)

  sessions.containers.{enable,max,mode,workspaceBase},
  sessions.pointer.{enable,leaseTtl,leaseMax}, rendered into the generated
  TOML with §10's key names. hhstream-pointer.service: the operator's account,
  supplementary group hhstream, After=/BindsTo= the compositor,
  Restart=on-failure, KillMode=process, and the command line §3.4 freezes.
  tmpfiles for /run/hhstream plus RuntimeDirectoryPreserve on the broker,
  without which restarting the broker deletes the driver's socket out from
  under it. The runtime-directory length assertion is untouched.

  And ExecStartPost on the compositor, opening its IPC directory to its own
  group: Hyprland creates <runtime>/hypr with S_IRWXU, so the broker -- a
  different account by design -- could not reach the request or event socket
  at all, and degraded silently exactly as it is built to.

Checks (checks/**)

  vm-ipc: both sockets at their frozen names, measured against sun_path's 108
  bytes, and a socket2 subscriber that receives an openwindow for a window the
  test creates. The subscriber is the shipped HyprlandIPC, not a second
  implementation: this check exists to prove that this client hears this
  compositor.

  vm-containers: one hh-* output per session at the container mode, the window
  on it, a second window of the same application beside it, an unmatched
  window quarantined, grim -o isolating one container, teardown removing the
  output and freeing the slot. PLAN experiment 3 is answered: `hyprctl output
  create headless` works.

  vm-pointer: a version-2 virtual pointer, a warp and a click whose
  coordinates the *application* reports, BUSY naming the holder, an expired
  lease swept with its held button lifted, and a broker restart that leaves
  the lock free, the bookmark intact and the driver reachable.

  checks/lib gains the guest the three share, their test-script helpers (no
  assertions live there), an event-socket subscriber and a Wayland client that
  says where it was clicked -- nothing in nixpkgs both honours
  $HHSTREAM_APP_ID and reports pointer events.

Reported, not worked around: the broker reads none of the container or
pointer config (policy.py takes [defaults] and [apps] only), `start` has no
no_container field so --no-container is refused rather than silently ignored,
and nix/hyprland-conf.nix sets input:follow_mouse = 0, which sends pointer
events to the focused window only -- measured, with the cursor at the aimed
pixel and no client told. README.dev.md has the evidence for each.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQy29x9AfHWyzNd1f7XPGK
2026-10-03 06:58:44 +00:00

152 lines
7.4 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# examples
## `agent-eyes` — the short commands
The everyday front door to the session: one subcommand each, no environment
juggling, and it re-execs itself under `sg hhstream` when the login predates the
group membership.
```bash
examples/agent-eyes status # services, socket, group access, liveness probe
examples/agent-eyes shot --out /tmp/s.jpg # one frame of the screen
examples/agent-eyes run -- chromium ... # launch a GUI app INTO the session
examples/agent-eyes record --seconds 10 --out clip.mp4
```
- `shot` defaults to `/var/lib/hermes/public/agent-eyes/latest.jpg`, which the
local file server already publishes, so the newest frame is always one
browser refresh away.
- `run` sets `XDG_RUNTIME_DIR`, `WAYLAND_DISPLAY` (it discovers whichever
`wayland-N` exists rather than assuming `wayland-0`) and `XDG_SESSION_TYPE`,
then `exec`s the command. Everything it launches runs as *your* user, not the
service user.
- `record` reads frames off the socket for the given wall-clock seconds and
encodes at the **measured** rate, so the clip plays at true speed. It strips
metadata for the media stores, same as the demo.
Both `shot` and `record` want nothing but `hhstream-client`, `python3` and
`ffmpeg`, which are all on the system PATH; only `chromium` needs `nix develop`.
## The session verbs these helpers do not wrap
`agent-eyes` is the *frame* side of the session: look at the screen, put an app
on it, film it. Everything that tracks an application, gives it its own screen
or drives a cursor at it lives on the `hhstream session` namespace and is not
wrapped here, because it needs the broker and `agent-eyes` deliberately does
not:
```bash
hhstream session start chromium -- https://example.com # prints the bare id
hhstream session list # WS, OUTPUT, POINTER
hhstream session container <id> # its output and workspace
hhstream shot --output hh-<id> --out /tmp/just-that.png # one container alone
hhstream session pointer acquire <id> # prints the lease id
hhstream session pointer move <id> --to 0.5,0.5 # 0..1, within ITS output
hhstream session pointer click <id>
hhstream session pointer release <id>
```
Two things worth knowing before using them:
- **Coordinates are output-normalised, 0..1**, inside that session's own
container — never window-local pixels. The broker's IPC client carries no
window geometry, so there is nothing honest to convert from.
- **No pointer verb counts as activity.** A session being driven is still a
session nobody has said they want kept; only `touch` resets the idle clock.
The lease id is cached under `$XDG_RUNTIME_DIR/hhstream`, which is why the
gesture verbs need no flag; `--lease` and `$HHSTREAM_LEASE` override it.
## `agent-eyes-demo.py`
Puts a real browser on the headless screen, drives it, and films the result by
reading frames off the published socket — the consumer side of
[`docs/CONTRACT.md`](../docs/CONTRACT.md).
```bash
nix develop
sg hhstream -c 'python3 examples/agent-eyes-demo.py --query "Ari Sosnovsky"'
```
Output is an MP4 (default `agent-eyes.mp4`): the Google homepage, the query
typed in a letter at a time, then the results page — captured from the
compositor, not from a screencast API.
Everything else is a flag: `--seconds`, `--capture-fps`, `--output`,
`--work-dir`, `--keep-work`, `--debug-port`, `--clear-screen`. `--help` is
accurate.
### What it checks rather than assumes
- the session is up and the socket is readable (a clear message if your user is
not in the stream's group);
- the browser actually reached a results page — it presses Enter, then waits for
the URL to change, and submits the form directly if Enter did nothing;
- it *fails* on Google's `/sorry/` bot wall instead of encoding a video of a
CAPTCHA, because a video that looks like a working demo and is not one is
worse than no video.
### Three things that will bite you
1. **Your shell needs the group.** The socket is `0660 hhstream:hhstream` inside
a `0750` directory; a browser also has to reach the session's Wayland socket
in the same directory. Add yourself to `hhstream` (this is what
`/etc/nixos/hyprland-stream.nix` does for `hermes`) and log in again — or use
`sg hhstream -c '...'` for the shell you already have open.
2. **Google rate-limits you.** The first search from a fresh profile usually
passes; the third in a minute gets `unusual traffic`. It is the IP, not the
browser. Wait a few minutes and re-run.
3. **The frame server was the bottleneck, and it was PNG.** Measured with `grim`
at 1280×720 on this software-rendered VM: PNG ~326 ms/frame (a ~3 fps
ceiling), PNG `-l 1` ~105 ms, **JPEG q80 46 ms**. The session therefore
streams JPEG and the demo records ~22 fps. The script never assumes the
format — it takes the extension from the Hello and hands it to ffmpeg — so
either setting works. Leave `--capture-fps` at its default of 30 so nothing
is thrown away: the encoder uses the *measured* rate, so the video plays at
real speed either way, and decimating only makes it choppier.
### It films a desktop, not a browser
The browser is **windowed by default** (1040×560, floating, centred), so the
session's wallpaper, gaps and window borders are all in shot — that is the
difference between "a screen recording of a browser" and "here is the machine I
look at". `--fullscreen` gives the old full-bleed kiosk window.
The desktop furniture itself lives in `/etc/nixos/hyprland-stream.nix` (wallpaper
via `swaybg`, `gaps_out = 36`, a catch-all float window rule), not in this repo:
it is this machine's taste, not part of the kit.
Two notes for anyone editing that file:
- Window rules are **v3** syntax here (`windowrule = match:class .*, float on`).
The v2 spelling (`float, class:^(chromium)$`) parses as an unknown field and
Hyprland draws the error *across the screen*, which lands in the recording.
Always check an edit with `Hyprland --verify-config --config <file>` — it does
not need a display, so it costs seconds instead of a rebuild.
- `hyprctl` **works** against this session; an older note here claimed the
compositor misnames its IPC socket, which was wrong. The real defect was path
length: `$XDG_RUNTIME_DIR` was `/run/hyprland-headless-stream` (29 chars) and
the event socket `hypr/<signature>/.socket2.sock` did not fit in `sun_path`, so
the compositor logged `ERR ]: Socket2 path is too long. (2) IPC will not work.`
while the request socket kept answering — which is why the wrong diagnosis
survived. The runtime directory is now `/run/hypr-stream`, and `nix/module.nix`
asserts it stays ≤ 25 characters.
### Posting the result to a chat
Xiph/Blossom media stores reject files that carry metadata. `encode()` in the
script already strips it with `-map_metadata -1 -fflags +bitexact -flags:v
+bitexact`; without those flags a Buzz relay answers
`422 media contains metadata or a non-canonical metadata channel` and the upload
fails. Uploading from the command line, with the same script-encoded file:
```bash
buzz upload file --file agent-eyes.mp4 # -> {"url": ".../media/<sha256>.mp4"}
buzz media get -o check.mp4 "<url>" # fetch it back and compare sha256
```
`buzz upload file` takes `--file <path>`, not a positional argument, and exits
**0** with a JSON error on stdout when the relay refuses the type — read the
output, never the exit status.