Files
hyprland-headless-stream-pl…/examples
Hermenix 4b5cb5197a docs+examples: log the cursor marker as open, and film each desk on its own
The marker is now written up as the *open* item rather than a research note: it
is not implemented, the video went out without it, and everything else about the
pointer is exact and readable.

proof-video.py is rewritten for desks, one command, no arguments. It films the
session's own output with `record --output` instead of the published socket --
with desks, the socket carries every output stitched together, so a viewer would
watch two thirds wallpaper. The scrolling beats are real pointer axis events,
down twice and back up once, so the movement cannot be mistaken for a page that
merely loaded a different part of itself.
2026-10-03 19:54:08 +00:00
..

examples

agent-eyes — the short commands

The everyday front door to the session: one subcommand each, no environment juggling, and it re-execs itself under sg hhstream when the login predates the group membership.

examples/agent-eyes status                    # services, socket, group access, liveness probe
examples/agent-eyes shot --out /tmp/s.jpg     # one frame of the screen
examples/agent-eyes run -- chromium ...       # launch a GUI app INTO the session
examples/agent-eyes record --seconds 10 --out clip.mp4
  • shot defaults to /var/lib/hermes/public/agent-eyes/latest.jpg, which the local file server already publishes, so the newest frame is always one browser refresh away.
  • run sets XDG_RUNTIME_DIR, WAYLAND_DISPLAY (it discovers whichever wayland-N exists rather than assuming wayland-0) and XDG_SESSION_TYPE, then execs the command. Everything it launches runs as your user, not the service user.
  • record reads frames off the socket for the given wall-clock seconds and encodes at the measured rate, so the clip plays at true speed. It strips metadata for the media stores, same as the demo.

Both shot and record want nothing but hhstream-client, python3 and ffmpeg, which are all on the system PATH; only chromium needs nix develop.

The session verbs these helpers do not wrap

agent-eyes is the frame side of the session: look at the screen, put an app on it, film it. Everything that tracks an application, gives it its own screen or drives a cursor at it lives on the hhstream session namespace and is not wrapped here, because it needs the broker and agent-eyes deliberately does not:

hhstream session start chromium -- https://example.com   # prints the bare id
hhstream session list                                    # WS, OUTPUT, POINTER
hhstream session container <id>                          # its output and workspace
hhstream shot --output hh-<id> --out /tmp/just-that.png  # one container alone

hhstream session pointer acquire <id>                    # prints the lease id
hhstream session pointer move <id> --to 0.5,0.5          # 0..1, within ITS output
hhstream session pointer click <id>
hhstream session pointer release <id>

Two things worth knowing before using them:

  • Coordinates are output-normalised, 0..1, inside that session's own container — never window-local pixels. The broker's IPC client carries no window geometry, so there is nothing honest to convert from.
  • No pointer verb counts as activity. A session being driven is still a session nobody has said they want kept; only touch resets the idle clock.

The lease id is cached under $XDG_RUNTIME_DIR/hhstream, which is why the gesture verbs need no flag; --lease and $HHSTREAM_LEASE override it.

agent-eyes-demo.py

Puts a real browser on the headless screen, drives it, and films the result by reading frames off the published socket — the consumer side of docs/CONTRACT.md.

nix develop
sg hhstream -c 'python3 examples/agent-eyes-demo.py --query "Ari Sosnovsky"'

Output is an MP4 (default agent-eyes.mp4): the Google homepage, the query typed in a letter at a time, then the results page — captured from the compositor, not from a screencast API.

Everything else is a flag: --seconds, --capture-fps, --output, --work-dir, --keep-work, --debug-port, --clear-screen. --help is accurate.

What it checks rather than assumes

  • the session is up and the socket is readable (a clear message if your user is not in the stream's group);
  • the browser actually reached a results page — it presses Enter, then waits for the URL to change, and submits the form directly if Enter did nothing;
  • it fails on Google's /sorry/ bot wall instead of encoding a video of a CAPTCHA, because a video that looks like a working demo and is not one is worse than no video.

Three things that will bite you

  1. Your shell needs the group. The socket is 0660 hhstream:hhstream inside a 0750 directory; a browser also has to reach the session's Wayland socket in the same directory. Add yourself to hhstream (this is what /etc/nixos/hyprland-stream.nix does for hermes) and log in again — or use sg hhstream -c '...' for the shell you already have open.
  2. Google rate-limits you. The first search from a fresh profile usually passes; the third in a minute gets unusual traffic. It is the IP, not the browser. Wait a few minutes and re-run.
  3. The frame server was the bottleneck, and it was PNG. Measured with grim at 1280×720 on this software-rendered VM: PNG ~326 ms/frame (a ~3 fps ceiling), PNG -l 1 ~105 ms, JPEG q80 46 ms. The session therefore streams JPEG and the demo records ~22 fps. The script never assumes the format — it takes the extension from the Hello and hands it to ffmpeg — so either setting works. Leave --capture-fps at its default of 30 so nothing is thrown away: the encoder uses the measured rate, so the video plays at real speed either way, and decimating only makes it choppier.

It films a desktop, not a browser

The browser is windowed by default (1040×560, floating, centred), so the session's wallpaper, gaps and window borders are all in shot — that is the difference between "a screen recording of a browser" and "here is the machine I look at". --fullscreen gives the old full-bleed kiosk window.

The desktop furniture itself lives in /etc/nixos/hyprland-stream.nix (wallpaper via swaybg, gaps_out = 36, a catch-all float window rule), not in this repo: it is this machine's taste, not part of the kit.

Two notes for anyone editing that file:

  • Window rules are v3 syntax here (windowrule = match:class .*, float on). The v2 spelling (float, class:^(chromium)$) parses as an unknown field and Hyprland draws the error across the screen, which lands in the recording. Always check an edit with Hyprland --verify-config --config <file> — it does not need a display, so it costs seconds instead of a rebuild.
  • hyprctl works against this session; an older note here claimed the compositor misnames its IPC socket, which was wrong. The real defect was path length: $XDG_RUNTIME_DIR was /run/hyprland-headless-stream (29 chars) and the event socket hypr/<signature>/.socket2.sock did not fit in sun_path, so the compositor logged ERR ]: Socket2 path is too long. (2) IPC will not work. while the request socket kept answering — which is why the wrong diagnosis survived. The runtime directory is now /run/hypr-stream, and nix/module.nix asserts it stays ≤ 25 characters.

Posting the result to a chat

Xiph/Blossom media stores reject files that carry metadata. encode() in the script already strips it with -map_metadata -1 -fflags +bitexact -flags:v +bitexact; without those flags a Buzz relay answers 422 media contains metadata or a non-canonical metadata channel and the upload fails. Uploading from the command line, with the same script-encoded file:

buzz upload file --file agent-eyes.mp4          # -> {"url": ".../media/<sha256>.mp4"}
buzz media get -o check.mp4 "<url>"             # fetch it back and compare sha256

buzz upload file takes --file <path>, not a positional argument, and exits 0 with a JSON error on stdout when the relay refuses the type — read the output, never the exit status.