# examples ## `agent-eyes` — the short commands The everyday front door to the session: one subcommand each, no environment juggling, and it re-execs itself under `sg hhstream` when the login predates the group membership. ```bash examples/agent-eyes status # services, socket, group access, liveness probe examples/agent-eyes shot --out /tmp/s.jpg # one frame of the screen examples/agent-eyes run -- chromium ... # launch a GUI app INTO the session examples/agent-eyes record --seconds 10 --out clip.mp4 ``` - `shot` defaults to `/var/lib/hermes/public/agent-eyes/latest.jpg`, which the local file server already publishes, so the newest frame is always one browser refresh away. - `run` sets `XDG_RUNTIME_DIR`, `WAYLAND_DISPLAY` (it discovers whichever `wayland-N` exists rather than assuming `wayland-0`) and `XDG_SESSION_TYPE`, then `exec`s the command. Everything it launches runs as *your* user, not the service user. - `record` reads frames off the socket for the given wall-clock seconds and encodes at the **measured** rate, so the clip plays at true speed. It strips metadata for the media stores, same as the demo. Both `shot` and `record` want nothing but `hhstream-client`, `python3` and `ffmpeg`, which are all on the system PATH; only `chromium` needs `nix develop`. ## The session verbs these helpers do not wrap `agent-eyes` is the *frame* side of the session: look at the screen, put an app on it, film it. Everything that tracks an application, gives it its own screen or drives a cursor at it lives on the `hhstream session` namespace and is not wrapped here, because it needs the broker and `agent-eyes` deliberately does not: ```bash hhstream session start chromium -- https://example.com # prints the bare id hhstream session list # WS, OUTPUT, POINTER hhstream session container # its output and workspace hhstream shot --output hh- --out /tmp/just-that.png # one container alone hhstream session pointer acquire # prints the lease id hhstream session pointer move --to 0.5,0.5 # 0..1, within ITS output hhstream session pointer click hhstream session pointer release ``` Two things worth knowing before using them: - **Coordinates are output-normalised, 0..1**, inside that session's own container — never window-local pixels. The broker's IPC client carries no window geometry, so there is nothing honest to convert from. - **No pointer verb counts as activity.** A session being driven is still a session nobody has said they want kept; only `touch` resets the idle clock. The lease id is cached under `$XDG_RUNTIME_DIR/hhstream`, which is why the gesture verbs need no flag; `--lease` and `$HHSTREAM_LEASE` override it. ## `agent-eyes-demo.py` Puts a real browser on the headless screen, drives it, and films the result by reading frames off the published socket — the consumer side of [`docs/CONTRACT.md`](../docs/CONTRACT.md). ```bash nix develop sg hhstream -c 'python3 examples/agent-eyes-demo.py --query "Ari Sosnovsky"' ``` Output is an MP4 (default `agent-eyes.mp4`): the Google homepage, the query typed in a letter at a time, then the results page — captured from the compositor, not from a screencast API. Everything else is a flag: `--seconds`, `--capture-fps`, `--output`, `--work-dir`, `--keep-work`, `--debug-port`, `--clear-screen`. `--help` is accurate. ### What it checks rather than assumes - the session is up and the socket is readable (a clear message if your user is not in the stream's group); - the browser actually reached a results page — it presses Enter, then waits for the URL to change, and submits the form directly if Enter did nothing; - it *fails* on Google's `/sorry/` bot wall instead of encoding a video of a CAPTCHA, because a video that looks like a working demo and is not one is worse than no video. ### Three things that will bite you 1. **Your shell needs the group.** The socket is `0660 hhstream:hhstream` inside a `0750` directory; a browser also has to reach the session's Wayland socket in the same directory. Add yourself to `hhstream` (this is what `/etc/nixos/hyprland-stream.nix` does for `hermes`) and log in again — or use `sg hhstream -c '...'` for the shell you already have open. 2. **Google rate-limits you.** The first search from a fresh profile usually passes; the third in a minute gets `unusual traffic`. It is the IP, not the browser. Wait a few minutes and re-run. 3. **The frame server was the bottleneck, and it was PNG.** Measured with `grim` at 1280×720 on this software-rendered VM: PNG ~326 ms/frame (a ~3 fps ceiling), PNG `-l 1` ~105 ms, **JPEG q80 46 ms**. The session therefore streams JPEG and the demo records ~22 fps. The script never assumes the format — it takes the extension from the Hello and hands it to ffmpeg — so either setting works. Leave `--capture-fps` at its default of 30 so nothing is thrown away: the encoder uses the *measured* rate, so the video plays at real speed either way, and decimating only makes it choppier. ### It films a desktop, not a browser The browser is **windowed by default** (1040×560, floating, centred), so the session's wallpaper, gaps and window borders are all in shot — that is the difference between "a screen recording of a browser" and "here is the machine I look at". `--fullscreen` gives the old full-bleed kiosk window. The desktop furniture itself lives in `/etc/nixos/hyprland-stream.nix` (wallpaper via `swaybg`, `gaps_out = 36`, a catch-all float window rule), not in this repo: it is this machine's taste, not part of the kit. Two notes for anyone editing that file: - Window rules are **v3** syntax here (`windowrule = match:class .*, float on`). The v2 spelling (`float, class:^(chromium)$`) parses as an unknown field and Hyprland draws the error *across the screen*, which lands in the recording. Always check an edit with `Hyprland --verify-config --config ` — it does not need a display, so it costs seconds instead of a rebuild. - `hyprctl` **works** against this session; an older note here claimed the compositor misnames its IPC socket, which was wrong. The real defect was path length: `$XDG_RUNTIME_DIR` was `/run/hyprland-headless-stream` (29 chars) and the event socket `hypr//.socket2.sock` did not fit in `sun_path`, so the compositor logged `ERR ]: Socket2 path is too long. (2) IPC will not work.` while the request socket kept answering — which is why the wrong diagnosis survived. The runtime directory is now `/run/hypr-stream`, and `nix/module.nix` asserts it stays ≤ 25 characters. ### Posting the result to a chat Xiph/Blossom media stores reject files that carry metadata. `encode()` in the script already strips it with `-map_metadata -1 -fflags +bitexact -flags:v +bitexact`; without those flags a Buzz relay answers `422 media contains metadata or a non-canonical metadata channel` and the upload fails. Uploading from the command line, with the same script-encoded file: ```bash buzz upload file --file agent-eyes.mp4 # -> {"url": ".../media/.mp4"} buzz media get -o check.mp4 "" # fetch it back and compare sha256 ``` `buzz upload file` takes `--file `, not a positional argument, and exits **0** with a JSON error on stdout when the relay refuses the type — read the output, never the exit status.