One front door to the headless session, for reuse by any consumer: it discovers whichever wayland-N socket exists, re-execs itself under sg hhstream when the login predates the group membership, and encodes recordings at the measured frame rate so clips play at true speed.
117 lines
5.6 KiB
Markdown
117 lines
5.6 KiB
Markdown
# examples
|
||
|
||
## `agent-eyes` — the short commands
|
||
|
||
The everyday front door to the session: one subcommand each, no environment
|
||
juggling, and it re-execs itself under `sg hhstream` when the login predates the
|
||
group membership.
|
||
|
||
```bash
|
||
examples/agent-eyes status # services, socket, group access, liveness probe
|
||
examples/agent-eyes shot --out /tmp/s.jpg # one frame of the screen
|
||
examples/agent-eyes run -- chromium ... # launch a GUI app INTO the session
|
||
examples/agent-eyes record --seconds 10 --out clip.mp4
|
||
```
|
||
|
||
- `shot` defaults to `/var/lib/hermes/public/agent-eyes/latest.jpg`, which the
|
||
local file server already publishes, so the newest frame is always one
|
||
browser refresh away.
|
||
- `run` sets `XDG_RUNTIME_DIR`, `WAYLAND_DISPLAY` (it discovers whichever
|
||
`wayland-N` exists rather than assuming `wayland-0`) and `XDG_SESSION_TYPE`,
|
||
then `exec`s the command. Everything it launches runs as *your* user, not the
|
||
service user.
|
||
- `record` reads frames off the socket for the given wall-clock seconds and
|
||
encodes at the **measured** rate, so the clip plays at true speed. It strips
|
||
metadata for the media stores, same as the demo.
|
||
|
||
Both `shot` and `record` want nothing but `hhstream-client`, `python3` and
|
||
`ffmpeg`, which are all on the system PATH; only `chromium` needs `nix develop`.
|
||
|
||
## `agent-eyes-demo.py`
|
||
|
||
Puts a real browser on the headless screen, drives it, and films the result by
|
||
reading frames off the published socket — the consumer side of
|
||
[`docs/CONTRACT.md`](../docs/CONTRACT.md).
|
||
|
||
```bash
|
||
nix develop
|
||
sg hhstream -c 'python3 examples/agent-eyes-demo.py --query "Ari Sosnovsky"'
|
||
```
|
||
|
||
Output is an MP4 (default `agent-eyes.mp4`): the Google homepage, the query
|
||
typed in a letter at a time, then the results page — captured from the
|
||
compositor, not from a screencast API.
|
||
|
||
Everything else is a flag: `--seconds`, `--capture-fps`, `--output`,
|
||
`--work-dir`, `--keep-work`, `--debug-port`, `--clear-screen`. `--help` is
|
||
accurate.
|
||
|
||
### What it checks rather than assumes
|
||
|
||
- the session is up and the socket is readable (a clear message if your user is
|
||
not in the stream's group);
|
||
- the browser actually reached a results page — it presses Enter, then waits for
|
||
the URL to change, and submits the form directly if Enter did nothing;
|
||
- it *fails* on Google's `/sorry/` bot wall instead of encoding a video of a
|
||
CAPTCHA, because a video that looks like a working demo and is not one is
|
||
worse than no video.
|
||
|
||
### Three things that will bite you
|
||
|
||
1. **Your shell needs the group.** The socket is `0660 hhstream:hhstream` inside
|
||
a `0750` directory; a browser also has to reach the session's Wayland socket
|
||
in the same directory. Add yourself to `hhstream` (this is what
|
||
`/etc/nixos/hyprland-stream.nix` does for `hermes`) and log in again — or use
|
||
`sg hhstream -c '...'` for the shell you already have open.
|
||
2. **Google rate-limits you.** The first search from a fresh profile usually
|
||
passes; the third in a minute gets `unusual traffic`. It is the IP, not the
|
||
browser. Wait a few minutes and re-run.
|
||
3. **The frame server was the bottleneck, and it was PNG.** Measured with `grim`
|
||
at 1280×720 on this software-rendered VM: PNG ~326 ms/frame (a ~3 fps
|
||
ceiling), PNG `-l 1` ~105 ms, **JPEG q80 46 ms**. The session therefore
|
||
streams JPEG and the demo records ~22 fps. The script never assumes the
|
||
format — it takes the extension from the Hello and hands it to ffmpeg — so
|
||
either setting works. Leave `--capture-fps` at its default of 30 so nothing
|
||
is thrown away: the encoder uses the *measured* rate, so the video plays at
|
||
real speed either way, and decimating only makes it choppier.
|
||
|
||
### It films a desktop, not a browser
|
||
|
||
The browser is **windowed by default** (1040×560, floating, centred), so the
|
||
session's wallpaper, gaps and window borders are all in shot — that is the
|
||
difference between "a screen recording of a browser" and "here is the machine I
|
||
look at". `--fullscreen` gives the old full-bleed kiosk window.
|
||
|
||
The desktop furniture itself lives in `/etc/nixos/hyprland-stream.nix` (wallpaper
|
||
via `swaybg`, `gaps_out = 36`, a catch-all float window rule), not in this repo:
|
||
it is this machine's taste, not part of the kit.
|
||
|
||
Two notes for anyone editing that file:
|
||
|
||
- Window rules are **v3** syntax here (`windowrule = match:class .*, float on`).
|
||
The v2 spelling (`float, class:^(chromium)$`) parses as an unknown field and
|
||
Hyprland draws the error *across the screen*, which lands in the recording.
|
||
Always check an edit with `Hyprland --verify-config --config <file>` — it does
|
||
not need a display, so it costs seconds instead of a rebuild.
|
||
- `hyprctl` **cannot connect** to this session: the compositor names its IPC
|
||
socket `.socket.so`/`.socket.s` instead of `.socket.sock`, so nothing using the
|
||
documented path works. Do not plan on `hyprctl` for window control — measure
|
||
what you need from the frames instead.
|
||
|
||
### Posting the result to a chat
|
||
|
||
Xiph/Blossom media stores reject files that carry metadata. `encode()` in the
|
||
script already strips it with `-map_metadata -1 -fflags +bitexact -flags:v
|
||
+bitexact`; without those flags a Buzz relay answers
|
||
`422 media contains metadata or a non-canonical metadata channel` and the upload
|
||
fails. Uploading from the command line, with the same script-encoded file:
|
||
|
||
```bash
|
||
buzz upload file --file agent-eyes.mp4 # -> {"url": ".../media/<sha256>.mp4"}
|
||
buzz media get -o check.mp4 "<url>" # fetch it back and compare sha256
|
||
```
|
||
|
||
`buzz upload file` takes `--file <path>`, not a positional argument, and exits
|
||
**0** with a JSON error on stdout when the relay refuses the type — read the
|
||
output, never the exit status.
|