Commit Graph
5 Commits
Author SHA1 Message Date
Hermenix 7190960120 capture: encode JPEG in-process too, so the live stream keeps working
PNG-only was a real defect, not a nice-to-have: `hhstream_live.server` serves
`multipart/x-mixed-replace` with `image/jpeg` parts and its own capture default is
`grim -c -t jpeg -q 80 -`, so switching this backend on would have broken video
streaming while looking faster in a benchmark.

JPEG now goes through libjpeg-turbo by ctypes, using its `TJPF_BGRX` input format
-- which is exactly the compositor's B, G, R, X memory order, so the frame goes
into the encoder with no copy and no colour conversion. Measured on this VM
against the real compositor, the deployment's own settings (Virtual-1, 1280x720,
jpeg q80, 200 frames each, two runs):

                        p50 ms   fps    our CPU   compositor CPU
    in-process           76.2   13.1     2.1 ms      51.5 ms
    grim -c -t jpeg     137.0    7.3     0.5 ms      80.2 ms
                                                          (second run: 84/144,
                                                           55.1/79.8 -> 1.71x)

So ~1.7-1.8x the frame rate and ~1.5x less of the compositor's CPU, with frames
of the same order (188 KB against grim's 222 KB). PNG stays available through the
stdlib for a deployment that would rather not carry the library.

Both segfault-class traps from the spike are handled and documented: every
argtype is declared (an undeclared pitch is a crash, not a wrong number), and
`*jpegBuf` is reset to NULL after `tjFree` (turbojpeg reads a non-NULL pointer as
a caller buffer, so the next frame would write into freed memory). The decoder is
here because a lossy codec can only be tested by round-tripping pixels -- the new
suite encodes solid blocks whose channels differ enough that a swap is
unmistakable, and includes a padded stride.

Packaging: libjpeg-turbo is reached through ctypes, so nothing links it and the
path must reach the loader -- the console scripts get a wrapper with it on
LD_LIBRARY_PATH, and the unit check exports it too. A missing library is reported
at startup rather than once per frame: a server that is up but can never produce
a frame is worse than one that refuses to start.

Verified end to end: server with `--capture wayland:Virtual-1 --format jpeg`,
client Hello `image/jpeg`, three frames read, each magic `ffd8`.
2026-10-11 22:58:28 +00:00
Hermenix a7d9357b43 capture: an in-process Wayland backend, and the measurements that justify it
The frame server could only capture by running a program once per frame
(grim). That works, but ~95% of a grim frame is waiting -- fork, connect,
handshake, round trip -- not encoding, and grim makes the compositor import
a *new* shared-memory buffer for every single frame.

This adds `--capture wayland[:OUTPUT]`: one long-lived zwlr-screencopy-v1
client per server, with one shm buffer reused for the whole run, capturing
inside the process. Measured here at 1280x720 (spike capture-speed, 200
frames; live re-check on Virtual-1, 60 frames):

    grim (subprocess)    117.3 ms/frame   8.1 fps   compositor 116.4 ms
    grim-rs (Rust crate) 124.6 ms/frame   7.5 fps   compositor 123.1 ms
    this backend          78.0 ms/frame  13.9 fps   compositor  67.5 ms

    live, PNG both sides: this 128.5 ms/frame against grim's 389.3 ms (3.0x)

Three findings, each of which shaped the code:

* The win is buffer reuse, not language. grim and grim-rs both hand the
  compositor a fresh buffer per frame, which is why their compositor CPU is
  nearly double ours. The per-frame allocate-copy-swizzle that grim-rs pays
  was measured in isolation at 2.4 ms of *client* time; the rest of its
  deficit lands inside the compositor.
* ext-image-copy-capture-v1 is no faster (78.0 vs 77.7 ms) and its
  create_frame path appears in a Hyprland segfault stack, so this keeps the
  deprecated protocol grim uses. The switch is one constant when that crash
  is fixed.
* PNG only, deliberately: PNG is the one encoding this package produces
  without a dependency, so --format jpeg is refused rather than publishing a
  Hello whose MIME type the frames do not have. A PNG frame is ~2.1 MB
  against ~120 KB for JPEG q80, which is why the command backend stays the
  default and why a libjpeg-turbo encoder (1.7 ms/frame, measured) is the
  obvious follow-up.

The timeout is the load-bearing part of the client: pywayland only reads
events from read()/dispatch(block=True), neither of which takes a timeout,
and the server captures in a worker thread where SIGALRM is unavailable. It
therefore polls the display fd itself, so a compositor that stops answering
produces a CaptureError instead of a capture thread that never returns.

The protocol bindings are vendored under src/hhstream/_protocol/ exactly the
way hhstream_pointer already vendors its two, with regenerate.py recording
where each XML came from. No new dependency: pywayland was already required
by the input driver.

Contract: the new server flag and the two new module options are documented
in docs/CONTRACT.md, and docs/ARCHITECTURE.md's prediction that a native
screencopy client would be "a swap rather than a rewrite" is now the record
of what the swap cost.

Unit suite: 977 passed, 2 skipped, 0 failed (nix build -L .#checks.unit).
The live numbers were taken on this VM against the real compositor.
2026-10-11 20:34:33 +00:00
HermenixandClaude Opus 5 de408e4b7f the surface: CLI verbs, the module's options, and three checks the compositor votes in
Everything below the CLI was merged and green against fakes; none of it was
reachable and none of it had ever met a compositor. This wires it up and then
puts the compositor in a position to disagree.

CLI (src/hhstream/cli.py, src/hhstream_session/**)

  session container [ID], session reconcile [ID], and the seven pointer verbs
  from PLAN §9. `start` still prints the bare session id and nothing else.
  `list` gains WS / OUTPUT / POINTER, with the gap marker where a value is
  genuinely unknown rather than a blank that reads as zero -- the POINTER
  column needs a second, read-only `pointer.state` call because the `list`
  reply carries no lease state.

  The lease id is cached in `hhstream_session.lease` (under
  $XDG_RUNTIME_DIR), because PLAN §9's gesture verbs take no --lease and the
  wire requires a lease_id on every one of them; --lease and $HHSTREAM_LEASE
  override it, and a `no_lease` refusal drops the stale entry.

  Exit codes are the contract's: a gesture with no lease is 9 and never
  reaches the broker, and a compositor failure is 10 with a message that says
  the session is still running. `kill` now warns about
  `session.container_survivors` -- windows the compositor still shows -- and
  distinguishes `null` ("could not ask") from `[]`.

  `shot`/`record --output NAME` capture one container through `grim -o`; the
  frame socket publishes the main output only, so it cannot do it.

Module (nix/module.nix)

  sessions.containers.{enable,max,mode,workspaceBase},
  sessions.pointer.{enable,leaseTtl,leaseMax}, rendered into the generated
  TOML with §10's key names. hhstream-pointer.service: the operator's account,
  supplementary group hhstream, After=/BindsTo= the compositor,
  Restart=on-failure, KillMode=process, and the command line §3.4 freezes.
  tmpfiles for /run/hhstream plus RuntimeDirectoryPreserve on the broker,
  without which restarting the broker deletes the driver's socket out from
  under it. The runtime-directory length assertion is untouched.

  And ExecStartPost on the compositor, opening its IPC directory to its own
  group: Hyprland creates <runtime>/hypr with S_IRWXU, so the broker -- a
  different account by design -- could not reach the request or event socket
  at all, and degraded silently exactly as it is built to.

Checks (checks/**)

  vm-ipc: both sockets at their frozen names, measured against sun_path's 108
  bytes, and a socket2 subscriber that receives an openwindow for a window the
  test creates. The subscriber is the shipped HyprlandIPC, not a second
  implementation: this check exists to prove that this client hears this
  compositor.

  vm-containers: one hh-* output per session at the container mode, the window
  on it, a second window of the same application beside it, an unmatched
  window quarantined, grim -o isolating one container, teardown removing the
  output and freeing the slot. PLAN experiment 3 is answered: `hyprctl output
  create headless` works.

  vm-pointer: a version-2 virtual pointer, a warp and a click whose
  coordinates the *application* reports, BUSY naming the holder, an expired
  lease swept with its held button lifted, and a broker restart that leaves
  the lock free, the bookmark intact and the driver reachable.

  checks/lib gains the guest the three share, their test-script helpers (no
  assertions live there), an event-socket subscriber and a Wayland client that
  says where it was clicked -- nothing in nixpkgs both honours
  $HHSTREAM_APP_ID and reports pointer events.

Reported, not worked around: the broker reads none of the container or
pointer config (policy.py takes [defaults] and [apps] only), `start` has no
no_container field so --no-container is refused rather than silently ignored,
and nix/hyprland-conf.nix sets input:follow_mouse = 0, which sends pointer
events to the focused window only -- measured, with the cursor at the aimed
pixel and no client told. README.dev.md has the evidence for each.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQy29x9AfHWyzNd1f7XPGK
2026-10-03 06:58:44 +00:00
Hermenix 30f456c07e integrate the session broker: one CLI, a module unit, and a VM check
M0 of the sessions design, wired into the flake and gated by a real check.

What is here now:

- `src/hhstreamd/` — the broker: session registry, lifecycle verbs, process-group
  teardown, app allowlist, audit log, idle reaper on CLOCK_BOOTTIME.
- `src/hhstream_session/` — the CLI handlers, registered into the one front door.
- `src/hhstream/cli.py` — `hhstream status|shot|run|record|session …`, so the
  frame verbs and the session namespace are one binary.
- `nix/module.nix` — `sessions.*` options and the `hhstream-sessions.service`
  unit: the operator's own account (never a service account, because launched
  applications should see that account's home), the compositor's group as a
  supplementary group (without it a launched GUI client cannot open the Wayland
  socket), the generated app allowlist, and `--compositor-runtime-dir` so
  applications are handed the compositor's session rather than the broker's.
- `checks/vm-sessions.nix` — 18 subtests on a real compositor in a VM.

Two bugs found by driving it, both fixed:

- BUG-01: launched applications could not reach the compositor at all — the
  broker spawned them with its own inherited environment, which names the login
  session. `hhstreamd/compositor.py` discovers the socket and builds the child
  environment; a directory with no usable socket is a startup refusal, not a
  warning.
- BUG-02: `kill` refused to signal a live application and said nothing was
  wrong. The identity gate hashed `/proc/<pid>/cmdline`, which every `execve`
  replaces — and on NixOS applications are reached through wrapper scripts that
  do exactly that. The gate now rests on `starttime` (+ boot id, + session id
  where it is immutable) and records command-line drift as evidence of nothing.

KillMode=process is set on the unit on purpose: the default control-group kill
would take the applications down with the broker, which would make adoption and
a broker restart pointless.
2026-10-01 12:23:16 +00:00
Hermenix db886faa38 docs: pin the server command line in the contract; add ARCHITECTURE.md 2026-09-28 11:53:19 +00:00