Compare commits

..
Author SHA1 Message Date
Hermenix d770bcd298 feat(live): one port, a router, and a phrase per session (stage 1)
The session-stream web app: a single HTTP listener (9988) that serves a page
per enabled session, keyed by a spoken phrase.

  src/hhstream_live/phrase.py     four EFF-large words + a 5-char base-36 tail
                                  (~77.5 bits); the phrase is the password
  src/hhstream_live/publisher.py  one capture thread + one frame slot per
                                  output, one slot/no queue, capture paused at
                                  zero viewers
  src/hhstream_live/server.py     the router: /, /<phrase>/, /stream.mjpg,
                                  /frame.jpg, /status.json; unknown phrase is
                                  an empty 404, rate-limited
  src/hhstream_live/pages.py      the built-in dark pages (no CDN, no build)
  src/hhstream_live/__main__.py   hhstream-live serve|phrase, standalone

The root page is a form, never a list: the phrase is known only by whoever
enabled the stream. Tests: tests/live (51), all green with the full suite
(958). Not yet wired to the broker -- that is stage 2.
2026-10-06 13:26:02 +00:00
Hermenix edbf15f259 Merge keyboard input (feat/keyboard-input @561e0c2) for a live trial on this machine
The merge request stays open for review; this is a local trial only.
Undo: git reset --hard 02d5006, then rebuild.
2026-10-05 12:31:41 +00:00
Hermenix 02d5006524 docs(plan): one port, a router, and a phrase per session
The operator rejected the port-range/token shape: a single port (9988) with
a router, four routes per session output, and a spoken phrase as the key.
Streams are opt-in per session (default off) and capture only while watched.
Phrase is four EFF-large words (~51.6 bits); index is loopback-only by
default. Supersedes 2026-10-03-network-stream.md.
2026-10-04 00:30:21 +00:00
Hermenix deccfb592c docs(plan): a live network stream per session — approach and open questions 2026-10-03 23:55:44 +00:00
25 changed files with 94 additions and 3400 deletions

No files matched your search

-1
View File
@@ -1 +0,0 @@
use flake
-7
View File
@@ -48,9 +48,6 @@ and the pointer driver.
| `vm-ipc` | the **compositor's IPC**: both sockets exist at the names `docs/CONTRACT-sessions.md` §5 freezes and fit inside `sun_path`, `hyprctl` answers, the broker's own `HyprlandIPC` reaches it, and a subscriber on the **event** socket receives an `openwindow` for a window the test creates | yes |
| `vm-containers` | one `hh-*` output per session at the container mode, the application's window on it, a second window of the same application beside it, an unmatched window **quarantined**, `grim -o` capturing that container alone, and teardown removing the output and freeing its workspace slot | yes |
| `vm-pointer` | the **lease and the driver**: a version-2 virtual pointer, a warp that lands where it was aimed *as the application reports it*, a click delivered as one press and one release, a contended acquire refused with BUSY, an expired lease swept with its held button lifted, and a broker restart that leaves the lock free, the bookmark intact and the driver reachable | yes |
| `vm-keyboard` | the **keymap, the focus and a decoded symbol**: the keymap the driver uploads reaching a real compositor, `pointer acquire` leaving the aimed window holding the *keyboard* and not merely the cursor, and `hhs-keyboard-target` decoding `hello` from the keymap it was **forwarded** — so a driver that stopped uploading one arrives as `sym=?` and goes red | yes |
| `vm-keyboard-de` | the same file with `sessions.keyboard.layout = "de"`, because that option reaches the uploaded keymap through exactly one path — `--xkb-layout` on the driver unit — and a single-layout check cannot tell that path from a hardcoded `us`: keysym `y`'s evdev code swaps (21 against 44), the forwarded `symbols=` name with it, and `ü` goes from exit 2 to `udiaeresis` | yes |
| `vm-stream-firewall` | the stream's **port**: the guest's own running firewall accepts the configured port and not the module's default, a peer outside the guest reaches it, and is refused on a port beside it that was never opened | yes |
All of them at once:
@@ -72,9 +69,6 @@ nix build -L .#checks.x86_64-linux.vm-sessions
nix build -L .#checks.x86_64-linux.vm-ipc
nix build -L .#checks.x86_64-linux.vm-containers
nix build -L .#checks.x86_64-linux.vm-pointer
nix build -L .#checks.x86_64-linux.vm-keyboard
nix build -L .#checks.x86_64-linux.vm-keyboard-de
nix build -L .#checks.x86_64-linux.vm-stream-firewall
```
`-L` is not optional in practice: without it, Nix hides the build log, and the
@@ -229,7 +223,6 @@ checks/
symbol a client decoded. Exposed TWICE -- `vm-keyboard`
(layout us) and `vm-keyboard-de` -- because the layout
reaches the keymap only through the driver unit's argv
vm-stream-firewall.nix nixosTest, the stream's port in the guest's firewall
lib/test-tools.nix packages the tests/ helpers for the guest
lib/session-tools.nix packages the two guest programs below
lib/session-machine.nix the guest the three session checks share
-7
View File
@@ -9,7 +9,6 @@
# vm-pointer -- the lease, the driver, and a click the application saw
# vm-keyboard -- the uploaded keymap, keyboard focus, and a decoded symbol
# vm-keyboard-de -- the same check with sessions.keyboard.layout = "de"
# vm-stream-firewall -- the stream's port in the guest's own running firewall
#
# What the guests need lives in checks/lib: `test-tools.nix` for the fake
# capture and the contract asserter, `session-tools.nix` for the event
@@ -67,10 +66,4 @@ in
vm-keyboard-de = pkgs.callPackage ./vm-keyboard.nix (
sessionArgs // { keyboardLayout = "de"; }
);
# Not a `sessionArgs` consumer: this one is about the module's own options,
# not about the broker or the compositor (see the file's header).
vm-stream-firewall = pkgs.callPackage ./vm-stream-firewall.nix {
inherit hhstream hhsModule testTools;
};
}
-243
View File
@@ -1,243 +0,0 @@
# checks.<system>.vm-stream-firewall -- the stream's port, and only that port.
#
# `sessions.stream.openFirewall` is the one part of the session-stream web app
# that a machine declares before the app has a unit (it is the piece of
# docs/plans/2026-10-04-session-stream-webapp.md §3 that does not need the
# broker), so it is the one part worth gating on its own.
#
# The option's effect is a firewall rule, and a rule is only a rule once the
# firewall has loaded it -- so this boots the module rather than evaluating it
# and reading `config.networking.firewall.allowedTCPPorts` back. A test that
# reads the option out of the evaluation proves the option is plumbed; this one
# asks the running firewall, and then asks it from *outside* the guest, which
# is the only place a firewall exists for.
#
# Four things about the shape are deliberate:
#
# * **A second machine is the peer.** Nothing outside a guest can reach it
# except another guest: the test driver runs on the host and is not on the
# test VLAN, so a probe from the driver fails whether the port is open or
# not. The peer is a bare machine on the same VLAN with no module on it --
# a peer running the module would be testing the machine against itself.
# * The configured port is **not** the module's default (9123, with 9988 as
# the default it must ignore). The interesting failure is an option that
# opens its own default whatever it is set to, and a check that configures
# 9988 cannot see it.
# * A listener is put on the unopened neighbour too, so "refused" means the
# firewall refused rather than nothing being there to connect to. Both
# listeners answer on loopback inside the guest first.
# * The address the peer probes is read out of the guest instead of assumed,
# because a check that guesses an interface name is testing the guess -- and
# the guest carries two addresses (the qemu NAT one and the test VLAN's),
# only one of which the peer can reach.
{
testers,
hhstream,
testTools,
hhsModule,
python3,
curl,
# The port the module is told to open; one beside it that it is not; and the
# module's own default, which must not appear at all.
configuredPort ? 9123,
unopenedPort ? 9124,
defaultPort ? 9988,
}:
testers.runNixOSTest {
name = "hhstream-vm-stream-firewall";
# Two guests on a two-core machine is slower than one; cap it so a hang is a
# failure rather than a wedged builder.
globalTimeout = 1800;
nodes.machine = {
# gpu-session.nix is what makes the compositor able to start at all; this
# check is not about the compositor, but `sessions.stream` lives under a
# module that boots one, and a machine kept from starting its own units is
# not the machine anyone runs.
imports = [ hhsModule ./lib/gpu-session.nix ];
services.hyprland-headless-stream = {
enable = true;
# The module declares `package` with no default (see README.dev.md), so
# the test has to supply the flake's build.
package = hhstream;
frameRate = 30;
outputs = [
{
width = 1280;
height = 720;
refresh = 30;
}
];
# The fake capture, as in vm-basic: the compositor's own frames are not
# this check's subject, and a real `grim` here would only add minutes of
# software rendering to it.
captureCommand = [
"${testTools}/bin/hhs-fake-capture"
"--width"
"1280"
"--height"
"720"
];
sessions.stream = {
port = configuredPort;
# Not a loopback address, because the module refuses that pairing with
# `openFirewall` on: opening a port nothing can reach is a hole with no
# service behind it (see the assertion in nix/module.nix).
bind = "0.0.0.0";
openFirewall = true;
};
};
# Two listeners, each bound on every interface: the configured port and a
# neighbour the module was never told about. They exist so that a refusal
# out here is the firewall's doing rather than an empty port.
systemd.services.probe-configured = {
wantedBy = [ "multi-user.target" ];
serviceConfig.ExecStart = "${python3}/bin/python3 -m http.server ${toString configuredPort}";
};
systemd.services.probe-unopened = {
wantedBy = [ "multi-user.target" ];
serviceConfig.ExecStart = "${python3}/bin/python3 -m http.server ${toString unopenedPort}";
};
environment.systemPackages = [
# `curl` is the loopback probe; python3 is the listeners themselves.
python3
curl
];
virtualisation = {
memorySize = 2048;
cores = 2;
diskSize = 4096;
graphics = false;
# The VLAN the peer is on. Without it the two machines share nothing but
# the driver's console, and "outside" stops existing.
vlans = [ 1 ];
};
};
# The peer: a bare machine on the same VLAN, no module, no compositor. Its
# whole job is to fail to reach one port and succeed on another.
nodes.peer = {
environment.systemPackages = [ curl ];
virtualisation = {
memorySize = 1024;
cores = 1;
vlans = [ 1 ];
};
};
testScript = ''
CONFIGURED = ${toString configuredPort}
UNOPENED = ${toString unopenedPort}
DEFAULT = ${toString defaultPort}
def dump(label):
print(f"########## diagnostics: {label} ##########")
for cmd in (
"systemctl --no-pager -l status firewall.service",
"systemctl --no-pager -l status probe-configured.service probe-unopened.service",
"nixos-firewall-tool show",
"ip -4 -o addr show",
):
status, out = machine.execute(cmd, timeout=60)
print(out)
print("########## end diagnostics ##########")
def peer_reaches(address, port, timeout=8):
"""Ask the peer to open the port: out there, a firewall is a firewall."""
status, out = peer.execute(
f"curl -sS -f -o /dev/null --max-time {timeout} http://{address}:{port}/",
timeout=timeout + 10,
)
return status == 0
start_all()
# Every global address the machine has, waited for and then read off the
# machine rather than assumed: a bare test machine carries two (the qemu NAT
# one and the test VLAN's), and only one of them is a path the peer can
# take.
#
# The wait is not decoration. Addresses appear a moment after `start_all()`
# returns, and reading too early yields an empty list -- which reads as "the
# peer can reach nothing" rather than as "asked a second too soon".
machine.wait_until_succeeds(
"ip -4 -o addr show scope global | grep -q ' inet '", timeout=90
)
addresses = []
for line in machine.succeed("ip -4 -o addr show scope global").splitlines():
# "2: eth0 inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic ..."
fields = line.split()
if len(fields) >= 4 and fields[2] == "inet":
addresses.append(fields[3].split("/")[0])
print(f"the machine's addresses: {addresses}")
assert addresses, "the machine reported no global address to probe"
with subtest("a machine with a stream declared is still a machine that boots"):
try:
machine.wait_for_unit("hyprland-headless.service", timeout=120)
machine.wait_for_unit("hyprland-headless-stream.service", timeout=120)
except Exception:
dump("the module's own units did not come up")
raise
with subtest("the running firewall accepts the configured port, and not the default"):
try:
machine.wait_for_unit("firewall.service")
rules = machine.succeed("nixos-firewall-tool show")
print(rules)
except Exception:
dump("the firewall did not come up")
raise
assert f"dpt:{CONFIGURED}" in rules, (
f"the firewall does not accept {CONFIGURED}: the option did not reach "
f"the rule set"
)
assert f"dpt:{DEFAULT}" not in rules, (
f"the firewall accepts {DEFAULT}: the module opened its own default "
f"port instead of the configured one"
)
with subtest("both probe listeners are up and answer inside the guest"):
try:
machine.wait_for_unit("probe-configured.service", timeout=60)
machine.wait_for_unit("probe-unopened.service", timeout=60)
for port in (CONFIGURED, UNOPENED):
machine.succeed(f"curl -sS -f -o /dev/null http://127.0.0.1:{port}/")
except Exception:
dump("a probe listener did not come up")
raise
with subtest("the peer reaches the configured port"):
reached = [a for a in addresses if peer_reaches(a, CONFIGURED)]
print(f"the peer reached {CONFIGURED} on: {reached}")
if not reached:
dump(f"the peer could not reach {CONFIGURED} on any of {addresses}")
raise AssertionError(
f"{CONFIGURED} is accepted in the rule set but the peer cannot reach "
f"it on any address the machine has ({addresses})"
)
with subtest("and is refused on the port the module never opened"):
# On the same addresses that just answered for the configured port, so
# the refusal is the firewall's doing and not a missing route.
for address in reached:
if peer_reaches(address, UNOPENED):
dump(f"the peer reached {UNOPENED} on {address}")
raise AssertionError(
f"the peer reached {UNOPENED} on {address}, a port the module was "
f"never told to open: the firewall is letting through more than "
f"the option names"
)
'';
}
-1
View File
@@ -14,7 +14,6 @@ Each file answers three questions and stops:
| [scroll-has-no-compositor-proof](scroll-has-no-compositor-proof.md) | medium | works live; unproven in a gate |
| [reconcile-while-containers-are-off](reconcile-while-containers-are-off.md) | medium | partly fixed: discovery now survives a restart; teardown still lives inside the feature |
| [kill-can-leave-the-app-alive](kill-can-leave-the-app-alive.md) | medium | found by counting windows, not sessions |
| [capture-binary-and-the-unit-path](capture-binary-and-the-unit-path.md) | medium | found the first time the app was run as a unit: a unit's `PATH` has no `grim` |
Closed since this folder was written:
@@ -1,56 +0,0 @@
# The capture command needs an absolute path under a unit
## The issue
`hhstream-live` runs its capture command (`grim -c -t jpeg -q 80 -o <output> -`)
by name, from the `PATH` it inherits. Under **systemd** that path is
`/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin` — the system
profile is deliberately not on it — so the capture fails, ten times, and the
stream gives up:
```
WARNING hhstream.live: stream on hh-chromium-yhvpms: giving up after 10
consecutive capture failures: capture command ['grim', '-c', '-t', 'jpeg',
'-q', '80', '-o', 'hh-chromium-yhvpms', '-'] could not be run:
[Errno 2] No such file or directory: 'grim'
```
Measured on 2026-10-11 on the deployment machine, the first time the app was
run **as a unit** rather than from a login shell. A login shell has
`/run/current-system/sw/bin` on its `PATH`, which is why every run before this
one — the stage-1 work, the standalone `serve --stream` runs, the whole test
suite — never saw it.
What makes it worth a file rather than a line: the failure is quiet in the
worst way. The unit starts, the port opens, the landing page answers, and
`status.json` reports a stream that is simply `"stopped": true` with
`"failures": 10`. A viewer sees an empty frame area and nothing else says why.
Only the unit's log names the reason, one line above the giving-up entry.
## Ideas
1. **An absolute binary, as the module already does elsewhere** *(recommended)*.
`sessions.apps` requires an absolute store path for exactly this reason —
"a unit's default `PATH` does not include the system profile", which is why
the allowlist spells `foot` as `${pkgs.foot}/bin/foot`. The stream's unit
should not inherit `grim` by name either: pass
`${pkgs.grim}/bin/grim` as the capture command from the module (or set
`PATH` on the unit, which is what made it work on the deployment machine).
The absolute path is also the stronger control: it names one exact binary
instead of whatever `PATH` resolves to at the moment the unit starts.
2. **Resolve early and fail loudly.** Have the app resolve the binary itself at
start-up and refuse to serve with a named error (`capture_binary_missing`)
if it is not there. A unit that dies at start is a five-second diagnosis; a
unit that serves an empty frame is an afternoon.
3. **Both.** The module passes the absolute path *and* the app checks it once
at start-up: the first is the fix, the second is what catches the next
deployment shape nobody has thought of yet.
## Why it matters
Stage 3's unit is the point at which running under systemd stops being an
experiment and becomes the only path every deployment takes — and a stream
that answers, reports a healthy port, and shows nothing is a worse failure than
one that does not start. Whichever idea is taken, the check for it belongs in
the same stage as the unit: a `vm-*` check that starts the unit and reads one
frame would have caught this on the day it was written.
+9 -29
View File
@@ -85,20 +85,14 @@ session record so it survives a broker restart and a re-enable.
- Shape: `<word>-<word>-<word>-<word>-<xxxxx>`, e.g.
`amber-otter-lantern-harbor-7k2qp`.
- Words come from a **vendored EFF large wordlist** (7776 lines, ~70 KB, in the
- Words come from a **vendored EFF large wordlist** (7776 words, ~70 KB, in the
repo — no network at runtime, no dependency) and the tail is five characters
of `[a-z0-9]`. Four of those lines are hyphenated (`drop-down`, `felt-tip`,
`t-shirt`, `yo-yo`) and are dropped on load, because a hyphen is the phrase's
own separator and such a word would make a six-token phrase nobody could hear
the shape of; **7772 words** are usable. Four words is `4 · log2(7772) ≈ 51.7`
bits and the tail adds `5 · log2(36) ≈ 25.8`, for **≈ 77.5 bits** —
comfortably past anything a LAN request rate can enumerate, and the operator's
stated shape ("5 with a random 5 character string for the last word").
- `--phrase` on enable sets your own, and it is held to **exactly the generated
shape** (four `[a-z]{3,}` words plus a five-character `[a-z0-9]` tail): the
phrase is the stream's only credential, so anything a `--phrase` may be is a
password an operator can choose, and a looser check would accept `a-b-c`.
`disable` then `enable` **rotates** it. A rotation is one record write.
of `[a-z0-9]`. Four words is `4 · log2(7776) ≈ 51.7` bits and the tail adds
`5 · log2(36) ≈ 25.8`, for **≈ 77.5 bits** — comfortably past anything a LAN
request rate can enumerate, and the operator's stated shape ("5 with a random
5 character string for the last word").
- `--phrase` on enable sets your own (checked against the shape); `disable`
then `enable` **rotates** it. A rotation is one record write.
- The phrase is the only credential. It is in the URL, in cleartext on the LAN
when the LAN is opted into — the threat model is "another device on the LAN
can watch if it can guess the phrase", not "the phrase is hidden from the
@@ -156,7 +150,7 @@ capture without one.
| 0 | Contract text: `docs/CONTRACT-sessions.md` → **sessions-v3** (the four verbs, `/run/hhstream/live.sock`, `hhstream-live`, the single port, the viewer-touch exception, schema 4, error names). | S | Freezing the wrong default. This plan is the review. |
| 1 | `hhstream-live`: the HTTP server, the router, the phrase (wordlist + generator), one output, injected capture, **unit tests**. No broker. | M | A phone that will not paint `multipart/x-mixed-replace`; the index leaking phrases when made public. |
| 2 | Broker + CLI: the record field, schema 4, the four verbs, teardown order, viewer heartbeat, integration test on `127.0.0.1`. | L | A second capture on re-enable; `disable` racing `output remove`; a heartbeat that keeps a session alive with zero viewers. |
| 3 | Module: the unit, `/run/hhstream/live.sock`, `port` 9988, `bind` loopback, `fps` 5/cap 10, `enable` false, `openFirewall` false, `publicIndex` false. No host firewall edit here. | M | A second `RuntimeDirectory` on `/run/hhstream` deleting `control.sock`; and the capture binary — a unit's `PATH` has no `grim`, so the stream serves an empty frame area without saying why (`docs/TODO/capture-binary-and-the-unit-path.md`). |
| 3 | Module: the unit, `/run/hhstream/live.sock`, `port` 9988, `bind` loopback, `fps` 5/cap 10, `enable` false, `openFirewall` false, `publicIndex` false. No host firewall edit here. | M | A second `RuntimeDirectory` on `/run/hhstream` deleting `control.sock`. |
| 4 | Host pass: enable a real session, open the URL in the machine's own Chromium, confirm pixels change without a reload; write down what was on screen. | S | An empty container looking like a broken stream. Confirm the window is on `hh-<id>` first. |
Test layers are v1's: `tests/unit` for the HTTP/router/phrase with a fake
@@ -178,18 +172,4 @@ one at a time; no browser in a VM check.
**Stage 1 is built** (`src/hhstream_live/`, `tests/live/`, 51 unit tests): the
router, the pages, the phrase and the publisher, runnable standalone via
`hhstream-live serve --stream PHRASE=OUTPUT`. The broker verbs and
`/run/hhstream/live.sock` are Stage 2.
**Stage 3 has started, out of order and on purpose** (2026-10-11). A machine
had to be able to *reach* a stream before the rest of the app is settled: the
operator asked for the port to be open on the deployment machine, where the app
runs by hand, and a hand-written firewall line in one machine's config is the
wrong place for a fact about this flake. So `sessions.stream { port, bind,
openFirewall }` and the single rule it stands for are in `nix/module.nix`, with
`checks/vm-stream-firewall.nix` asking the *running* firewall — and a peer
outside the guest — whether the configured port is the one that opened.
What that does **not** do: no unit, no phrases, no `session stream
enable|disable|list|status`, and nothing in the module turns a stream on. The
app is still started by hand, and the option's whole job is to describe where a
stream will be reachable from.
`/run/hhstream/live.sock` are Stage 2.
-88
View File
@@ -47,11 +47,6 @@ let
controlSocket = "${brokerRuntimeDir}/control.sock";
pointerSocket = "${brokerRuntimeDir}/pointer.sock";
# Addresses that mean "this listener answers on loopback only". The spelling
# `localhost` is included because it is what a person writes; it is not a
# resolution the module performs.
loopbackBinds = [ "127.0.0.1" "::1" "localhost" ];
socketDir = builtins.dirOf cfg.socketPath;
# Hyprland refuses `--socket` unless it is paired with `--wayland-fd` (that
@@ -699,63 +694,6 @@ in {
};
};
# The session-stream web app — `hhstream-live`, the router that serves a
# live view of one session per phrase, over one port
# (`docs/plans/2026-10-04-session-stream-webapp.md`).
#
# Only the *reach* of it is declared here so far: the port every stream
# shares, the address it will bind, and whether that port is opened in
# the firewall. The unit that runs the app, the phrases the broker keeps
# in its records and the `session stream enable` verbs are later stages
# of that plan — and deliberately so: a stream is switched on per
# session, never for all of them, so nothing here turns one on.
stream = {
port = lib.mkOption {
type = lib.types.port;
default = 9988;
description = ''
The single TCP port every stream shares.
One listener rather than one per session is what makes the phrase
speakable: there is one address to give a person, one thing to
firewall and one thing to stop. It is also why this is a fixed
number with no range to fall back on — a stream at a port nobody
was told about is worse than a stream that is down, so a taken port
fails loudly instead of moving.
'';
};
bind = lib.mkOption {
type = lib.types.str;
default = "127.0.0.1";
description = ''
The address the web app will bind.
Loopback by default, where the reachable path is an ssh tunnel
(`ssh -L 9988:127.0.0.1:9988`) and nothing else. Set it to an
address a peer can reach (`"0.0.0.0"`) when
{option}`sessions.stream.openFirewall` is set: opening the port for
a listener that is not on it is a hole with no service behind it,
which is why that pairing is refused rather than allowed.
'';
};
openFirewall = lib.mkOption {
type = lib.types.bool;
default = false;
description = ''
Open {option}`sessions.stream.port` in the firewall.
Off by default, and deliberately not implied by
{option}`sessions.enable`: a live view of this desktop is readable
by anyone who can reach the port, and the phrase in the URL is its
only credential — no TLS, no second factor
(`docs/plans/2026-10-04-…` §4). Opening it is an explicit act, and
it opens exactly one port.
'';
};
};
stateDir = lib.mkOption {
type = lib.types.str;
default = "/var/lib/hhstream";
@@ -831,21 +769,6 @@ in {
so without the broker it would bind a socket nothing connects to.
'';
}
{
assertion =
!cfg.sessions.stream.openFirewall
|| !(lib.elem cfg.sessions.stream.bind loopbackBinds);
message = ''
services.hyprland-headless-stream.sessions.stream.openFirewall is true
while sessions.stream.bind is "${cfg.sessions.stream.bind}" — a loopback
address. The port would be open in the firewall with nothing answering on
it from anywhere but this machine, which is a hole rather than a feature.
Set sessions.stream.bind to an address a peer can reach ("0.0.0.0"), or
leave the firewall closed and reach the stream through a tunnel
(`ssh -L 9988:127.0.0.1:9988`).
'';
}
{
assertion = !cfg.sessions.enable || lib.hasAttr cfg.sessions.user config.users.users;
message = ''
@@ -908,17 +831,6 @@ in {
})
];
# The stream's one port, in the firewall.
#
# `lib.optional`, not a conditional block: the option is one line and its
# effect is one port in the rule set, so there is nothing here that could
# open a second one. Everything else about the app — the phrase per
# session, the capture, the enable/disable verbs — is later stages of the
# plan; this is the only part of the stream that a machine declares before
# any of that exists.
networking.firewall.allowedTCPPorts =
lib.optional cfg.sessions.stream.openFirewall cfg.sessions.stream.port;
users.users.${cfg.user} = {
isSystemUser = true;
group = cfg.group;
+3 -19
View File
@@ -35,28 +35,12 @@ python3.pkgs.buildPythonApplication {
"hhstream_pointer.keymap"
"hhstream_pointer._protocol.wlr_virtual_pointer_unstable_v1"
"hhstream_pointer._protocol.virtual_keyboard_unstable_v1"
# The web app: importing the server pulls in the router, the publisher and
# the pages.
# The web app. Importing the server pulls in the router, the publisher and
# the pages; the phrase generator's wordlist is checked by the unit suite,
# which reads the *installed* package.
"hhstream_live.server"
];
# `pythonImportsCheck` only *imports*, and `hhstream_live.phrase` reads its
# wordlist lazily -- so a wheel that dropped `wordlist.txt` would sail past it
# and fail at the first `enable` instead. Read the list for real, against the
# installed tree, and mint a phrase so the generator and its validator are
# proven on the same words. `doCheck = false` (the suite runs in the flake's
# `checks`), so this is the only place the packaged data is exercised.
postFixup = ''
echo "checking the installed wordlist..."
PYTHONPATH="$out/${python3.sitePackages}:$PYTHONPATH" ${python3.interpreter} -c '
import hhstream_live.phrase as phrase
count = len(phrase.words())
assert count > 7000, f"wordlist has only {count} words; is wordlist.txt installed?"
assert phrase.validate(phrase.generate())
print(f"wordlist ok: {count} words, ~{phrase.entropy_bits():.1f} bits a phrase")
'
'';
meta = {
description = "Publish a headless Hyprland session's frames on a Unix socket";
license = lib.licenses.mit;
@@ -1 +0,0 @@
/target
File diff suppressed because it is too large. Load diff
@@ -1,16 +0,0 @@
[package]
name = "grimrs-capture"
version = "0.1.0"
edition = "2021"
publish = false
description = "Long-lived grim-rs capture producer for the frozen HARNESS.md contract"
[[bin]]
name = "grimrs-capture"
path = "src/main.rs"
[dependencies]
# Exact: the raw-mode stop point depends on what 0.2.0 does inside capture_*.
grim-rs = "=0.2.0"
libc = "0.2"
serde_json = { version = "1", features = ["preserve_order"] }
@@ -1,53 +0,0 @@
#!/bin/sh
# Eight measured cells, still screen, two runs each.
# Stdout of each cell is one JSON line. Progress goes to runs/progress.log.
# Does not set XDG_RUNTIME_DIR. The caller must export WAYLAND_DISPLAY to the
# absolute socket path.
set -u
cd "$(dirname "$0")"
if [ "${WAYLAND_DISPLAY:-}" != "/run/hypr-stream/wayland-1" ]; then
echo "WAYLAND_DISPLAY must be /run/hypr-stream/wayland-1" >&2
exit 2
fi
if [ "${XDG_RUNTIME_DIR:-}" = "/run/hypr-stream" ]; then
echo "refusing XDG_RUNTIME_DIR=/run/hypr-stream" >&2
exit 2
fi
OUTPUT=${1:?usage: run-cells.sh OUTPUT_NAME}
BIN=./target/release/grimrs-capture
if [ ! -x "$BIN" ]; then
echo "missing $BIN; build with nix-shell --run 'cargo build --release'" >&2
exit 2
fi
mkdir -p runs
LOG=runs/progress.log
: > "$LOG"
# The frame server is a system unit. Do not stop it. Record that it was
# capturing during the run so the numbers are not read as an idle compositor.
ps -eo pid,user,cmd > runs/ps-at-start.txt
echo "BEGIN runner $(date -Is) WAYLAND_DISPLAY=$WAYLAND_DISPLAY XDG_RUNTIME_DIR=${XDG_RUNTIME_DIR-}" >> "$LOG"
fail=0
run() {
name=$1
shift
echo "BEGIN $name $(date -Is)" >> "$LOG"
start=$(date +%s)
"$BIN" "$@" > "runs/$name.stdout" 2> "runs/$name.stderr"
rc=$?
end=$(date +%s)
echo "END $name rc=$rc elapsed=$((end - start)) $(date -Is)" >> "$LOG"
if [ "$rc" -ne 0 ]; then
echo "CELL_FAIL $name rc=$rc" >> "$LOG"
fail=1
fi
}
run output-raw-1 --output "$OUTPUT" --frames 300 --warmup 10 --mode raw --json
run output-raw-2 --output "$OUTPUT" --frames 300 --warmup 10 --mode raw --json
run output-jpeg-1 --output "$OUTPUT" --frames 300 --warmup 10 --mode jpeg --json
run output-jpeg-2 --output "$OUTPUT" --frames 300 --warmup 10 --mode jpeg --json
run all-raw-1 --all --frames 300 --warmup 10 --mode raw --json
run all-raw-2 --all --frames 300 --warmup 10 --mode raw --json
run all-jpeg-1 --all --frames 300 --warmup 10 --mode jpeg --json
run all-jpeg-2 --all --frames 300 --warmup 10 --mode jpeg --json
echo "ALL_DONE fail=$fail $(date -Is)" >> "$LOG"
exit "$fail"
@@ -1,13 +0,0 @@
# Spike-local toolchain for route 001a. Rust stays out of the repository flake
# on purpose: this shell is the only place this spike gets rustc and cargo.
#
# Do not set WAYLAND_DISPLAY or XDG_RUNTIME_DIR here. The run commands in
# README.md set WAYLAND_DISPLAY to the absolute socket path. Pointing
# XDG_RUNTIME_DIR at /run/hypr-stream makes libwayland reject the runtime dir.
{ pkgs ? import <nixpkgs> { } }:
pkgs.mkShell {
packages = with pkgs; [
rustc # 1.98 from the machine's nixpkgs
cargo
];
}
@@ -1,641 +0,0 @@
//! Long-lived capture producer for the frozen measurement contract.
//!
//! One Wayland connection (`Grim::new_wlr`, so the crate cannot pick
//! `ext-image-copy-capture-v1`), then warmup frames, then N measured frames.
//! Stdout is exactly one JSON object, on the last line. Everything else goes
//! to stderr.
use std::env;
use std::io::{self, Write};
use std::process::ExitCode;
use std::time::Instant;
use grim_rs::Grim;
const ROUTE: &str = "grim-rs";
const JPEG_QUALITY: u8 = 80;
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
enum Mode {
Raw,
Jpeg,
}
impl Mode {
fn as_str(self) -> &'static str {
match self {
Mode::Raw => "raw",
Mode::Jpeg => "jpeg",
}
}
}
#[derive(Clone, Debug)]
enum Target {
Output(String),
All,
}
impl Target {
fn label(&self) -> &str {
match self {
Target::Output(name) => name,
Target::All => "all",
}
}
}
struct Args {
target: Target,
frames: u32,
warmup: u32,
mode: Mode,
}
struct Held {
width: u32,
height: u32,
/// JPEG byte length. `None` in raw mode.
jpeg_len: Option<usize>,
/// Pixel or JPEG bytes. Kept alive until the caller drops it, so the
/// frame timer can stop when the bytes are complete and not when they
/// are freed.
bytes: Vec<u8>,
}
fn main() -> ExitCode {
match run() {
Ok(()) => ExitCode::SUCCESS,
Err(err) => {
eprintln!("grimrs-capture: {err}");
ExitCode::from(1)
}
}
}
fn run() -> Result<(), String> {
let args = parse_args(env::args().skip(1))?;
// Absolute WAYLAND_DISPLAY is required. Leaving XDG_RUNTIME_DIR alone is
// also required: this process does not own /run/hypr-stream.
match env::var("WAYLAND_DISPLAY") {
Ok(value) if !value.is_empty() => {}
_ => {
return Err(
"WAYLAND_DISPLAY is empty. Set it to the absolute socket \
path /run/hypr-stream/wayland-1 and do not set \
XDG_RUNTIME_DIR=/run/hypr-stream"
.to_string(),
);
}
}
if env::var("XDG_RUNTIME_DIR").ok().as_deref() == Some("/run/hypr-stream") {
return Err(
"XDG_RUNTIME_DIR=/run/hypr-stream is set. Unset it; libwayland \
refuses a runtime dir the caller does not own"
.to_string(),
);
}
// Force zwlr-screencopy. Grim::new() prefers ext-image-copy-capture-v1,
// which is the dma-buf-capable path this machine must not take.
let mut grim = Grim::new_wlr().map_err(|err| {
format!(
"Grim::new_wlr failed ({err}). Refusing to fall back to \
ext-image-copy-capture-v1"
)
})?;
let outputs = grim.get_outputs().map_err(|err| err.to_string())?;
if outputs.is_empty() {
return Err("compositor advertised no outputs".to_string());
}
let layout = describe_outputs(&outputs);
if let Target::Output(name) = &args.target {
if !outputs.iter().any(|output| output.name() == name) {
return Err(format!(
"output {name:?} not found. Outputs: {layout}"
));
}
}
eprintln!(
"grimrs-capture: route {ROUTE} protocol zwlr-screencopy-v1 \
mode {} target {} warmup {} frames {}",
args.mode.as_str(),
args.target.label(),
args.warmup,
args.frames
);
eprintln!("grimrs-capture: outputs {layout}");
let mut last_warmup_nonzero: Option<u64> = None;
let mut last_warmup_jpeg_ok = true;
for i in 0..args.warmup {
let held = grab(&mut grim, &args.target, args.mode)?;
if args.mode == Mode::Jpeg && !held.bytes.starts_with(&[0xFF, 0xD8]) {
last_warmup_jpeg_ok = false;
}
last_warmup_nonzero = Some(count_nonzero(&held.bytes));
eprintln!(
"grimrs-capture: warmup {}/{} {}x{} bytes {} nonzero {}",
i + 1,
args.warmup,
held.width,
held.height,
held.bytes.len(),
last_warmup_nonzero.unwrap_or(0)
);
}
if args.mode == Mode::Jpeg && !last_warmup_jpeg_ok {
return Err(
"warmup JPEG did not start with FF D8; refusing to measure"
.to_string(),
);
}
let cpu_before = process_cpu_ms()?;
let wall_start = Instant::now();
let mut samples_ms: Vec<f64> = Vec::with_capacity(args.frames as usize);
let mut jpeg_sizes: Vec<u64> = Vec::new();
let mut geometry: Option<(u32, u32)> = None;
for _ in 0..args.frames {
let tick = Instant::now();
let held = grab(&mut grim, &args.target, args.mode)?;
// Touch the length so the buffer is complete before the clock stops.
// Do not read pixels: the contract puts stride and checksum outside
// the measurement.
std::hint::black_box(held.bytes.len());
let ms = tick.elapsed().as_secs_f64() * 1000.0;
match geometry {
None => geometry = Some((held.width, held.height)),
Some((w, h)) if w != held.width || h != held.height => {
return Err(format!(
"geometry changed during the run: {w}x{h} then {}x{}",
held.width, held.height
));
}
Some(_) => {}
}
if let Some(len) = held.jpeg_len {
jpeg_sizes.push(len as u64);
}
samples_ms.push(ms);
// Free between frames, outside the frame timer. wall_ms includes it.
drop(held);
}
let wall_ms = wall_start.elapsed().as_secs_f64() * 1000.0;
let cpu_after = process_cpu_ms()?;
let (width, height) = geometry.ok_or("no measured frames")?;
let cpu_ms_per_frame = match (cpu_before, cpu_after) {
(Some(before), Some(after)) if after >= before => {
Some((after - before) / f64::from(args.frames))
}
_ => None,
};
let mut notes = format!(
"grim-rs 0.2.0 via Grim::new_wlr(); copy target is a wl_shm buffer \
created inside each capture_* call, not a dma-buf the producer \
imports. Cursor not overlaid (the frozen argv has no cursor flag; \
hhstream-live's grim command passes -c). JPEG is Grim::to_jpeg, \
quality {JPEG_QUALITY}, the crate default and the live-stream \
setting. Raw timer stops when capture_* returns, which is after \
the crate copies the mapping into a Vec and swizzles XRGB/ARGB to \
RGBA (and applies any output transform or y-invert). The public \
API does not expose the mapped buffer before that, so raw includes \
work the contract says to leave outside the timer. Percentiles are \
linear interpolation on the sorted samples. cpu_ms_per_frame is \
getrusage(RUSAGE_SELF) utime+stime over the measured loop only, \
divided by frames. Frame buffers are freed between frames; that \
free is inside wall_ms and outside frame_ms. Outputs: {layout}. \
No GPU on this machine; dma-buf was not used and is not claimed. \
This process does not move the pointer and does not start or stop \
a session; the pixels are whatever the compositor is already \
showing."
);
if let Some(nonzero) = last_warmup_nonzero {
notes.push_str(&format!(
" Last warmup frame nonzero bytes: {nonzero}."
));
if nonzero == 0 {
notes.push_str(" WARNING: last warmup frame was all zeros.");
}
}
let report = Report {
route: ROUTE,
language: "rust",
protocol: "zwlr-screencopy-v1",
output: args.target.label().to_string(),
geometry: [width, height],
mode: args.mode.as_str(),
warmup: args.warmup,
frames: args.frames,
frame_ms: summarize(&samples_ms),
fps_sustained: sustained_fps(&samples_ms),
wall_ms,
cpu_ms_per_frame,
bytes_p50: if args.mode == Mode::Jpeg {
Some(percentile_u64(&jpeg_sizes, 0.50))
} else {
None
},
persistent: true,
notes,
};
let line = serde_json::to_string(&report.to_json())
.map_err(|err| err.to_string())?;
let mut stdout = io::stdout().lock();
writeln!(stdout, "{line}").map_err(|err| err.to_string())?;
stdout.flush().map_err(|err| err.to_string())?;
Ok(())
}
fn grab(grim: &mut Grim, target: &Target, mode: Mode) -> Result<Held, String> {
let captured = match target {
Target::Output(name) => grim.capture_output(name),
Target::All => grim.capture_all(),
}
.map_err(|err| err.to_string())?;
let width = captured.width();
let height = captured.height();
if width == 0 || height == 0 {
return Err(format!("capture returned {width}x{height}"));
}
match mode {
Mode::Raw => Ok(Held {
width,
height,
jpeg_len: None,
bytes: captured.into_data(),
}),
Mode::Jpeg => {
let jpeg = grim
.to_jpeg_with_quality(captured.data(), width, height, JPEG_QUALITY)
.map_err(|err| err.to_string())?;
let jpeg_len = jpeg.len();
Ok(Held {
width,
height,
jpeg_len: Some(jpeg_len),
bytes: jpeg,
})
}
}
}
fn describe_outputs(outputs: &[grim_rs::Output]) -> String {
let mut ordered: Vec<_> = outputs.iter().collect();
ordered.sort_by(|a, b| {
let ag = a.geometry();
let bg = b.geometry();
ag.x()
.cmp(&bg.x())
.then(ag.y().cmp(&bg.y()))
.then(a.name().cmp(b.name()))
});
ordered
.into_iter()
.map(|output| {
let geometry = output.geometry();
format!(
"{}@{},{} {}x{} scale {}",
output.name(),
geometry.x(),
geometry.y(),
geometry.width(),
geometry.height(),
output.scale()
)
})
.collect::<Vec<_>>()
.join("; ")
}
fn count_nonzero(bytes: &[u8]) -> u64 {
bytes.iter().filter(|byte| **byte != 0).count() as u64
}
fn process_cpu_ms() -> Result<Option<f64>, String> {
let mut usage: libc::rusage = unsafe { std::mem::zeroed() };
let rc = unsafe { libc::getrusage(libc::RUSAGE_SELF, &mut usage) };
if rc != 0 {
return Ok(None);
}
Ok(Some(timeval_ms(usage.ru_utime) + timeval_ms(usage.ru_stime)))
}
fn timeval_ms(value: libc::timeval) -> f64 {
value.tv_sec as f64 * 1000.0 + value.tv_usec as f64 / 1000.0
}
struct Summary {
min: f64,
p50: f64,
mean: f64,
p95: f64,
max: f64,
}
fn summarize(samples: &[f64]) -> Summary {
let mut sorted = samples.to_vec();
sorted.sort_by(|a, b| a.total_cmp(b));
let sum: f64 = sorted.iter().sum();
Summary {
min: sorted[0],
p50: percentile_linear(&sorted, 0.50),
mean: sum / sorted.len() as f64,
p95: percentile_linear(&sorted, 0.95),
max: *sorted.last().expect("at least one sample"),
}
}
fn sustained_fps(samples: &[f64]) -> f64 {
let sum = samples.iter().sum::<f64>();
if sum <= 0.0 {
return 0.0;
}
samples.len() as f64 / (sum / 1000.0)
}
/// Linear interpolation between closest ranks. `samples` must be sorted and
/// non-empty. This is the same definition as numpy's default percentile.
fn percentile_linear(sorted: &[f64], p: f64) -> f64 {
if sorted.len() == 1 {
return sorted[0];
}
let rank = p * (sorted.len() - 1) as f64;
let lo = rank.floor() as usize;
let hi = rank.ceil() as usize;
let frac = rank - lo as f64;
sorted[lo] * (1.0 - frac) + sorted[hi] * frac
}
fn percentile_u64(samples: &[u64], p: f64) -> f64 {
let as_f64: Vec<f64> = samples.iter().map(|value| *value as f64).collect();
let mut sorted = as_f64;
sorted.sort_by(|a, b| a.total_cmp(b));
percentile_linear(&sorted, p)
}
struct Report {
route: &'static str,
language: &'static str,
protocol: &'static str,
output: String,
geometry: [u32; 2],
mode: &'static str,
warmup: u32,
frames: u32,
frame_ms: Summary,
fps_sustained: f64,
wall_ms: f64,
cpu_ms_per_frame: Option<f64>,
bytes_p50: Option<f64>,
persistent: bool,
notes: String,
}
impl Report {
fn to_json(&self) -> serde_json::Value {
serde_json::json!({
"route": self.route,
"language": self.language,
"protocol": self.protocol,
"output": self.output,
"geometry": self.geometry,
"mode": self.mode,
"warmup": self.warmup,
"frames": self.frames,
"frame_ms": {
"min": round3(self.frame_ms.min),
"p50": round3(self.frame_ms.p50),
"mean": round3(self.frame_ms.mean),
"p95": round3(self.frame_ms.p95),
"max": round3(self.frame_ms.max),
},
"fps_sustained": round3(self.fps_sustained),
"wall_ms": round3(self.wall_ms),
"cpu_ms_per_frame": self.cpu_ms_per_frame.map(round3),
"bytes_p50": self.bytes_p50.map(round3),
"persistent": self.persistent,
"notes": self.notes,
})
}
}
fn round3(value: f64) -> f64 {
(value * 1000.0).round() / 1000.0
}
fn parse_args(args: impl IntoIterator<Item = String>) -> Result<Args, String> {
let args: Vec<String> = args.into_iter().collect();
let mut output: Option<String> = None;
let mut all = false;
let mut frames: Option<u32> = None;
let mut warmup: Option<u32> = None;
let mut mode: Option<Mode> = None;
let mut json = false;
let mut index = 0;
while index < args.len() {
let arg = &args[index];
match arg.as_str() {
"--json" => {
if json {
return Err("repeated --json".to_string());
}
json = true;
}
"--all" => {
if all {
return Err("repeated --all".to_string());
}
all = true;
}
"--output" => {
output = Some(need_value(&args, &mut index, "--output")?);
}
"--frames" => {
frames = Some(parse_u32(&need_value(&args, &mut index, "--frames")?, "--frames")?);
}
"--warmup" => {
warmup = Some(parse_u32(&need_value(&args, &mut index, "--warmup")?, "--warmup")?);
}
"--mode" => {
let value = need_value(&args, &mut index, "--mode")?;
mode = Some(match value.as_str() {
"raw" => Mode::Raw,
"jpeg" => Mode::Jpeg,
other => {
return Err(format!(
"--mode must be raw or jpeg, got {other:?}"
))
}
});
}
"--help" | "-h" => {
return Err(usage());
}
other => {
return Err(format!("unknown argument {other}. {}", usage()));
}
}
index += 1;
}
if !json {
return Err(format!("--json is required. {}", usage()));
}
if all && output.is_some() {
return Err("--all and --output are mutually exclusive".to_string());
}
let target = if all {
Target::All
} else if let Some(name) = output {
if name.is_empty() {
return Err("--output name is empty".to_string());
}
Target::Output(name)
} else {
return Err(format!("one of --output or --all is required. {}", usage()));
};
let frames = frames.ok_or_else(|| format!("--frames is required. {}", usage()))?;
if frames == 0 {
return Err("--frames must be at least 1".to_string());
}
let warmup = warmup.ok_or_else(|| format!("--warmup is required. {}", usage()))?;
let mode = mode.ok_or_else(|| format!("--mode is required. {}", usage()))?;
Ok(Args {
target,
frames,
warmup,
mode,
})
}
fn need_value(args: &[String], index: &mut usize, flag: &str) -> Result<String, String> {
let next = index.checked_add(1).and_then(|i| args.get(i));
match next {
Some(value) if !value.starts_with("--") => {
*index += 1;
Ok(value.clone())
}
_ => Err(format!("{flag} needs a value")),
}
}
fn parse_u32(value: &str, flag: &str) -> Result<u32, String> {
value
.parse::<u32>()
.map_err(|_| format!("{flag} must be a non-negative integer, got {value:?}"))
}
fn usage() -> String {
"usage: grimrs-capture (--output NAME | --all) --frames N --warmup W \
--mode raw|jpeg --json"
.to_string()
}
#[cfg(test)]
mod tests {
use super::*;
fn parse(line: &str) -> Result<Args, String> {
parse_args(line.split_whitespace().map(str::to_string))
}
#[test]
fn accepts_the_contract_forms() {
let output = parse(
"--output hh-chromium-yrfadn --frames 300 --warmup 10 --mode raw --json",
)
.unwrap();
assert!(matches!(output.target, Target::Output(name) if name == "hh-chromium-yrfadn"));
assert_eq!(output.frames, 300);
assert_eq!(output.warmup, 10);
assert_eq!(output.mode, Mode::Raw);
let all = parse("--all --frames 300 --warmup 10 --mode jpeg --json").unwrap();
assert!(matches!(all.target, Target::All));
assert_eq!(all.mode, Mode::Jpeg);
}
#[test]
fn rejects_mixed_targets_and_missing_json() {
assert!(parse("--all --output x --frames 1 --warmup 0 --mode raw --json").is_err());
assert!(parse("--output x --frames 1 --warmup 0 --mode raw").is_err());
assert!(parse("--output x --frames 0 --warmup 0 --mode raw --json").is_err());
assert!(parse("--frames 1 --warmup 0 --mode raw --json").is_err());
assert!(parse("--output x --frames 1 --warmup 0 --mode png --json").is_err());
}
#[test]
fn percentile_matches_linear_interpolation() {
let samples = [1.0, 2.0, 3.0, 4.0];
assert_eq!(percentile_linear(&samples, 0.50), 2.5);
assert_eq!(percentile_linear(&samples, 0.0), 1.0);
assert_eq!(percentile_linear(&samples, 1.0), 4.0);
// rank = 0.95 * 3 = 2.85 -> 3.0 * 0.15 + 4.0 * 0.85
let p95 = percentile_linear(&samples, 0.95);
assert!((p95 - 3.85).abs() < 1e-9);
}
#[test]
fn sustained_fps_uses_the_sum_of_frame_times() {
// Two frames of 100 ms -> 2 / 0.2 = 10 fps.
let fps = sustained_fps(&[100.0, 100.0]);
assert!((fps - 10.0).abs() < 1e-9);
}
#[test]
fn json_object_has_the_contract_keys() {
let report = Report {
route: ROUTE,
language: "rust",
protocol: "zwlr-screencopy-v1",
output: "hh-chromium-yrfadn".to_string(),
geometry: [1280, 720],
mode: "jpeg",
warmup: 10,
frames: 300,
frame_ms: Summary {
min: 1.0,
p50: 2.0,
mean: 3.0,
p95: 4.0,
max: 5.0,
},
fps_sustained: 10.0,
wall_ms: 20.0,
cpu_ms_per_frame: None,
bytes_p50: Some(100.5),
persistent: true,
notes: "n".to_string(),
};
let value = report.to_json();
for key in [
"route",
"language",
"protocol",
"output",
"geometry",
"mode",
"warmup",
"frames",
"frame_ms",
"fps_sustained",
"wall_ms",
"cpu_ms_per_frame",
"bytes_p50",
"persistent",
"notes",
] {
assert!(value.get(key).is_some(), "missing {key}");
}
assert!(value["cpu_ms_per_frame"].is_null());
assert_eq!(value["bytes_p50"], 100.5);
assert_eq!(value["persistent"], true);
assert_eq!(value["geometry"][0], 1280);
}
}
+21 -80
View File
@@ -17,16 +17,12 @@ from __future__ import annotations
import argparse
import logging
import signal
import sys
import threading
from . import phrase as phrase_mod
from .publisher import DEFAULT_FPS, FPS_CAP, StreamRegistry, StreamSpec
from .server import DEFAULT_HOST, DEFAULT_PORT, LiveServer
log = logging.getLogger("hhstream.live")
#: The session's own capture command, JPEG for the same reason the host config
#: spells it out: a 1280x720 JPEG q80 is ~46 ms and the point is a video, not a
#: slideshow. `-c` asks grim to include the cursor.
@@ -57,27 +53,13 @@ def parse_stream(arg: str) -> tuple[str, str]:
raise argparse.ArgumentTypeError(str(exc)) from exc
def _add_log_level(parser: argparse.ArgumentParser, default: str) -> None:
"""Put `--log-level` on a parser.
It is a global option, but it has to exist on the subparsers too: with no
subcommand the whole line is parsed as `serve ...`, and `serve` is then the
parser that sees it. `SUPPRESS` as the subparser default is what stops
`hhstream-live --log-level debug serve` from being reset to `info` -- a
subparser argument that defaults to `SUPPRESS` and is not supplied leaves
the namespace alone.
"""
parser.add_argument("--log-level", default=default,
choices=["debug", "info", "warning", "error"])
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(prog="hhstream-live", description=__doc__)
_add_log_level(parser, "info")
parser.add_argument("--log-level", default="info",
choices=["debug", "info", "warning", "error"])
subs = parser.add_subparsers(dest="command")
serve = subs.add_parser("serve", help="run the web app (default)")
_add_log_level(serve, argparse.SUPPRESS)
serve.add_argument("--port", type=int, default=DEFAULT_PORT,
help=f"the one port every stream shares (default {DEFAULT_PORT})")
serve.add_argument("--bind", default=DEFAULT_HOST,
@@ -94,36 +76,11 @@ def build_parser() -> argparse.ArgumentParser:
"only shape that survives words starting with '-')")
gen = subs.add_parser("phrase", help="print a fresh phrase and exit")
_add_log_level(gen, argparse.SUPPRESS)
gen.add_argument("--count", type=int, default=1)
return parser
#: The subcommands, named here because the default one is chosen before the
#: parser runs. See `parse_args`.
COMMANDS = ("serve", "phrase")
def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
"""Parse the command line, defaulting to `serve`.
`serve` is the default, and its options live on its own subparser, so the
word has to be *inserted* rather than filled in afterwards: a namespace from
the top-level parser has no `capture_command`, no `port` and no `stream`, and
`cmd_serve` would die on the first attribute it read. Inserting it has to
happen before parsing, too -- `hhstream-live --port 9999` would otherwise be
rejected for an unknown `--port` before anything could be defaulted.
`--help` is left alone so that a bare `hhstream-live --help` still shows the
program's own help rather than `serve`'s.
"""
raw = list(sys.argv[1:] if argv is None else argv)
if not any(word in COMMANDS or word in ("-h", "--help") for word in raw):
raw = ["serve", *raw]
return build_parser().parse_args(raw)
def make_registry(streams: list[tuple[str, str]], capture_base: list[str],
fps: int) -> StreamRegistry:
from hhstream.capture import CommandCapture
@@ -140,58 +97,42 @@ def cmd_serve(args: argparse.Namespace) -> int:
registry = make_registry(args.stream, capture_base, args.fps)
server = LiveServer(registry, host=args.bind, port=args.port)
# A service is stopped with SIGTERM, not with Ctrl-C: without a handler the
# default action kills the process outright and `server.stop` never runs, so
# the listener and the capture threads go down with no ordering at all.
# Installed *before* the listener exists, so there is no window in which a
# supervisor's very first signal is the fatal one.
stopping = threading.Event()
previous = None
def _request_stop(signum, frame) -> None: # noqa: ARG001 - handler signature
stopping.set()
print(f"landing page: http://{server.address}:{args.port}/", file=sys.stderr)
if not args.stream:
print("no --stream given; only the landing page will answer", file=sys.stderr)
server.start() # raises OSError loudly if the port is taken
for phrase in registry.phrases():
print(server.url_for(phrase))
sys.stdout.flush()
try:
previous = signal.signal(signal.SIGTERM, _request_stop)
except ValueError:
# Not the main thread: an embedder's problem to solve, not ours.
log.debug("SIGTERM handler not installed (not the main thread)")
import time
try:
server.start() # raises OSError loudly if the port is taken
# Printed after `start`, because a requested port of 0 only has an answer
# once it is bound. `describe_bind` names the real bind, so `0.0.0.0` is
# not reported as loopback just because that is the address you click.
print(server.describe_bind(), file=sys.stderr)
print(f"landing page: http://{server.address}:{server.port}/", file=sys.stderr)
if not args.stream:
print("no --stream given; only the landing page will answer",
file=sys.stderr)
for phrase in registry.phrases():
print(server.url_for(phrase))
sys.stdout.flush()
stopping.wait()
print("stopping", file=sys.stderr)
while True:
time.sleep(3600)
except KeyboardInterrupt:
print("stopping", file=sys.stderr)
finally:
if previous is not None:
signal.signal(signal.SIGTERM, previous)
server.stop()
return 0
def main(argv: list[str] | None = None) -> int:
args = parse_args(argv)
parser = build_parser()
args = parser.parse_args(argv)
logging.basicConfig(
level=getattr(logging, (getattr(args, "log_level", None) or "info").upper()),
level=getattr(logging, (args.log_level or "info").upper()),
format="%(asctime)s %(levelname)s %(name)s: %(message)s",
)
if args.command == "phrase":
command = args.command or "serve"
if command == "phrase":
for _ in range(args.count):
print(phrase_mod.generate())
return 0
return cmd_serve(args)
if command == "serve":
return cmd_serve(args)
parser.error(f"unknown command {command!r}") # pragma: no cover
return 2
if __name__ == "__main__":
+17 -38
View File
@@ -1,7 +1,7 @@
"""The phrase: a stream's only credential, shaped to be said out loud.
A phrase is ``word-word-word-word-xxxxx``: four words off the EFF large
wordlist and a five-character base-36 tail. That is ``4*log2(7772) +
wordlist and a five-character base-36 tail. That is ``4*log2(7776) +
5*log2(36)`` ~= **77.5 bits** of entropy, against a word set small enough to be
a file in the repository and a tail short enough to type on a phone.
@@ -11,13 +11,8 @@ base64 string is not. The job is identical (it *is* the password to the
stream); only the shape is chosen for a human.
The list is vendored (`wordlist.txt`, 7776 lines) so a stream never needs the
network to start. Four of those lines are hyphenated (`drop-down`, `felt-tip`,
`t-shirt`, `yo-yo`) and are dropped on load: a hyphen is the phrase's own
separator, so a phrase built from one would be six tokens and a listener could
not hear where the tail starts. 7772 words costs 0.003 bits.
`generate` uses `SystemRandom`/`secrets`, so the phrase is not reproducible
from the process seed.
network to start. `generate` uses `SystemRandom`/`secrets`, so the phrase is
not reproducible from the process seed.
"""
from __future__ import annotations
@@ -29,29 +24,18 @@ import secrets
from pathlib import Path
#: Four words plus the tail. The plan's answer to "how many"; four is
#: `4 * log2(7772)` ~= 51.7 bits on its own, and the tail pushes it past 77.
#: `4 * log2(7776)` ~= 51.7 bits on its own, and the tail pushes it past 77.
WORD_COUNT = 4
SUFFIX_LEN = 5
#: Lowercase letters and digits: what a person can retype without a shift key,
#: and 36 symbols is still `5 * log2(36)` ~= 25.8 bits.
SUFFIX_ALPHABET = "abcdefghijklmnopqrstuvwxyz0123456789"
#: The smallest token a word may be. The wordlist's own floor is 3, so this
#: rejects nothing `generate` can produce and everything an operator might type
#: as a stand-in ("a-b-c-d-12345").
MIN_WORD_LEN = 3
#: A phrase is exactly what `generate` makes: `WORD_COUNT` alphabetic words of
#: at least `MIN_WORD_LEN` characters, then a `SUFFIX_LEN` tail of `[a-z0-9]`.
#: An operator-supplied `--phrase` is a password for a LAN-facing stream, so it
#: is held to the generated shape rather than to a looser "looks like a phrase"
#: pattern -- a looser one accepts `a-b-c`, and nothing stops an operator from
#: typing that.
_SHAPE = re.compile(
rf"^[a-z]{{{MIN_WORD_LEN},}}(?:-[a-z]{{{MIN_WORD_LEN},}}){{{WORD_COUNT - 1}}}"
rf"-[a-z0-9]{{{SUFFIX_LEN}}}$"
)
#: Only the words whose letters are the whole line; see the module docstring.
_WORD_SHAPE = re.compile(r"^[a-z]+$")
#: A phrase is one or more hyphen-joined tokens of lowercase letters/digits,
#: at least three of them (so a caller-supplied phrase cannot be a single word
#: or a passcode-free stub). Generated phrases are five tokens; this is the
#: looser shape a `--phrase` must satisfy.
_SHAPE = re.compile(r"^[a-z0-9]+(?:-[a-z0-9]+){2,}$")
MAX_LENGTH = 96
_WORDLIST = Path(__file__).with_name("wordlist.txt")
@@ -59,11 +43,7 @@ _cache: tuple[str, ...] | None = None
def words() -> tuple[str, ...]:
"""The usable wordlist, read once and remembered.
Only plain `[a-z]+` lines survive: the EFF list contains four hyphenated
entries and the hyphen is the phrase's separator, so one of them would make
a six-token phrase nobody could read out correctly.
"""The wordlist, read once and remembered.
Raises `RuntimeError` on an empty file rather than returning nothing: a
generator that picks from an empty list would raise a confusing `IndexError`
@@ -72,10 +52,9 @@ def words() -> tuple[str, ...]:
global _cache
if _cache is None:
text = _WORDLIST.read_text(encoding="utf-8")
stripped = (line.strip() for line in text.splitlines())
loaded = tuple(word for word in stripped if _WORD_SHAPE.match(word))
loaded = tuple(word.strip() for word in text.splitlines() if word.strip())
if not loaded:
raise RuntimeError(f"wordlist {_WORDLIST} has no usable words")
raise RuntimeError(f"wordlist {_WORDLIST} is empty")
_cache = loaded
return _cache
@@ -112,12 +91,12 @@ def validate(phrase: str) -> str:
raise ValueError(f"phrase is {len(normalized)} characters; the maximum is {MAX_LENGTH}")
if not _SHAPE.match(normalized):
raise ValueError(
f"phrase must be {WORD_COUNT} lowercase words of at least "
f"{MIN_WORD_LEN} letters plus a {SUFFIX_LEN}-character [a-z0-9] "
f"tail (e.g. amber-otter-lantern-harbor-7k2qp), not {phrase!r}"
"phrase must be lowercase words/digits joined by hyphens, at least "
"three tokens (e.g. four-words-and-a-tail), not "
f"{phrase!r}"
)
return normalized
__all__ = ["WORD_COUNT", "SUFFIX_LEN", "MIN_WORD_LEN", "generate", "normalize",
"validate", "entropy_bits", "words", "MAX_LENGTH"]
__all__ = ["WORD_COUNT", "SUFFIX_LEN", "generate", "normalize", "validate",
"entropy_bits", "words", "MAX_LENGTH"]
+8 -21
View File
@@ -31,12 +31,9 @@ log = logging.getLogger("hhstream.live")
#: the box is also running the compositor and the main publisher.
DEFAULT_FPS = 5
FPS_CAP = 10
#: After this many *consecutive* capture failures a stream gives up and closes
#: its viewers. `grim` against a removed output ("unknown output") fails every
#: time, and retrying forever would be a busy loop on a dead session -- which is
#: as true after a thousand good frames as it is from a cold start, so the count
#: is of failures in a row, not of failures-before-the-first-frame. A successful
#: grab resets it; giving up does not reset, because a dead output stays dead.
#: After this many capture failures with no successful frame, a stream gives up
#: and closes its viewers. `grim` against a removed output ("unknown output")
#: fails every time, and retrying forever would be a busy loop on a dead session.
FAILURE_LIMIT = 10
@@ -98,11 +95,8 @@ class Stream:
# ---- lifecycle ---------------------------------------------------
def start(self) -> None:
# Named for the output, never the phrase: faulthandler, the default
# `threading.excepthook` and any profiler that samples the process all
# print thread names, and the phrase is this stream's only credential.
self._thread = threading.Thread(
target=self._run, name=f"hhstream-live-{self.spec.output}", daemon=True
target=self._run, name=f"hhstream-live-{self.spec.phrase}", daemon=True
)
self._thread.start()
@@ -135,22 +129,15 @@ class Stream:
with self._cond:
self._failures += 1
self._consecutive_failures += 1
give_up = self._consecutive_failures >= FAILURE_LIMIT
give_up = self._consecutive_failures >= FAILURE_LIMIT and self._captured == 0
if give_up:
# Sticky on purpose: a stream that reached the cap is
# not re-armed by `add_viewer`, because the case this
# guards -- an output that was removed -- does not come
# back. Re-enabling the session is how it comes back.
self._stopped = True
self._cond.notify_all()
if give_up:
# By output, not by phrase: a log line is the easiest place
# for a credential to escape to.
log.warning("stream on %s: giving up after %d consecutive "
"capture failures: %s",
self.spec.output, FAILURE_LIMIT, exc)
log.warning("stream %s: giving up after %d capture failures: %s",
self.spec.phrase, FAILURE_LIMIT, exc)
return
log.debug("stream on %s: capture failed: %s", self.spec.output, exc)
log.debug("stream %s: capture failed: %s", self.spec.phrase, exc)
if self._sleep(0.5):
return
continue
+14 -91
View File
@@ -22,7 +22,6 @@ nothing a dependency would make easier.
from __future__ import annotations
import logging
import socket
import threading
import time
from collections import deque
@@ -44,11 +43,6 @@ DEFAULT_HOST = "127.0.0.1"
#: Wrong-phrase lookups allowed per client per window before 429.
RATE_LIMIT = 30
RATE_WINDOW_S = 60.0
#: How long a connection may stay silent before it is dropped. It covers the
#: request line and the headers only: a client that connects and says nothing
#: would otherwise hold a handler thread in `readline` for ever, and on the
#: MJPEG route it would hold a viewer slot -- which keeps `grim` running.
HEADER_TIMEOUT_S = 15.0
class RateLimiter:
@@ -57,10 +51,6 @@ class RateLimiter:
A phrase carries ~77 bits, so guessing is already hopeless; this is here so
that a LAN attacker cannot even *try* at line speed, and so the attempt is
visible in one place. Only misses are counted.
The table holds only clients that missed inside the current window: the key
is a client address, which on a LAN is the attacker's to choose, so a table
that only ever grew would be a slow memory leak driven from the network.
"""
def __init__(self, limit: int = RATE_LIMIT, window: float = RATE_WINDOW_S,
@@ -73,29 +63,16 @@ class RateLimiter:
def allow(self, key: str) -> bool:
now = self._clock()
cutoff = now - self.window
with self._lock:
# Prune every key, not just this one. Pruning only `key` would leave
# a client that missed once and never came back in the table for the
# life of the process; the sweep is over the clients seen in the
# last window, which is what the limit already bounds.
for seen, hits in list(self._hits.items()):
while hits and hits[0] < cutoff:
hits.popleft()
if not hits:
del self._hits[seen]
hits = self._hits.setdefault(key, deque())
cutoff = now - self.window
while hits and hits[0] < cutoff:
hits.popleft()
if len(hits) >= self.limit:
return False
hits.append(now)
return True
@property
def tracked(self) -> int:
"""How many clients the table is holding. For the leak test and status."""
with self._lock:
return len(self._hits)
def reset(self) -> None:
with self._lock:
self._hits.clear()
@@ -116,9 +93,6 @@ class _Handler(BaseHTTPRequestHandler):
protocol_version = "HTTP/1.1"
server_version = "hhstream-live"
sys_version = ""
#: `StreamRequestHandler.setup` applies this to the socket. Without it a
#: client that opens a connection and sends nothing pins a handler thread.
timeout = HEADER_TIMEOUT_S
# ---- plumbing ----------------------------------------------------
@@ -126,31 +100,8 @@ class _Handler(BaseHTTPRequestHandler):
def registry(self) -> StreamRegistry:
return self.server.registry # type: ignore[attr-defined]
def setup(self) -> None:
super().setup()
# Keepalive is for the other half of the same problem: a viewer whose
# network black-holes never closes, so `sendall` blocks and the viewer
# count never drops -- which keeps `grim` capturing for nobody. The
# kernel's probes end that connection instead.
try:
self.connection.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
except OSError as exc: # a platform without it is not worth failing over
log.debug("could not enable TCP keepalive: %s", exc)
def log_message(self, format: str, *args) -> None: # noqa: A002 - base signature
"""Log that something happened, never *what was asked for*.
The base class formats `self.requestline` into almost every message it
emits, and the request line contains the phrase -- the stream's only
credential. So the format string and its arguments are dropped here and
the method is logged on its own; `log_request` adds the status.
"""
log.debug("%s %s", self.address_string(), self.command or "-")
def log_request(self, code="-", size="-") -> None:
if isinstance(code, HTTPStatus):
code = code.value
log.debug("%s %s -> %s", self.address_string(), self.command or "-", code)
log.debug("%s %s", self.address_string(), format % args)
def _send(self, status: HTTPStatus, body: bytes, content_type: str,
headers: dict[str, str] | None = None) -> None:
@@ -177,8 +128,7 @@ class _Handler(BaseHTTPRequestHandler):
except (BrokenPipeError, ConnectionResetError):
pass # the viewer went away mid-write; that is not an error
except OSError as exc:
# Not the path: it carries the phrase.
log.debug("client error from %s: %s", self.address_string(), exc)
log.debug("client error on %s: %s", path, exc)
def _route(self, path: str) -> None:
if path in ("", "/"):
@@ -241,19 +191,12 @@ class _Handler(BaseHTTPRequestHandler):
self.send_header("Content-Type", f"multipart/x-mixed-replace; boundary={BOUNDARY}")
self.send_header("Cache-Control", "no-store")
self.send_header("X-Content-Type-Options", "nosniff")
# The connection *is* the stream: it has no `Content-Length` and is not
# chunked, so "the response ended" can only mean "the connection ended".
# HTTP/1.1 keeps a connection alive by default, and an intermediary that
# believed that would be entitled to buffer this response until an EOF
# that never comes. Saying so has to happen before `end_headers`, or it
# is not a header at all -- `send_header` also sets `close_connection`.
self.send_header("Connection", "close")
self.end_headers()
# The header timeout has done its job; from here the handler waits on
# frames, not on the client, and a quiet minute is normal.
self.connection.settimeout(None)
stream.add_viewer()
# The connection is the stream: after this handler returns there is no
# next request to read, so it must not be kept alive.
self.close_connection = True
last_seq = 0
try:
while True:
@@ -282,9 +225,6 @@ class _Handler(BaseHTTPRequestHandler):
class LiveServer:
"""Owns the listener and its lifetime. One instance, many streams."""
#: Binds that mean "every interface" rather than one address of its own.
WILDCARDS = ("", "0.0.0.0", "::")
def __init__(self, registry: StreamRegistry, *, host: str = DEFAULT_HOST,
port: int = DEFAULT_PORT, limiter: RateLimiter | None = None) -> None:
self.registry = registry
@@ -302,28 +242,12 @@ class LiveServer:
return self.requested_port
return self._httpd.server_address[1]
@property
def binds_everywhere(self) -> bool:
return self.host in self.WILDCARDS
@property
def address(self) -> str:
"""A host a browser can dial. A wildcard bind has none of its own, so
the loopback name stands in; `describe_bind` is what tells the truth
about the reach."""
return "127.0.0.1" if self.binds_everywhere else self.host
def describe_bind(self) -> str:
"""One line naming the real bind, and saying when it is every interface.
`url_for` has to print something dialable, which for `0.0.0.0` is a lie
of omission -- the stream is on the LAN, not on loopback. This is where
that is said out loud.
"""
where = f"{self.host or '0.0.0.0'}:{self.port}"
if self.binds_everywhere:
return f"bound to {where} -- all interfaces, reachable from the LAN"
return f"bound to {where}"
advertised = self.host
if advertised in ("0.0.0.0", "::"):
advertised = "127.0.0.1"
return advertised
def url_for(self, phrase: str) -> str:
return f"http://{self.address}:{self.port}/{phrase}/"
@@ -340,7 +264,7 @@ class LiveServer:
name="hhstream-live-http", daemon=True)
self._thread.start()
self.ready.set()
log.info("listening, %s", self.describe_bind())
log.info("listening on http://%s:%d", self.host, self.port)
def stop(self) -> None:
if self._httpd is not None:
@@ -353,5 +277,4 @@ class LiveServer:
self.registry.stop_all()
__all__ = ["LiveServer", "RateLimiter", "DEFAULT_PORT", "DEFAULT_HOST", "BOUNDARY",
"HEADER_TIMEOUT_S"]
__all__ = ["LiveServer", "RateLimiter", "DEFAULT_PORT", "DEFAULT_HOST", "BOUNDARY"]
+2 -12
View File
@@ -25,19 +25,11 @@ PHRASE = "amber-otter-lantern-harbor-7k2qp"
class StubCapture:
"""A `Capture` under the test's control: counts grabs, can be told to fail.
"""A `Capture` under the test's control: counts grabs, can be told to fail."""
`fail_first` is a capture that is broken from the start (a cold start that
never works); `fail_after` is one that works and then stops -- the output
was removed while someone was watching, which is grim's "unknown output" on
every grab from then on.
"""
def __init__(self, frames=None, fail_first: int = 0, delay: float = 0.0,
fail_after: int | None = None):
def __init__(self, frames=None, fail_first: int = 0, delay: float = 0.0):
self.frames = list(frames) if frames else [jpeg_bytes(i) for i in range(1, 50)]
self.fail_first = fail_first
self.fail_after = fail_after
self.delay = delay
self.calls = 0
self._lock = threading.Lock()
@@ -50,8 +42,6 @@ class StubCapture:
time.sleep(self.delay)
if call <= self.fail_first:
raise RuntimeError(f"stub capture failing on purpose (call {call})")
if self.fail_after is not None and call > self.fail_after:
raise RuntimeError(f"stub capture: unknown output (call {call})")
return self.frames[(call - 1 - self.fail_first) % len(self.frames)]
+2 -77
View File
@@ -3,22 +3,10 @@
from __future__ import annotations
import io
import os
import pathlib
import re
import signal
import subprocess
import sys
import time
from contextlib import redirect_stdout
import pytest
from hhstream_live.__main__ import (DEFAULT_CAPTURE, capture_argv, main, parse_args,
parse_stream)
#: What `hhstream-live phrase` must print: four alphabetic words and a tail.
PHRASE_SHAPE = re.compile(r"^[a-z]+(?:-[a-z]+){3}-[a-z0-9]{5}$")
_SRC = pathlib.Path(__file__).resolve().parents[2] / "src"
from hhstream_live.__main__ import DEFAULT_CAPTURE, capture_argv, main, parse_stream
def test_capture_command_is_aimed_by_inserting_the_output_before_stdout():
@@ -52,9 +40,7 @@ def test_the_phrase_subcommand_prints_a_phrase():
with redirect_stdout(out):
assert main(["phrase"]) == 0
printed = out.getvalue().strip()
# Not just `count("-") == 4`: a hyphenated word on the list would satisfy
# the count with five words and four hyphens in the wrong places.
assert PHRASE_SHAPE.match(printed), printed
assert printed.count("-") == 4 # four words + tail
def test_the_phrase_subcommand_can_print_several():
@@ -62,64 +48,3 @@ def test_the_phrase_subcommand_can_print_several():
with redirect_stdout(out):
main(["phrase", "--count", "3"])
assert len(out.getvalue().split()) == 3
# ---- the default subcommand, and shutdown ----------------------------------
def test_a_bare_command_line_means_serve():
"""`hhstream-live` with no subcommand is `hhstream-live serve`.
The serve options live on the `serve` subparser, so the namespace has to be
the subparser's; reading `capture_command` off the top-level one is the
`AttributeError` this guards.
"""
args = parse_args([])
assert args.command == "serve"
assert args.capture_command is None
assert args.port and args.bind and args.stream == []
assert args.log_level == "info"
def test_serve_options_work_without_naming_the_subcommand():
args = parse_args(["--port", "0", "--bind", "0.0.0.0"])
assert (args.command, args.port, args.bind) == ("serve", 0, "0.0.0.0")
def test_a_global_option_survives_the_subcommand():
"""`--log-level` is on both parsers; the subparser must not reset it."""
assert parse_args(["--log-level", "debug"]).log_level == "debug"
assert parse_args(["--log-level", "debug", "serve"]).log_level == "debug"
assert parse_args(["--log-level", "debug", "phrase"]).log_level == "debug"
def test_a_bare_run_serves_and_sigterm_stops_it_cleanly():
"""The whole path: no subcommand, then the signal a service is stopped with.
`SIGTERM`'s default action kills the process outright, so without a handler
`server.stop()` never runs and the exit status is a signal, not a 0.
"""
env = dict(os.environ, PYTHONPATH=os.pathsep.join(
[str(_SRC), os.environ.get("PYTHONPATH", "")]).rstrip(os.pathsep))
proc = subprocess.Popen(
[sys.executable, "-m", "hhstream_live", "--port", "0"],
stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, env=env,
)
try:
deadline = time.monotonic() + 20
banner = ""
while "landing page:" not in banner and time.monotonic() < deadline:
line = proc.stderr.readline()
if not line:
break
banner += line
assert "landing page:" in banner, banner
assert "bound to 127.0.0.1:" in banner, banner
proc.send_signal(signal.SIGTERM)
out, err = proc.communicate(timeout=20)
except BaseException:
proc.kill()
proc.communicate()
raise
assert proc.returncode == 0, (proc.returncode, banner, err)
assert "Traceback" not in err, err
+4 -46
View File
@@ -8,7 +8,6 @@ import pytest
from hhstream_live import phrase
SHAPE = re.compile(r"^[a-z]+(-[a-z]+){3}-[a-z0-9]{5}$")
WORD = re.compile(r"^[a-z]+$")
def test_generated_phrase_has_the_agreed_shape():
@@ -25,24 +24,11 @@ def test_entropy_is_what_the_plan_claims():
assert phrase.entropy_bits() > 75
def test_wordlist_is_the_eff_large_list_minus_its_hyphenated_entries():
def test_wordlist_is_the_eff_large_list():
words = phrase.words()
# The file is the EFF large list, 7776 lines; four of them are hyphenated
# and the loader drops them, because a hyphen is the phrase's separator.
assert len(words) == 7772
assert len(set(words)) == 7772
assert len(words) == 7776
assert len(set(words)) == 7776
assert "abacus" in words
assert "drop-down" not in words
def test_every_loaded_word_is_plain_lowercase_letters():
"""Not a sampling test: one hyphenated word makes a six-token phrase.
`test_generated_phrase_has_the_agreed_shape` would only catch that by luck
-- four draws out of 7776 hit a hyphenated entry about 0.2% of the time --
so the property is asserted over the whole list instead.
"""
assert all(WORD.match(word) for word in phrase.words())
def test_generate_is_not_reproducible_from_a_seed():
@@ -61,15 +47,7 @@ def test_normalize_lowercases_and_trims():
def test_validate_accepts_a_good_phrase_and_returns_the_normalized_form():
assert phrase.validate("Amber-Otter-Lantern-Harbor-7K2QP") == (
"amber-otter-lantern-harbor-7k2qp"
)
def test_validate_accepts_everything_generate_produces():
for _ in range(50):
generated = phrase.generate()
assert phrase.validate(generated) == generated
assert phrase.validate("Amber-Otter-Lantern-7K2QP") == "amber-otter-lantern-7k2qp"
@pytest.mark.parametrize("bad", ["", "single", "two-tokens", "has space-phrase-x",
@@ -80,26 +58,6 @@ def test_validate_rejects_a_phrase_that_is_not_the_shape(bad):
phrase.validate(bad)
@pytest.mark.parametrize("weak", [
"a-b-c", # the reviewer's example: three letters
"ab-cd-ef-gh-12345", # four "words" of two letters
"amber-otter-lantern-7k2qp", # three words, not four
"amber-otter-lantern-harbor", # four words, no tail
"amber-otter-lantern-harbor-7k2", # a three-character tail
"amber-otter-lantern-harbor-7k2qpz", # a six-character tail
"amber-otter-lantern-harbor-hill-7k2qp", # five words
"amber-0tter-lantern-harbor-7k2qp", # a digit inside a word
])
def test_validate_rejects_a_phrase_weaker_than_the_generated_shape(weak):
"""`--phrase` is a password for a LAN-facing stream, not a label.
Anything `validate` accepts is something an operator can set, so the floor
has to be the shape `generate` produces -- otherwise `a-b-c` opens a stream.
"""
with pytest.raises(ValueError):
phrase.validate(weak)
def test_validate_rejects_an_overlong_phrase():
with pytest.raises(ValueError):
phrase.validate("-".join(["averyverylongword"] * 8))
-59
View File
@@ -2,7 +2,6 @@
from __future__ import annotations
import logging
import time
import pytest
@@ -95,64 +94,6 @@ def test_a_dead_capture_gives_up_and_closes_viewers():
stream.stop()
def test_a_capture_that_dies_after_a_good_frame_also_gives_up():
"""The cap counts failures *in a row*, not failures before the first frame.
An output removed while someone is watching fails every grab from then on.
A give-up rule that only applied to a cold start would leave this stream
retrying twice a second for as long as the viewer held the page open --
exactly the busy loop on a dead session the cap exists to stop.
"""
capture = StubCapture(fail_after=1) # one good frame, then the output is gone
stream = make_stream(capture, fps=10)
stream.start()
try:
stream.add_viewer() # and the viewer stays attached for the whole test
frame, seq = stream.wait_frame(0, timeout=5)
assert frame is not None and seq >= 1
# Poll through the viewer's own wait, so a `wait_frame` that hangs is a
# failure here rather than a test that quietly waits for the deadline.
deadline = time.monotonic() + 15
while not stream.stopped and time.monotonic() < deadline:
assert stream.wait_frame(seq, timeout=1.0) == (None, seq)
assert stream.stopped, "a dead output must stop the stream, frame or no frame"
# And the viewer in the loop is released at once rather than on timeout.
started = time.monotonic()
assert stream.wait_frame(seq, timeout=10.0) == (None, seq)
assert time.monotonic() - started < 1.0
assert capture.calls >= FAILURE_LIMIT
finally:
stream.stop()
def test_giving_up_never_writes_the_phrase_to_the_log(caplog):
"""A log line is the easiest place for the stream's one credential to leak.
faulthandler and `threading.excepthook` print thread names, so the capture
thread is checked here too.
"""
capture = StubCapture(fail_first=10_000)
stream = make_stream(capture, fps=10)
with caplog.at_level(logging.DEBUG, logger="hhstream.live"):
stream.start()
thread_name = stream._thread.name
try:
stream.add_viewer()
assert stream.wait_frame(0, timeout=15) == (None, 0)
assert stream.stopped
finally:
stream.stop()
messages = [record.getMessage() for record in caplog.records]
assert messages, "the give-up must be logged at all"
assert any("hh-demo" in message for message in messages)
for message in messages:
assert PHRASE not in message, message
assert PHRASE not in thread_name and "hh-demo" in thread_name
# ---- registry --------------------------------------------------------------
def test_registry_adds_and_finds_a_phrase_case_insensitively():
+1 -105
View File
@@ -2,13 +2,11 @@
from __future__ import annotations
import logging
import socket
import threading
import time
from live_helpers import PHRASE, StubCapture, read_parts
from hhstream_live.server import BOUNDARY, HEADER_TIMEOUT_S, LiveServer, RateLimiter
from hhstream_live.server import BOUNDARY
WRONG = "no-such-phrase-here-00000"
@@ -96,10 +94,6 @@ def test_stream_is_mjpeg_and_delivers_valid_jpeg_parts(serve, http_conn):
assert resp.getheader("Content-Type") == (
f"multipart/x-mixed-replace; boundary={BOUNDARY}"
)
# The stream has no length and is not chunked, so the only thing that
# can delimit it is the close -- which an intermediary is only entitled
# to expect if the response says so.
assert resp.getheader("Connection") == "close"
parts = read_parts(resp, 2)
assert len(parts) == 2
for part in parts:
@@ -156,104 +150,6 @@ def test_no_capture_until_a_request_arrives(serve):
assert capture.calls >= 1
def test_a_silent_client_does_not_pin_a_handler_thread(serve):
"""A connection that never sends a request line must not hold a thread.
The socket timeout is the only thing that ends it; without one the handler
sits in `readline` for the life of the process.
"""
assert HEADER_TIMEOUT_S <= 30 # a cap that never fires is not a cap
with serve() as (server, _):
before = threading.active_count()
sock = socket.create_connection(("127.0.0.1", server.port), timeout=5)
try:
time.sleep(0.2)
assert threading.active_count() > before # a handler is waiting on us
sock.settimeout(HEADER_TIMEOUT_S + 10)
started = time.monotonic()
# The server closes on timeout, which arrives here as EOF.
assert sock.recv(4096) == b""
waited = time.monotonic() - started
assert waited <= HEADER_TIMEOUT_S + 5, waited
finally:
sock.close()
def test_a_viewer_that_drops_mid_stream_releases_its_slot(serve, http_conn):
"""A dropped viewer must bring `viewers` back to 0, or grim keeps running."""
import json
with serve(fps=10) as (server, _):
sock = socket.create_connection(("127.0.0.1", server.port), timeout=10)
sock.sendall(f"GET /{PHRASE}/stream.mjpg HTTP/1.1\r\n"
f"Host: 127.0.0.1\r\n\r\n".encode("ascii"))
assert sock.recv(4096) # headers, and at least the start of a part
def viewers() -> int:
return json.loads(get(http_conn(server), f"/{PHRASE}/status.json")[2])["viewers"]
# The headers can reach us a moment before the handler counts itself in.
deadline = time.monotonic() + 10
while viewers() == 0 and time.monotonic() < deadline:
time.sleep(0.05)
assert viewers() == 1
sock.close() # the phone walked out of range
deadline = time.monotonic() + 10
while viewers() and time.monotonic() < deadline:
time.sleep(0.1)
assert viewers() == 0
def test_the_access_log_never_carries_the_phrase(serve, http_conn, caplog):
"""Debug logging is on in development, and the request line is the phrase."""
with caplog.at_level(logging.DEBUG, logger="hhstream.live"):
with serve() as (server, _):
assert get(http_conn(server), f"/{PHRASE}/status.json")[0] == 200
assert get(http_conn(server), f"/{WRONG}/")[0] == 404
messages = [record.getMessage() for record in caplog.records]
assert messages, "something must be logged, or this proves nothing"
for message in messages:
assert PHRASE not in message, message
assert WRONG not in message, message
def test_the_server_says_when_it_is_bound_to_every_interface(registry):
"""`url_for` must print something dialable; the bind line tells the truth."""
loopback = LiveServer(registry, host="127.0.0.1", port=9988)
assert loopback.describe_bind() == "bound to 127.0.0.1:9988"
assert not loopback.binds_everywhere
everywhere = LiveServer(registry, host="0.0.0.0", port=9988)
assert everywhere.binds_everywhere
assert everywhere.address == "127.0.0.1" # still dialable
described = everywhere.describe_bind()
assert "0.0.0.0:9988" in described and "all interfaces" in described
def test_the_rate_limiter_counts_each_client_separately_and_forgets_them():
"""One noisy client must not spend another's budget, and must not be kept.
`_hits` is keyed by client address, so a table that only grew would be a
memory leak any LAN host could drive.
"""
now = [0.0]
limiter = RateLimiter(limit=2, window=60.0, clock=lambda: now[0])
assert limiter.allow("10.0.0.1")
assert limiter.allow("10.0.0.1")
assert not limiter.allow("10.0.0.1") # that client is out of budget
assert limiter.allow("10.0.0.2") # this one is not
assert limiter.tracked == 2
now[0] += 61.0 # both windows have passed
assert limiter.allow("10.0.0.1")
assert limiter.tracked == 1, "a client that went quiet is not kept"
def test_repeated_wrong_phrases_are_rate_limited(serve, http_conn):
with serve(limit=3) as (server, _):
conn = http_conn(server)
+13 -38
View File
@@ -75,18 +75,12 @@ def test_a_failure_after_a_good_frame_keeps_the_known_size():
def test_env_is_overlaid_on_our_own_environment():
argv = [
sys.executable,
"-c",
"import os,sys;sys.stdout.write(os.environ['WAYLAND_DISPLAY']+':'+"
+ "('PATH' in os.environ and 'haspath' or 'nopath'))",
]
before = os.environ.get("WAYLAND_DISPLAY")
argv = [sys.executable, "-c",
"import os,sys;sys.stdout.write(os.environ['WAYLAND_DISPLAY']+':'+"
"('PATH' in os.environ and 'haspath' or 'nopath'))"]
capture = CommandCapture(argv, env={"WAYLAND_DISPLAY": "wayland-9"})
assert capture.grab() == b"wayland-9:haspath"
assert os.environ.get("WAYLAND_DISPLAY") == before, (
"our own environment was mutated"
)
assert "WAYLAND_DISPLAY" not in os.environ, "our own environment was mutated"
def test_empty_argv_and_bad_timeout_are_rejected_up_front():
@@ -97,10 +91,8 @@ def test_empty_argv_and_bad_timeout_are_rejected_up_front():
def test_the_helpers_stdin_is_closed_so_it_cannot_steal_ours():
code = (
"import sys;"
"sys.stdout.buffer.write(b'EMPTY' if not sys.stdin.read() else b'DATA')"
)
code = ("import sys;"
"sys.stdout.buffer.write(b'EMPTY' if not sys.stdin.read() else b'DATA')")
assert CommandCapture([sys.executable, "-c", code]).grab() == b"EMPTY"
@@ -110,32 +102,15 @@ def test_image_size_reads_png_headers(width, height):
def test_image_size_reads_a_jpeg_sof_header():
jpeg = (
b"\xff\xd8"
+ b"\xff\xe0"
+ (16).to_bytes(2, "big")
+ b"JFIF\x00"
+ b"\x00" * 9
+ b"\xff\xc0"
+ (17).to_bytes(2, "big")
+ b"\x08"
+ (720).to_bytes(2, "big")
+ (1280).to_bytes(2, "big")
+ b"\x03"
+ b"\x00" * 9
)
jpeg = (b"\xff\xd8"
+ b"\xff\xe0" + (16).to_bytes(2, "big") + b"JFIF\x00" + b"\x00" * 9
+ b"\xff\xc0" + (17).to_bytes(2, "big") + b"\x08"
+ (720).to_bytes(2, "big") + (1280).to_bytes(2, "big")
+ b"\x03" + b"\x00" * 9)
assert image_size(jpeg) == (1280, 720)
@pytest.mark.parametrize(
"data",
[
b"",
b"not an image",
b"\x89PNG\r\n\x1a\n",
b"\xff\xd8",
b"\xff\xd8\xff\xc0\x00\x11",
],
)
@pytest.mark.parametrize("data", [b"", b"not an image", b"\x89PNG\r\n\x1a\n",
b"\xff\xd8", b"\xff\xd8\xff\xc0\x00\x11"])
def test_image_size_returns_none_when_it_cannot_tell(data):
assert image_size(data) is None