From 9744d35945fd5ddb63d0e1437cba684d5ff18734 Mon Sep 17 00:00:00 2001 From: Hermenix Date: Tue, 6 Oct 2026 13:31:41 +0000 Subject: [PATCH 1/8] feat(live): one port, a router, and a phrase per session (stage 1) The session-stream web app: a single HTTP listener (9988) that serves a page per enabled session, keyed by a spoken phrase. Off by default, so a stream exists only for the sessions it was enabled on. src/hhstream_live/phrase.py four EFF-large words + a 5-char base-36 tail (~77.5 bits); the phrase is the password src/hhstream_live/publisher.py one capture thread + one frame slot per output; one slot/no queue; capture paused at zero viewers src/hhstream_live/server.py the router: /, //, /stream.mjpg, /frame.jpg, /status.json; unknown phrase is an empty 404, rate-limited src/hhstream_live/pages.py the built-in dark pages (no CDN, no build) src/hhstream_live/__main__.py hhstream-live serve|phrase, standalone The root page is a form, never a list: the phrase is known only by whoever enabled the stream. Transport is MJPEG (multipart/x-mixed-replace), reusing the JPEG grim already produces. Not yet wired to the broker -- that is stage 2. Tests: tests/live (51). --- docs/plans/2026-10-03-network-stream.md | 129 + .../plans/2026-10-04-session-stream-webapp.md | 175 + nix/package.nix | 4 + pyproject.toml | 10 + src/hhstream_live/__init__.py | 15 + src/hhstream_live/__main__.py | 139 + src/hhstream_live/pages.py | 115 + src/hhstream_live/phrase.py | 102 + src/hhstream_live/publisher.py | 264 + src/hhstream_live/server.py | 280 + src/hhstream_live/wordlist.txt | 7776 +++++++++++++++++ tests/live/conftest.py | 86 + tests/live/fake_jpeg.py | 31 + tests/live/live_helpers.py | 82 + tests/live/test_live_cli.py | 50 + tests/live/test_live_phrase.py | 63 + tests/live/test_live_publisher.py | 125 + tests/live/test_live_server.py | 165 + 18 files changed, 9611 insertions(+) create mode 100644 docs/plans/2026-10-03-network-stream.md create mode 100644 docs/plans/2026-10-04-session-stream-webapp.md create mode 100644 src/hhstream_live/__init__.py create mode 100644 src/hhstream_live/__main__.py create mode 100644 src/hhstream_live/pages.py create mode 100644 src/hhstream_live/phrase.py create mode 100644 src/hhstream_live/publisher.py create mode 100644 src/hhstream_live/server.py create mode 100644 src/hhstream_live/wordlist.txt create mode 100644 tests/live/conftest.py create mode 100644 tests/live/fake_jpeg.py create mode 100644 tests/live/live_helpers.py create mode 100644 tests/live/test_live_cli.py create mode 100644 tests/live/test_live_phrase.py create mode 100644 tests/live/test_live_publisher.py create mode 100644 tests/live/test_live_server.py diff --git a/docs/plans/2026-10-03-network-stream.md b/docs/plans/2026-10-03-network-stream.md new file mode 100644 index 0000000..4c60882 --- /dev/null +++ b/docs/plans/2026-10-03-network-stream.md @@ -0,0 +1,129 @@ +> **Superseded (2026-10-04):** the operator asked for one port (9988) with a +> router and a phrase per session, only for enabled sessions — see +> `docs/plans/2026-10-04-session-stream-webapp.md`. This file is kept for its +> transport table and its capture/idle-clock reasoning, which the new plan +> inherits. + +I'll explore the frame server, session CLI, broker, contracts, and the NixOS module first, then write a plan only — no edits.The tree is a Python frame server, a session broker, and a pointer daemon. Next I’ll read the contracts and the capture/session paths those decisions depend on.The publisher cannot select a session today. `hhstream-server` serves one output on `/run/hypr-stream/stream.sock` and its command line is frozen. `hhstream shot --output hh-` and `record --output` are separate `grim -o` one-shots. A phone view should be a new, view-only listener beside that, not a change to the socket Hermenix already reads. + +## Decisions + +### 1. Transport + +**Primary: MJPEG over HTTP.** **Fallback: a WebSocket carrying the existing `HHS1` frames, only if a real phone browser will not render the MJPEG.** Not HLS, not VNC, not WebRTC. + +| Option | Latency | CPU on this box | Already here | Plain browser | +|---|---|---|---|---| +| MJPEG (`multipart/x-mixed-replace`) | One frame. At the recommended 5 fps that is about 200 ms plus a ~46 ms `grim` JPEG. | Reuses the JPEG the host already measured: 1280×720 q80 is ~46 ms; PNG is ~326 ms and is the wrong codec. One capture, many viewers. | `CommandCapture`, the single-slot drop policy, JPEG size parsing. Stdlib HTTP only. | Yes. An `` needs no plugin and no script. | +| WebSocket of `HHS1` | Same pixels, same latency. | Same capture cost. | The framing code. | No. A phone that opens the URL sees nothing unless we also serve a page that blits JPEGs. | +| HLS | Seconds (segment delay). | A second encode (`libx264`) on top of `grim`. `ffmpeg` is only used today to mux a finished recording. | Nothing of the live path. | Safari yes, Chrome only with a script. | +| VNC / RFB | Fine. | A new codec, and a client can click. | Nothing. | No. noVNC is a script, and clicks would bypass the pointer lease. | +| WebRTC | Best, and irrelevant. | An encoder plus ICE/DTLS. Worst cost for a two-core software renderer. | Nothing. | Only via a script and a signaling dance. | + +The URL we print is a tiny same-origin HTML page whose only job is ``. Top-level multipart documents are unreliable; the image tag is the one that works on phones. The same process also serves one `frame.jpg` for tests. No CDN, no extra Python dependencies. The WebSocket fallback is the same JPEG bytes and the same page, and it is not built in the first slice. + +Capture rate for a session stream defaults to **5 fps, hard cap 10**. The main publisher already spends roughly one core on its own `grim`. A second full-rate capture would take the other core. Five frames times ~46 ms is about a quarter of one core, and only while someone is connected. + +### 2. Where the listener lives + +**A new long-lived unit, `hhstream-live.service`. One process, many session outputs. Not the frame server, not the broker, and not one unit per session.** + +It follows the pointer driver, not the frame server: + +- The frame server's contract is one socket, one output, one slot, and a command line of `--socket`, `--fps`, `--format`, and `--capture-command`. Clients send nothing. A LAN parser bug or a slow phone must not be able to take down the socket Hermenix reads. The server also does not know session ids, idle deadlines, or teardown. +- The broker is the control plane (`/run/hhstream/control.sock`, newline JSON, mode `0660`). `grim` takes tens to hundreds of milliseconds and its timeout is 5 s. That block must not freeze `touch`, `kill`, or the reaper. That is why the pointer is already a separate process. +- A systemd template per session would be a second supervisor. Sessions are broker records, and `SessionManager.teardown` is deliberately the only kill path. A child unit also races `KillMode=` and adopt-on-restart. + +`hhstream-live` is an executor. It holds no session policy. The broker tells it what to publish over a private socket, `/run/hhstream/live.sock` (mode `0660`, next to `pointer.sock`). It must not take `RuntimeDirectory=/run/hhstream`; the pointer unit's comments record that a second owner deletes the broker's socket. + +It runs as the compositor user `hhstream`, with `video` and `render`, `ProtectHome` and `ProtectSystem=strict`, the same shape as `hyprland-headless-stream`. It discovers `WAYLAND_DISPLAY` the way that unit already does. It does not run as `hermes`: a network process should not have the agent's home. Group `hhstream` can already open the broker socket; this does not widen that. + +**One session's picture.** The broker sends the output name `hh-` and the full capture argv, built from the module's `captureCommand` plus `-o hh-`. On this host that is `grim -o hh- -c -t jpeg -q 80 -`. One capture thread and one frame slot per output, same drop rule as `FrameServer`: a slow viewer skips frames and is never queued. The daemon does not touch `stream.sock` and does not capture the main output. + +**No container, no stream.** If the session's container is missing or `degraded`, `stream start` returns the existing `no_container` error (exit 10). Falling back to the main output would put every other session on the phone. Containers are on in `/etc/nixos/hyprland-stream.nix` as a trial, and a window that ignores `HHSTREAM_APP_ID` still lands in quarantine. The stream will then honestly show an empty output. That is a container bug, not a reason to capture the main screen. + +**When the session dies.** `teardown` (kill, reap, and gc) tells the daemon to stop that output and close its clients **before** `hyprctl output remove`. If the output disappears anyway, `grim`'s "unknown output" ends the stream after a few failures. It does not retry forever the way the main publisher must. The port is freed when the listener closes. + +A broker restart does not rebind anything. If the daemon is still up, the broker asks it for the list and reattaches those listeners to the session records so status and teardown still see them. If the daemon restarted too, the stream is gone and status says so. An open port is an explicit act. + +### 3. Addressing and lifecycle + +**The caller may pass a port. Otherwise the broker takes the lowest free port in a configured range. Default range `9120–9131`.** Not a single configured port (two sessions must fit; `max_containers` is 2). Not an ephemeral port (a phone URL has to be writable, and a firewall cannot name port 0). `9120` starts just above **9119**, which `/etc/nixos/common.nix` already opens for the Hermes dashboard. Twelve ports covers the container budget plus a restart where the old socket is still in `TIME_WAIT`. + +``` +hhstream session stream start [--port N] [--bind ADDR] [--fps N] +hhstream session stream stop +hhstream session stream status [id] +``` + +Nested like `pointer`: one precondition (a container output) and one warning (this is a network listener). `start` prints **only the URL** on stdout, the same trick as the session id and the lease id: + +``` +http://127.0.0.1:9120// +``` + +Stderr says it is view-only, names the output, and prints the stop command. `--json` adds `url`, `port`, `bind`, `token`, `output`, `fps`, and `viewers`. When bind is `0.0.0.0`, the printed host is the default-route IPv4, overridable with `--advertise ADDR`, plus the loopback URL. Do not print `0.0.0.0`. + +`start` on a session that already has a listener returns that listener (and its token) when port and bind match, and `stream_busy` (exit 8) when they do not. `request_id` covers `start` and `stop` so a retry cannot open a second port. `stop` only stops the listener; the session keeps running. A port outside the range is exit 2. The range being full is `stream_budget`, exit 6. The daemon being down is `stream_unavailable`, exit 10. No new exit-code numbers. + +The session record gains a `stream` object (`port`, `bind`, `token`, `output`, `fps`, `pid` or daemon id). Registry schema bumps to 4; older records migrate to `stream: null` and are adopted, not refused. + +**Idle clock.** `start`, `stop`, and `status` do **not** touch. Same rule as `pointer`: setting something up is not "still wanted". A **connected viewer is activity**. About once a minute the broker asks the daemon for viewer counts and, if any are non-zero, stamps `touch`. That is the one new way the clock moves, and it is not a CLI verb. It has to be: the idle timeout is three hours, and a session the operator is watching from his phone must not be reaped out from under him. An open port with nobody connected does not stamp, and it does not run `grim`. The reaper then kills the session on the normal deadline, and teardown closes the listener. A forgotten URL is a bound socket, not a hot core and not a pinned session. + +### 4. Security + +This would be the first listener meant to be opened from another device. Everything else is a unix socket in a `0750` directory, mode `0660`, group `hhstream`, with no authentication. The host firewall allows inbound TCP **22 and 9119 only**. The pixels can be a logged-in browser. + +**Default bind is `127.0.0.1`. The firewall stays closed. A path token is always required, including on loopback.** + +Loopback is still weaker than the group-gated frame socket: any local user can open a TCP port. The token is what replaces the group check. It is a 128-bit `secrets.token_urlsafe` value in the path, because an `` will send a path and will not send a custom header: + +``` +GET // the page +GET //stream.mjpg the multipart stream +GET //frame.jpg one JPEG +``` + +Anything else is a 404 with an empty body. Wrong-token responses do not echo the token. The audit log records port, bind, and output, and does not record the token. The token lives in the session record under the broker state dir (`/var/lib/hermes/hhstream-sessions` on this machine). `stop` then `start` rotates it. + +LAN is opt-in twice, in the host config, not in the module defaults: + +- `sessions.stream.bind = "0.0.0.0"` (or a specific address). `--bind` on the CLI can do the same for one session. +- `sessions.stream.openFirewall`, default **false**. The module must not add ports to `networking.firewall.allowedTCPPorts` unless that option is on. This machine should turn it on in `/etc/nixos` only when the phone path is wanted, and the range must not include 9119. + +The token is cleartext on the LAN. The threat model is "another device on the LAN can watch if it has the URL", not "the URL is secret from the network". There is no TLS in this slice; a private CA and a phone trust prompt are a worse default. Until the firewall is opened, the phone path is an SSH forward to loopback (`ssh -L 9120:127.0.0.1:9120`), which keeps the current posture. + +The listener is view-only. No pointer, no keyboard, no `hyprctl`. At most four viewers. The broker pulls viewer counts; the daemon cannot ask the broker to touch an arbitrary session. + +### 5. Tests + +Same layers as the rest of the repo. No GPU for unit tests. No browser in the VM checks. VM checks stay one at a time. + +| Test | Proves | +|---|---| +| `tests/unit` for the publisher, with `FakeCapture` | Multipart framing, one JPEG per part, `Content-Type`, token 404s with an empty body, a stalled viewer does not block another, capture does not run at zero viewers, "unknown output" ends the stream. This mirrors `tests/unit/test_server.py`. | +| `tests/sessions/unit` | Port allocator (lowest free, range exhausted, clash), CLI parse, stdout is the bare URL, stderr carries the warning, `start`/`stop`/`status` do not touch, a viewer heartbeat does, teardown issues stop before the container is destroyed. New error names map to exits 2, 6, 8, and 10 without renumbering. | +| `tests/sessions/integration` | A real daemon and a real `hhstream-live` on `127.0.0.1` with an ephemeral port. `urllib` reads two JPEGs, a wrong token fails, `stream stop` closes the port, and a real idle reap closes it too. Fake capture, no compositor. | +| A human browser pass, not a check | On the host, `stream start` against a session whose window is actually on `hh-` (the cooperating client from `checks/vm-containers.nix`, not `foot` or Chromium). Open the printed URL in the Chromium this machine already installs. Change pixels; they must show up without a reload. The phone pass is the same URL only after bind and firewall are opted in. | + +`urllib` reading a multipart body is the automated proof that a browser's `` has something valid to decode. A headless-Chrome screenshot is optional and is not a VM check: two software-rendered compositors plus a browser will flake on two cores. + +### 6. Stages + +Contracts bump first. `docs/CONTRACT.md` stays as it is (`stream.sock` is untouched). `docs/CONTRACT-sessions.md` needs a sessions-v3 for the verbs, `/run/hhstream/live.sock`, the unit name, the viewer-touch exception, and the error names. Schema 4 is the record bump. No v1 or v2 field changes meaning. + +| Stage | What | Size | Risk | +|---|---|---|---| +| 0 | Contract text only. | S | Freezing the wrong default. This plan is the review. | +| 1 | `hhstream-live`: HTTP, token, one output, injected capture, unit tests. No broker. | M | A phone that will not paint `multipart/x-mixed-replace`. That is the gate for the WebSocket fallback, and it is not built here. | +| 2 | Broker verbs, registry field, CLI, teardown order, viewer heartbeat, integration test. | L | A second port on retry; stop racing `output remove`; token leaking into `events.jsonl`; a heartbeat that keeps a session alive with zero viewers. | +| 3 | Module: the unit, the socket, `portRange`, `fps` default 5 cap 10, `bind` default loopback, `openFirewall` default false. No host firewall edit unless you ask for the phone path separately. | M | A second `RuntimeDirectory` on `/run/hhstream` deleting `control.sock`. Copy the pointer unit's avoidance. | +| 4 | Browser pass on the host, written down with what was actually on screen. | S | An empty container looking like a broken stream. Confirm the window is on `hh-` first. | + +**Open questions** + +1. Viewer-connected stamps the idle clock; the verbs do not. Rejected alternative: the operator must `touch` while watching, which will reap a session at three hours mid-view. +2. Listeners are not re-created after a restart. They are reattached if `hhstream-live` is still holding them. +3. The phone does not work until `/etc/nixos` opts into `0.0.0.0` and the port range. Defaults stay loopback and firewalled. SSH `-L` works before that. +4. The 5 fps cap is from the main output's ~46 ms JPEG, not from a measured `grim -o` of a container. Raise it only from a measurement. +5. IPv6 is out of this slice. \ No newline at end of file diff --git a/docs/plans/2026-10-04-session-stream-webapp.md b/docs/plans/2026-10-04-session-stream-webapp.md new file mode 100644 index 0000000..d86516c --- /dev/null +++ b/docs/plans/2026-10-04-session-stream-webapp.md @@ -0,0 +1,175 @@ +# Plan — a session-stream webapp (one port, a router, a phrase per session) + +Supersedes `docs/plans/2026-10-03-network-stream.md` (call it v1). v1 spread a +listener over a **port range** and handed each session a **128-bit URL token**; +the operator asked for the opposite shape: **one port, a router with a page per +session, and a spoken phrase** to open each one, with a stream existing **only +for the sessions it was enabled on**. Everything v1 says about the *capture* +path and the *separate unit* still holds and is not repeated here; this file +replaces §2–§4 of v1 and keeps its §1 (transport) and §6 stages in spirit. + +Decisions below are the ones worth disagreeing with. **Nothing is built yet** — +the branch carries only a plan. + +--- + +## 1. What is new, and why + +| v1 | Now | Why | +| --- | --- | --- | +| Lowest free port in `9120–9131`, one per session | **One port, default 9988**, shared by every stream | A phrase is easier to read out when there is one address, not one per session. One listener is also one thing to firewall and one thing to stop. | +| `//…`, a `secrets.token_urlsafe` blob | **`//…`**, four words off a fixed list | A URL a person can say out loud and retype on a phone. Same job as the token — it is the only credential — but shaped for a human. Entropy is discussed in §4. | +| `stream start` on any live session | **`stream enable`** on a session, remembered, default **off** | "Not on for all sessions at all times": a session does not stream because it exists; it streams because it was switched on. | +| Pages are token routes under one session's listener | **A router**: an index, and **four routes per session** (viewer, stream, snapshot, status) | A small app, not a file dump: `/` lists what is on, `//` is the thing you watch. | + +The transport is unchanged: **MJPEG over HTTP** (`multipart/x-mixed-replace`), +the JPEG `grim` already produces (`-c -t jpeg -q 80 -`), 5 fps default, hard cap +10, capture **only while a viewer is connected**. A still browser `` is the +whole client. No WebSocket, no HLS, no TLS in this slice. + +## 2. The unit, and the port + +`hhstream-live.service`, **one process**, exactly as v1 argued: an executor +beside the pointer driver, not the frame server (whose one-socket contract must +not grow a LAN parser) and not the broker (`grim` blocks tens to hundreds of ms +and must not stall `touch`/`kill`). It runs as the compositor user `hhstream`, +finds `WAYLAND_DISPLAY` the way `hyprland-headless-stream` does, does **not** +take `RuntimeDirectory=/run/hhstream` (that would delete the broker's socket), +and speaks newline-JSON to the broker on `/run/hhstream/live.sock`. + +It owns **the port**. A single listener means no allocator, no `TIME_WAIT` +dance, and one URL prefix. The port is a module option (`sessions.stream.port`, +default **9988**); the unit binds `sessions.stream.bind` (default `127.0.0.1`). +If the port is taken the unit fails loudly and says so — it does not hunt for +another one, because a stream at a port nobody was told is worse than a stream +that is down. + +**Bind and firewall defaults stay closed.** `openFirewall` defaults false; the +module adds no `networking.firewall.allowedTCPPorts` unless it is set. Until +that opt-in, the phone path is `ssh -L 9988:127.0.0.1:9988` from the device. + +## 3. The router + +One handler, four shapes, phrase-keyed. Everything else is a `404` with an +empty body; a wrong phrase is never echoed back. + +``` +GET / index: the enabled streams, their names, viewer + counts, and links into them +GET // the viewer page — an + in a dark frame with the session's name +GET //stream.mjpg the multipart/x-mixed-replace stream +GET //frame.jpg the most recent single JPEG (snapshot) +GET //status.json {session, output, fps, viewers, last_frame_age_s} +``` + +- The phrase is matched **case-insensitively** and tolerates a missing trailing + slash, because a person will type it by hand; it is stored lowercase. +- **The root page is a form, never a list.** The phrase is the password to a + stream and is known only by whoever enabled it, so `/` must not enumerate + sessions or leak phrases: it is a box you type a phrase into, which forwards + to `//`. `//…` always works from whatever the bind allows. +- The pages are **built-in** — one small HTML/CSS/JS string in the package, no + CDN, no framework, no extra Python dependency. "Webapp" here means a real + router and legible pages, not a build step. +- Multiple pages **per session output**: a session's output `hh-` gets its + own four routes; adding a session adds routes under its phrase, never a new + port. +- Wrong-phrase lookups are **rate-limited** (a small per-IP counter, ~20/min) so + the phrase is not enumerable at line speed over the LAN. + +## 4. The phrase + +Generated when the stream is **first enabled** for a session, and kept in the +session record so it survives a broker restart and a re-enable. + +- Shape: `----`, e.g. + `amber-otter-lantern-harbor-7k2qp`. +- Words come from a **vendored EFF large wordlist** (7776 words, ~70 KB, in the + repo — no network at runtime, no dependency) and the tail is five characters + of `[a-z0-9]`. Four words is `4 · log2(7776) ≈ 51.7` bits and the tail adds + `5 · log2(36) ≈ 25.8`, for **≈ 77.5 bits** — comfortably past anything a LAN + request rate can enumerate, and the operator's stated shape ("5 with a random + 5 character string for the last word"). +- `--phrase` on enable sets your own (checked against the shape); `disable` + then `enable` **rotates** it. A rotation is one record write. +- The phrase is the only credential. It is in the URL, in cleartext on the LAN + when the LAN is opted into — the threat model is "another device on the LAN + can watch if it can guess the phrase", not "the phrase is hidden from the + network". Stated plainly so nobody mistakes it for TLS. +- Secrets discipline from v1 carries over: the audit log records session id, + output, port and bind; it **never** records the phrase. + +## 5. Enable / disable, and the idle clock + +Only an enabled session has a phrase, a listener entry, and any capture at all. +Enabling is an explicit act, remembered in the record: + +``` +hhstream session stream enable [--phrase PHRASE] [--fps N] +hhstream session stream disable +hhstream session stream list # enabled streams, phrases, viewers +hhstream session stream status [id] +``` + +- `enable` requires the session to have a container output (`hh-`); without + one it is the existing `no_container` error (exit 10). A stream never falls + back to the main output — that would put every other session on the phone. +- `enable` prints **only the URL** on stdout (`http://127.0.0.1:9988//`); + stderr names the output and prints the `disable` command. `--json` adds + `url`, `phrase`, `output`, `fps`, `viewers`. Re-enabling while enabled returns + the same phrase; `--phrase` mismatch is `stream_busy` (exit 8). +- No new exit numbers: reuse 2 (bad port/fps), 6 (`stream_budget` — the module's + per-process viewer cap), 8 (busy), 10 (`stream_unavailable`/no container). +- `enable`, `disable`, `list`, `status` do **not** touch the idle clock — the + same rule as `pointer.*`. A **connected viewer is activity**: about once a + minute the broker asks the daemon for viewer counts and stamps `touch` when + any is non-zero. That is the one new way the clock moves; it is what keeps a + session the operator is watching on his phone from being reaped at three + hours. An enabled stream with nobody watching runs **no** `grim` and keeps no + session alive. +- The session record gains `stream = {enabled, phrase, fps, viewers, port}` + (or `null`). **Registry schema 3 → 4**; older records migrate to + `stream: null` and are adopted, not refused. +- Teardown order is v1's and it is load-bearing: `disable` the stream (close the + listener, drop the route) **before** `hyprctl output remove`. + +## 6. Restart behaviour + +A broker restart does **not** invent listeners. If `hhstream-live` is still up, +the broker asks it for the list and re-attaches them to the session records, so +`status` and teardown still see them. If the daemon restarted too, the enabled +records keep their phrases but the streams are off until re-enabled — an open +port and a running capture are explicit acts, and the phrase is not a licence to +capture without one. + +## 7. What to build, in what order + +| Stage | What | Size | Risk | +| --- | --- | --- | --- | +| 0 | Contract text: `docs/CONTRACT-sessions.md` → **sessions-v3** (the four verbs, `/run/hhstream/live.sock`, `hhstream-live`, the single port, the viewer-touch exception, schema 4, error names). | S | Freezing the wrong default. This plan is the review. | +| 1 | `hhstream-live`: the HTTP server, the router, the phrase (wordlist + generator), one output, injected capture, **unit tests**. No broker. | M | A phone that will not paint `multipart/x-mixed-replace`; the index leaking phrases when made public. | +| 2 | Broker + CLI: the record field, schema 4, the four verbs, teardown order, viewer heartbeat, integration test on `127.0.0.1`. | L | A second capture on re-enable; `disable` racing `output remove`; a heartbeat that keeps a session alive with zero viewers. | +| 3 | Module: the unit, `/run/hhstream/live.sock`, `port` 9988, `bind` loopback, `fps` 5/cap 10, `enable` false, `openFirewall` false, `publicIndex` false. No host firewall edit here. | M | A second `RuntimeDirectory` on `/run/hhstream` deleting `control.sock`. | +| 4 | Host pass: enable a real session, open the URL in the machine's own Chromium, confirm pixels change without a reload; write down what was on screen. | S | An empty container looking like a broken stream. Confirm the window is on `hh-` first. | + +Test layers are v1's: `tests/unit` for the HTTP/router/phrase with a fake +capture (multipart framing, each route, wrong phrase → empty 404, rate limit, +capture idle at zero viewers, "unknown output" ends the stream), `tests/sessions` +for the verbs and the clock, one integration test that runs a real daemon and a +real `hhstream-live` on loopback with `urllib` reading two JPEGs. VM checks stay +one at a time; no browser in a VM check. + +## 8. Decisions taken (2026-10-04) + +1. **The root page is a form, never a list.** The phrase is a password; the + index must not hand it out. (Was "loopback-only index".) +2. **Phrase = four EFF-large words + a five-character base-36 tail (~77.5 + bits).** The tail is what the operator asked for in place of a fifth word. +3. **Port 9988**, fixed and single, no range fallback. +4. **Capture pauses at zero viewers** (v1's choice, kept). + +**Stage 1 is built** (`src/hhstream_live/`, `tests/live/`, 51 unit tests): the +router, the pages, the phrase and the publisher, runnable standalone via +`hhstream-live serve --stream PHRASE=OUTPUT`. The broker verbs and +`/run/hhstream/live.sock` are Stage 2. \ No newline at end of file diff --git a/nix/package.nix b/nix/package.nix index fb402ca..67bd480 100644 --- a/nix/package.nix +++ b/nix/package.nix @@ -28,6 +28,10 @@ python3.pkgs.buildPythonApplication { pythonImportsCheck = [ "hhstream_pointer.ops" "hhstream_pointer._protocol.wlr_virtual_pointer_unstable_v1" + # The web app: importing the server pulls in the router, the publisher and + # the pages. The phrase generator's wordlist is covered by the unit suite, + # which reads the *installed* package. + "hhstream_live.server" ]; meta = { diff --git a/pyproject.toml b/pyproject.toml index 0e8c0f9..59054b5 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -24,6 +24,16 @@ hhstreamd = "hhstreamd.__main__:main" # The pointer driver. Its own process because a Wayland client needs its own # connection, and a blocked one must not stall the broker. hhstream-pointer = "hhstream_pointer.__main__:main" +# The session-stream web app (one port, a page per enabled session). Its own +# process beside the pointer driver: a LAN-facing HTTP parser must not be able +# to stall session lifecycle, and `grim` blocking must not reach the broker. +hhstream-live = "hhstream_live.__main__:main" [tool.setuptools.packages.find] where = ["src"] + +# The phrase generator reads this at runtime, so it has to be *installed*, not +# just present in the checkout: a wheel (and the Nix build) that dropped it +# would make every `enable` fail with an empty wordlist. +[tool.setuptools.package-data] +hhstream_live = ["wordlist.txt"] diff --git a/src/hhstream_live/__init__.py b/src/hhstream_live/__init__.py new file mode 100644 index 0000000..3bfc3ba --- /dev/null +++ b/src/hhstream_live/__init__.py @@ -0,0 +1,15 @@ +"""The per-session live web stream: one port, a router, a phrase per session. + +Stage 1 of `docs/plans/2026-10-04-session-stream-webapp.md`: the web application +itself -- the HTTP router, the viewer/snapshot/status pages, the MJPEG +publisher, and the phrase generator. No broker wiring yet; the broker's verbs +and the `/run/hhstream/live.sock` control channel are a separate stage. + +The design in one line: a session's stream exists only once it is *enabled*, +its key is a spoken phrase, and every enabled session is a route under one +listener rather than a port of its own. +""" + +from __future__ import annotations + +__all__ = ["phrase", "publisher", "server", "pages"] diff --git a/src/hhstream_live/__main__.py b/src/hhstream_live/__main__.py new file mode 100644 index 0000000..b0b3dd9 --- /dev/null +++ b/src/hhstream_live/__main__.py @@ -0,0 +1,139 @@ +"""``hhstream-live`` -- run the session-stream web app. + +Stage 1 runs it **standalone**: the streams are named on the command line as +``--stream =``, which is enough to watch a real container and to +drive the unit tests. Stage 2 adds the broker's control socket +(``/run/hhstream/live.sock``), at which point ``--stream`` becomes the way a +human debugs it rather than the way the broker configures it. + + hhstream-live serve --stream four-words-and-a-tail=hh-demo + hhstream-live phrase + +The capture command is one list of words; the output is aimed by inserting +``-o `` before the trailing ``-``, which is grim's "write to stdout". +""" + +from __future__ import annotations + +import argparse +import logging +import sys + +from . import phrase as phrase_mod +from .publisher import DEFAULT_FPS, FPS_CAP, StreamRegistry, StreamSpec +from .server import DEFAULT_HOST, DEFAULT_PORT, LiveServer + +#: The session's own capture command, JPEG for the same reason the host config +#: spells it out: a 1280x720 JPEG q80 is ~46 ms and the point is a video, not a +#: slideshow. `-c` asks grim to include the cursor. +DEFAULT_CAPTURE = ["grim", "-c", "-t", "jpeg", "-q", "80", "-"] + + +def capture_argv(base: list[str], output: str) -> list[str]: + """Aim the base capture command at one output. + + ``grim`` writes to stdout when its last argument is ``-``; the output + selector goes just before it so the shape holds for any tool that keeps that + convention, and a base without a trailing ``-`` simply gets ``-o`` appended. + """ + base = list(base) + if base and base[-1] == "-": + return base[:-1] + ["-o", output, "-"] + return base + ["-o", output] + + +def parse_stream(arg: str) -> tuple[str, str]: + """``=`` -> the pair, each side checked non-empty.""" + phrase, sep, output = arg.partition("=") + if not sep or not phrase.strip() or not output.strip(): + raise argparse.ArgumentTypeError(f"expected PHRASE=OUTPUT, got {arg!r}") + try: + return phrase_mod.validate(phrase), output.strip() + except ValueError as exc: + raise argparse.ArgumentTypeError(str(exc)) from exc + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser(prog="hhstream-live", description=__doc__) + parser.add_argument("--log-level", default="info", + choices=["debug", "info", "warning", "error"]) + subs = parser.add_subparsers(dest="command") + + serve = subs.add_parser("serve", help="run the web app (default)") + serve.add_argument("--port", type=int, default=DEFAULT_PORT, + help=f"the one port every stream shares (default {DEFAULT_PORT})") + serve.add_argument("--bind", default=DEFAULT_HOST, + help=f"address to bind (default {DEFAULT_HOST}, loopback)") + serve.add_argument("--fps", type=int, default=DEFAULT_FPS, + help=f"capture rate for streams that do not set one " + f"(default {DEFAULT_FPS}, cap {FPS_CAP})") + serve.add_argument("--stream", action="append", default=[], type=parse_stream, + metavar="PHRASE=OUTPUT", + help="enable a stream on this output; repeatable") + serve.add_argument("--capture-command", action="append", default=None, + metavar="WORD", + help="capture argv word; repeat (a repeated flag is the " + "only shape that survives words starting with '-')") + + gen = subs.add_parser("phrase", help="print a fresh phrase and exit") + gen.add_argument("--count", type=int, default=1) + + return parser + + +def make_registry(streams: list[tuple[str, str]], capture_base: list[str], + fps: int) -> StreamRegistry: + from hhstream.capture import CommandCapture + + registry = StreamRegistry() + for phrase, output in streams: + spec = StreamSpec(phrase=phrase, output=output, fps=fps, label=output) + registry.add(spec, CommandCapture(capture_argv(capture_base, output))) + return registry + + +def cmd_serve(args: argparse.Namespace) -> int: + capture_base = args.capture_command or DEFAULT_CAPTURE + registry = make_registry(args.stream, capture_base, args.fps) + server = LiveServer(registry, host=args.bind, port=args.port) + + print(f"landing page: http://{server.address}:{args.port}/", file=sys.stderr) + if not args.stream: + print("no --stream given; only the landing page will answer", file=sys.stderr) + server.start() # raises OSError loudly if the port is taken + for phrase in registry.phrases(): + print(server.url_for(phrase)) + sys.stdout.flush() + + try: + import time + + while True: + time.sleep(3600) + except KeyboardInterrupt: + print("stopping", file=sys.stderr) + finally: + server.stop() + return 0 + + +def main(argv: list[str] | None = None) -> int: + parser = build_parser() + args = parser.parse_args(argv) + logging.basicConfig( + level=getattr(logging, (args.log_level or "info").upper()), + format="%(asctime)s %(levelname)s %(name)s: %(message)s", + ) + command = args.command or "serve" + if command == "phrase": + for _ in range(args.count): + print(phrase_mod.generate()) + return 0 + if command == "serve": + return cmd_serve(args) + parser.error(f"unknown command {command!r}") # pragma: no cover + return 2 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/src/hhstream_live/pages.py b/src/hhstream_live/pages.py new file mode 100644 index 0000000..e95cbb7 --- /dev/null +++ b/src/hhstream_live/pages.py @@ -0,0 +1,115 @@ +"""The pages the router serves: the landing form and the per-session viewer. + +Everything here is one self-contained HTML string. No CDN, no framework, no +build step -- "webapp" means a router and legible pages, not a toolchain. The +viewer page is the one that matters: it is an ```` pointed at the MJPEG +route, because a top-level ``multipart/x-mixed-replace`` document is unreliable +on phones while an image tag is not. + +The landing page is a **form, never a list**. The phrase is the password to a +stream and is known only by whoever enabled it, so the root page must not +enumerate sessions or leak phrases; it just hands the typed phrase to +``//``. +""" + +from __future__ import annotations + +import html +from functools import lru_cache + +_STYLE = """ +:root{color-scheme:dark} +*{box-sizing:border-box} +body{margin:0;background:#0b0d12;color:#e6e8ee;font:15px/1.5 ui-sans-serif,system-ui,sans-serif} +main{max-width:960px;margin:0 auto;padding:24px} +h1{font-size:20px;margin:0 0 4px} +a{color:#7cc4ff} +.muted{color:#8b93a7} +.card{background:#141821;border:1px solid #232838;border-radius:12px;padding:20px} +img{width:100%;height:auto;display:block;border-radius:8px;background:#000} +input{width:100%;padding:12px 14px;margin:10px 0 14px;border-radius:8px; + border:1px solid #2c3346;background:#0e121b;color:inherit;font:inherit} +button{padding:11px 16px;border:0;border-radius:8px;background:#3b82f6;color:#fff; + font:inherit;cursor:pointer} +button:hover{background:#2f6fd6} +.badge{display:inline-flex;align-items:center;gap:6px;font-size:13px;color:#8b93a7} +.dot{width:8px;height:8px;border-radius:50%;background:#3ddc84;box-shadow:0 0 8px #3ddc84} +.row{display:flex;align-items:center;justify-content:space-between;gap:12px;margin-bottom:12px} +.tools{display:flex;gap:14px;font-size:13px;margin-top:12px} +""" + + +def _esc(text: str) -> str: + return html.escape(text, quote=True) + + +@lru_cache(maxsize=1) +def landing_page(app_name: str = "hhstream live") -> str: + """The root page: a box to type a phrase into. No session list, ever.""" + return f""" + + +{_esc(app_name)} +
+

{_esc(app_name)}

+

A live view is opened with the phrase printed when its session +was enabled.

+
+ + + +
+ +
+""" + + +def viewer_page(phrase: str, label: str, output: str, fps: int) -> str: + """The page you watch: an `` on the stream, plus a small live readout.""" + base = f"/{phrase}" + return f""" + + +{_esc(label)} — live +
+
+
+

{_esc(label)}

+
{_esc(output)} · {fps} fps
+
+ live +
+
+ live view of {_esc(label)} +
+ + +
+""" + + +__all__ = ["landing_page", "viewer_page"] diff --git a/src/hhstream_live/phrase.py b/src/hhstream_live/phrase.py new file mode 100644 index 0000000..f5a7c03 --- /dev/null +++ b/src/hhstream_live/phrase.py @@ -0,0 +1,102 @@ +"""The phrase: a stream's only credential, shaped to be said out loud. + +A phrase is ``word-word-word-word-xxxxx``: four words off the EFF large +wordlist and a five-character base-36 tail. That is ``4*log2(7776) + +5*log2(36)`` ~= **77.5 bits** of entropy, against a word set small enough to be +a file in the repository and a tail short enough to type on a phone. + +Why words and not the ``secrets.token_urlsafe`` blob the first plan proposed: +the phrase is read out -- "open amber-otter-lantern-harbor-7k2qp" -- and a +base64 string is not. The job is identical (it *is* the password to the +stream); only the shape is chosen for a human. + +The list is vendored (`wordlist.txt`, 7776 lines) so a stream never needs the +network to start. `generate` uses `SystemRandom`/`secrets`, so the phrase is +not reproducible from the process seed. +""" + +from __future__ import annotations + +import math +import random +import re +import secrets +from pathlib import Path + +#: Four words plus the tail. The plan's answer to "how many"; four is +#: `4 * log2(7776)` ~= 51.7 bits on its own, and the tail pushes it past 77. +WORD_COUNT = 4 +SUFFIX_LEN = 5 +#: Lowercase letters and digits: what a person can retype without a shift key, +#: and 36 symbols is still `5 * log2(36)` ~= 25.8 bits. +SUFFIX_ALPHABET = "abcdefghijklmnopqrstuvwxyz0123456789" + +#: A phrase is one or more hyphen-joined tokens of lowercase letters/digits, +#: at least three of them (so a caller-supplied phrase cannot be a single word +#: or a passcode-free stub). Generated phrases are five tokens; this is the +#: looser shape a `--phrase` must satisfy. +_SHAPE = re.compile(r"^[a-z0-9]+(?:-[a-z0-9]+){2,}$") +MAX_LENGTH = 96 + +_WORDLIST = Path(__file__).with_name("wordlist.txt") +_cache: tuple[str, ...] | None = None + + +def words() -> tuple[str, ...]: + """The wordlist, read once and remembered. + + Raises `RuntimeError` on an empty file rather than returning nothing: a + generator that picks from an empty list would raise a confusing `IndexError` + later, and the empty file is a packaging bug worth naming here. + """ + global _cache + if _cache is None: + text = _WORDLIST.read_text(encoding="utf-8") + loaded = tuple(word.strip() for word in text.splitlines() if word.strip()) + if not loaded: + raise RuntimeError(f"wordlist {_WORDLIST} is empty") + _cache = loaded + return _cache + + +def entropy_bits(word_count: int = WORD_COUNT, suffix_len: int = SUFFIX_LEN) -> float: + """How many bits a phrase of this shape carries, for the record.""" + return word_count * math.log2(len(words())) + suffix_len * math.log2(len(SUFFIX_ALPHABET)) + + +def generate(*, word_count: int = WORD_COUNT, suffix_len: int = SUFFIX_LEN) -> str: + """A fresh phrase, drawn from the system entropy pool.""" + if word_count < 1 or suffix_len < 0: + raise ValueError("word_count must be >= 1 and suffix_len >= 0") + rng = random.SystemRandom() + chosen = [rng.choice(words()) for _ in range(word_count)] + tail = "".join(secrets.choice(SUFFIX_ALPHABET) for _ in range(suffix_len)) + return "-".join([*chosen, tail]) if tail else "-".join(chosen) + + +def normalize(phrase: str) -> str: + """Fold a phrase to the one form the registry and the router compare on. + + Lowercase and trimmed, because a phrase is typed by hand; the router also + accepts it without a trailing slash. Nothing else is folded -- a phrase with + an internal space is simply invalid, not silently rewritten. + """ + return phrase.strip().lower() + + +def validate(phrase: str) -> str: + """Return the normalized phrase, or raise `ValueError` naming the shape.""" + normalized = normalize(phrase) + if len(normalized) > MAX_LENGTH: + raise ValueError(f"phrase is {len(normalized)} characters; the maximum is {MAX_LENGTH}") + if not _SHAPE.match(normalized): + raise ValueError( + "phrase must be lowercase words/digits joined by hyphens, at least " + "three tokens (e.g. four-words-and-a-tail), not " + f"{phrase!r}" + ) + return normalized + + +__all__ = ["WORD_COUNT", "SUFFIX_LEN", "generate", "normalize", "validate", + "entropy_bits", "words", "MAX_LENGTH"] diff --git a/src/hhstream_live/publisher.py b/src/hhstream_live/publisher.py new file mode 100644 index 0000000..081c2e2 --- /dev/null +++ b/src/hhstream_live/publisher.py @@ -0,0 +1,264 @@ +"""The publisher: one capture thread and one frame slot per enabled session. + +This is the same shape as the frame server's inner loop (`hhstream/server.py`), +scaled from one output to many and given two rules the network side needs: + +* **one slot, no queue.** A viewer reads the most recent frame when it is ready + for another one; a slow viewer misses frames instead of forcing a backlog, so + memory does not grow with viewers and one stalled phone cannot delay another. +* **no capture unless someone is watching.** The capture thread sleeps while a + stream has zero viewers. That is what lets a stream be *enabled* without a + `grim` running: the port is open, the route exists, and the CPU is idle until + a request arrives. It is also why the broker can safely treat a connected + viewer as activity and an enabled-but-unwatched stream as nothing at all. + +A `Stream` never touches the broker or the HTTP layer; it is a capture loop with +a frame slot and a viewer count, and the server hands it viewers. +""" + +from __future__ import annotations + +import logging +import threading +import time +from dataclasses import dataclass +from typing import Any, Callable, Protocol + +log = logging.getLogger("hhstream.live") + +#: The plan's cap. 5 fps default is "one frame per ~200 ms plus a ~46 ms grim", +#: chosen from the measured JPEG cost, not a guess; 10 is the ceiling because +#: the box is also running the compositor and the main publisher. +DEFAULT_FPS = 5 +FPS_CAP = 10 +#: After this many capture failures with no successful frame, a stream gives up +#: and closes its viewers. `grim` against a removed output ("unknown output") +#: fails every time, and retrying forever would be a busy loop on a dead session. +FAILURE_LIMIT = 10 + + +class Capture(Protocol): + def grab(self) -> bytes: # pragma: no cover - structural + ... + + +@dataclass(frozen=True) +class StreamSpec: + """What to publish, and under which phrase. + + `session` is the broker's session id (``None`` in a standalone run); `label` + is what the viewer page shows. `output` is the compositor output the capture + is aimed at (``hh-``), kept here so the page and the status + payload can name it without re-deriving it. + """ + + phrase: str + output: str + fps: int = DEFAULT_FPS + session: str | None = None + label: str = "" + + def public(self) -> dict[str, Any]: + return { + "phrase": self.phrase, + "output": self.output, + "fps": self.fps, + "session": self.session, + "label": self.label or self.session or self.output, + } + + +class Stream: + """One output's capture loop, frame slot and viewer count.""" + + def __init__(self, spec: StreamSpec, capture: Capture, + clock: Callable[[], float] = time.monotonic) -> None: + if isinstance(spec.fps, bool) or not isinstance(spec.fps, int) or spec.fps <= 0: + raise ValueError(f"fps must be a positive int, got {spec.fps!r}") + if spec.fps > FPS_CAP: + raise ValueError(f"fps {spec.fps} exceeds the cap of {FPS_CAP}") + self.spec = spec + self.capture = capture + self._clock = clock + + self._cond = threading.Condition() + self._frame: bytes | None = None + self._seq = 0 + self._viewers = 0 + self._captured = 0 + self._failures = 0 + self._consecutive_failures = 0 + self._last_frame_at: float | None = None + self._stopped = False + self._thread: threading.Thread | None = None + + # ---- lifecycle --------------------------------------------------- + + def start(self) -> None: + self._thread = threading.Thread( + target=self._run, name=f"hhstream-live-{self.spec.phrase}", daemon=True + ) + self._thread.start() + + def stop(self) -> None: + with self._cond: + self._stopped = True + self._cond.notify_all() + if self._thread is not None: + self._thread.join(timeout=5) + + def _sleep(self, seconds: float) -> bool: + """Wait `seconds`, or until stopped. True means 'stop now'.""" + with self._cond: + return self._cond.wait_for(lambda: self._stopped, timeout=seconds) + + def _run(self) -> None: + interval = 1.0 / self.spec.fps + while True: + # Idle at zero viewers: this is the "not on for all sessions at all + # times" rule made concrete -- an enabled stream costs nothing until + # a request arrives. + with self._cond: + while not self._stopped and self._viewers == 0: + self._cond.wait(0.5) + if self._stopped: + return + try: + frame = self.capture.grab() + except Exception as exc: # noqa: BLE001 - any backend failure is a skip + with self._cond: + self._failures += 1 + self._consecutive_failures += 1 + give_up = self._consecutive_failures >= FAILURE_LIMIT and self._captured == 0 + if give_up: + self._stopped = True + self._cond.notify_all() + if give_up: + log.warning("stream %s: giving up after %d capture failures: %s", + self.spec.phrase, FAILURE_LIMIT, exc) + return + log.debug("stream %s: capture failed: %s", self.spec.phrase, exc) + if self._sleep(0.5): + return + continue + with self._cond: + self._frame = frame + self._seq += 1 + self._captured += 1 + self._consecutive_failures = 0 + self._last_frame_at = self._clock() + self._cond.notify_all() + if self._sleep(interval): + return + + # ---- viewers ----------------------------------------------------- + + def add_viewer(self) -> int: + with self._cond: + self._viewers += 1 + self._cond.notify_all() + return self._viewers + + def remove_viewer(self) -> int: + with self._cond: + self._viewers = max(0, self._viewers - 1) + self._cond.notify_all() + return self._viewers + + @property + def viewers(self) -> int: + with self._cond: + return self._viewers + + def wait_frame(self, last_seq: int, timeout: float = 2.0) -> tuple[bytes | None, int]: + """Block for a frame newer than `last_seq`. + + Returns `(frame, seq)`; `(None, last_seq)` on timeout or after the + stream stopped with no new frame, which tells a viewer loop to re-check + whether it should keep going. + """ + with self._cond: + self._cond.wait_for( + lambda: self._seq > last_seq or self._stopped, timeout=timeout + ) + if self._seq > last_seq: + return self._frame, self._seq + return None, last_seq + + @property + def stopped(self) -> bool: + with self._cond: + return self._stopped + + def snapshot(self) -> bytes | None: + with self._cond: + return self._frame + + # ---- reporting --------------------------------------------------- + + def status(self) -> dict[str, Any]: + with self._cond: + age = None if self._last_frame_at is None else max(0.0, self._clock() - self._last_frame_at) + return { + "session": self.spec.session, + "label": self.spec.label or self.spec.session or self.spec.output, + "output": self.spec.output, + "fps": self.spec.fps, + "viewers": self._viewers, + "captured": self._captured, + "failures": self._failures, + "last_frame_age_s": age, + "stopped": self._stopped, + } + + +class StreamRegistry: + """The router's table: phrase -> `Stream`, compared case-insensitively. + + A `Stream` is added already started; removing it stops the capture thread. + This is the whole of what "enabled" means on the publisher side -- there is + no disabled stream held in reserve, so an unknown phrase and a never-enabled + session are the same 404. + """ + + def __init__(self) -> None: + self._streams: dict[str, Stream] = {} + + def add(self, spec: StreamSpec, capture: Capture, *, start: bool = True) -> Stream: + from . import phrase as phrase_mod + + key = phrase_mod.validate(spec.phrase) + if key in self._streams: + raise ValueError(f"phrase {key!r} is already enabled") + stream = Stream(spec, capture) + self._streams[key] = stream + if start: + stream.start() + return stream + + def remove(self, phrase: str) -> Stream | None: + from . import phrase as phrase_mod + + stream = self._streams.pop(phrase_mod.normalize(phrase), None) + if stream is not None: + stream.stop() + return stream + + def get(self, phrase: str) -> Stream | None: + from . import phrase as phrase_mod + + return self._streams.get(phrase_mod.normalize(phrase)) + + def phrases(self) -> list[str]: + return sorted(self._streams) + + def all(self) -> list[Stream]: + return [self._streams[key] for key in sorted(self._streams)] + + def stop_all(self) -> None: + for stream in self.all(): + stream.stop() + + +__all__ = ["StreamSpec", "Stream", "StreamRegistry", "Capture", "DEFAULT_FPS", + "FPS_CAP", "FAILURE_LIMIT"] diff --git a/src/hhstream_live/server.py b/src/hhstream_live/server.py new file mode 100644 index 0000000..bfff9c5 --- /dev/null +++ b/src/hhstream_live/server.py @@ -0,0 +1,280 @@ +"""The web app: one HTTP listener, a router, a page per enabled session. + +One port (default 9988) serves every stream. The routes are keyed by *phrase*, +which is the only credential: + + GET / the landing form (never a list of sessions) + GET // the viewer page + GET //stream.mjpg multipart/x-mixed-replace, JPEG parts + GET //frame.jpg the most recent single frame + GET //status.json {session, output, fps, viewers, ...} + +Anything else is `404` with an **empty** body, and a wrong phrase is never +echoed back. Unknown phrases are rate-limited per client so the phrase is not +enumerable at line speed. + +Deliberately the standard library: `http.server` plus the framing rules this +module writes itself. A phone's `` is the only client that must work, and +that needs `multipart/x-mixed-replace` with `Content-Type: image/jpeg` parts -- +nothing a dependency would make easier. +""" + +from __future__ import annotations + +import logging +import threading +import time +from collections import deque +from http import HTTPStatus +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +from urllib.parse import unquote, urlsplit + +from . import pages +from .publisher import Stream, StreamRegistry + +log = logging.getLogger("hhstream.live") + +#: The multipart separator. Fixed and simple; it never appears in a JPEG. +BOUNDARY = "hhstreamboundary" +#: How long a viewer loop waits for a new frame before re-checking liveness. +FRAME_WAIT_S = 2.0 +DEFAULT_PORT = 9988 +DEFAULT_HOST = "127.0.0.1" +#: Wrong-phrase lookups allowed per client per window before 429. +RATE_LIMIT = 30 +RATE_WINDOW_S = 60.0 + + +class RateLimiter: + """A small per-client counter for failed phrase lookups. + + A phrase carries ~77 bits, so guessing is already hopeless; this is here so + that a LAN attacker cannot even *try* at line speed, and so the attempt is + visible in one place. Only misses are counted. + """ + + def __init__(self, limit: int = RATE_LIMIT, window: float = RATE_WINDOW_S, + clock=time.monotonic) -> None: + self.limit = limit + self.window = window + self._clock = clock + self._hits: dict[str, deque[float]] = {} + self._lock = threading.Lock() + + def allow(self, key: str) -> bool: + now = self._clock() + with self._lock: + hits = self._hits.setdefault(key, deque()) + cutoff = now - self.window + while hits and hits[0] < cutoff: + hits.popleft() + if len(hits) >= self.limit: + return False + hits.append(now) + return True + + def reset(self) -> None: + with self._lock: + self._hits.clear() + + +class _Server(ThreadingHTTPServer): + daemon_threads = True + allow_reuse_address = True + + def __init__(self, address, handler, registry: StreamRegistry, + limiter: RateLimiter) -> None: + self.registry = registry + self.limiter = limiter + super().__init__(address, handler) + + +class _Handler(BaseHTTPRequestHandler): + protocol_version = "HTTP/1.1" + server_version = "hhstream-live" + sys_version = "" + + # ---- plumbing ---------------------------------------------------- + + @property + def registry(self) -> StreamRegistry: + return self.server.registry # type: ignore[attr-defined] + + def log_message(self, format: str, *args) -> None: # noqa: A002 - base signature + log.debug("%s %s", self.address_string(), format % args) + + def _send(self, status: HTTPStatus, body: bytes, content_type: str, + headers: dict[str, str] | None = None) -> None: + self.send_response(status) + self.send_header("Content-Type", content_type) + self.send_header("Content-Length", str(len(body))) + self.send_header("Cache-Control", "no-store") + self.send_header("X-Content-Type-Options", "nosniff") + for name, value in (headers or {}).items(): + self.send_header(name, value) + self.end_headers() + if body: + self.wfile.write(body) + + def _empty(self, status: HTTPStatus) -> None: + self._send(status, b"", "text/plain; charset=utf-8") + + # ---- routes ------------------------------------------------------ + + def do_GET(self) -> None: # noqa: N802 - the name http.server dispatches to + path = unquote(urlsplit(self.path).path) + try: + self._route(path) + except (BrokenPipeError, ConnectionResetError): + pass # the viewer went away mid-write; that is not an error + except OSError as exc: + log.debug("client error on %s: %s", path, exc) + + def _route(self, path: str) -> None: + if path in ("", "/"): + body = pages.landing_page().encode("utf-8") + self._send(HTTPStatus.OK, body, "text/html; charset=utf-8") + return + if path == "/healthz": + self._send(HTTPStatus.OK, b"ok\n", "text/plain; charset=utf-8") + return + if not path.startswith("/"): + self._empty(HTTPStatus.NOT_FOUND) + return + + head, _, rest = path.lstrip("/").partition("/") + stream = self.registry.get(head) + if stream is None: + # A miss. `head` is user input and is never reflected anywhere. + if not self.server.limiter.allow(self.client_address[0]): # type: ignore[attr-defined] + self._empty(HTTPStatus.TOO_MANY_REQUESTS) + else: + self._empty(HTTPStatus.NOT_FOUND) + return + + if rest in ("", "index.html"): + self._viewer(stream) + elif rest == "stream.mjpg": + self._mjpeg(stream) + elif rest == "frame.jpg": + self._snapshot(stream) + elif rest == "status.json": + self._status(stream) + else: + self._empty(HTTPStatus.NOT_FOUND) + + # ---- handlers ---------------------------------------------------- + + def _viewer(self, stream: Stream) -> None: + spec = stream.spec + label = spec.label or spec.session or spec.output + body = pages.viewer_page(spec.phrase, label, spec.output, spec.fps).encode("utf-8") + self._send(HTTPStatus.OK, body, "text/html; charset=utf-8") + + def _status(self, stream: Stream) -> None: + import json + + body = json.dumps(stream.status()).encode("utf-8") + self._send(HTTPStatus.OK, body, "application/json; charset=utf-8") + + def _snapshot(self, stream: Stream) -> None: + frame = stream.snapshot() + if frame is None: + # No frame yet: nobody has watched this stream, so nothing has been + # captured. 503 is the honest answer, not a blank image. + self._empty(HTTPStatus.SERVICE_UNAVAILABLE) + return + self._send(HTTPStatus.OK, frame, "image/jpeg") + + def _mjpeg(self, stream: Stream) -> None: + self.send_response(HTTPStatus.OK) + self.send_header("Content-Type", f"multipart/x-mixed-replace; boundary={BOUNDARY}") + self.send_header("Cache-Control", "no-store") + self.send_header("X-Content-Type-Options", "nosniff") + self.end_headers() + + stream.add_viewer() + # The connection is the stream: after this handler returns there is no + # next request to read, so it must not be kept alive. + self.close_connection = True + last_seq = 0 + try: + while True: + frame, seq = stream.wait_frame(last_seq, timeout=FRAME_WAIT_S) + if frame is None: + if stream.stopped: + return + continue # nothing new yet; loop and re-check liveness + last_seq = seq + self.wfile.write(self._part(frame)) + # unreachable; the loop exits by return/exception + finally: + stream.remove_viewer() + + @staticmethod + def _part(frame: bytes) -> bytes: + header = ( + f"--{BOUNDARY}\r\n" + "Content-Type: image/jpeg\r\n" + f"Content-Length: {len(frame)}\r\n" + "\r\n" + ).encode("ascii") + return header + frame + b"\r\n" + + +class LiveServer: + """Owns the listener and its lifetime. One instance, many streams.""" + + def __init__(self, registry: StreamRegistry, *, host: str = DEFAULT_HOST, + port: int = DEFAULT_PORT, limiter: RateLimiter | None = None) -> None: + self.registry = registry + self.host = host + self.requested_port = port + self.limiter = limiter or RateLimiter() + self._httpd: _Server | None = None + self._thread: threading.Thread | None = None + self.ready = threading.Event() + + @property + def port(self) -> int: + """The bound port (resolves a requested port of 0 to the real one).""" + if self._httpd is None: + return self.requested_port + return self._httpd.server_address[1] + + @property + def address(self) -> str: + advertised = self.host + if advertised in ("0.0.0.0", "::"): + advertised = "127.0.0.1" + return advertised + + def url_for(self, phrase: str) -> str: + return f"http://{self.address}:{self.port}/{phrase}/" + + def start(self) -> None: + """Bind and serve. Raises `OSError` if the port is taken -- loudly. + + Deliberately no hunting for a free port: a stream at a port nobody was + told is worse than a stream that refuses to start. + """ + self._httpd = _Server((self.host, self.requested_port), _Handler, + self.registry, self.limiter) + self._thread = threading.Thread(target=self._httpd.serve_forever, + name="hhstream-live-http", daemon=True) + self._thread.start() + self.ready.set() + log.info("listening on http://%s:%d", self.host, self.port) + + def stop(self) -> None: + if self._httpd is not None: + self._httpd.shutdown() + self._httpd.server_close() + self._httpd = None + if self._thread is not None: + self._thread.join(timeout=5) + self._thread = None + self.registry.stop_all() + + +__all__ = ["LiveServer", "RateLimiter", "DEFAULT_PORT", "DEFAULT_HOST", "BOUNDARY"] diff --git a/src/hhstream_live/wordlist.txt b/src/hhstream_live/wordlist.txt new file mode 100644 index 0000000..caf71f5 --- /dev/null +++ b/src/hhstream_live/wordlist.txt @@ -0,0 +1,7776 @@ +abacus +abdomen +abdominal +abide +abiding +ability +ablaze +able +abnormal +abrasion +abrasive +abreast +abridge +abroad +abruptly +absence +absentee +absently +absinthe +absolute +absolve +abstain +abstract +absurd +accent +acclaim +acclimate +accompany +account +accuracy +accurate +accustom +acetone +achiness +aching +acid +acorn +acquaint +acquire +acre +acrobat +acronym +acting +action +activate +activator +active +activism +activist +activity +actress +acts +acutely +acuteness +aeration +aerobics +aerosol +aerospace +afar +affair +affected +affecting +affection +affidavit +affiliate +affirm +affix +afflicted +affluent +afford +affront +aflame +afloat +aflutter +afoot +afraid +afterglow +afterlife +aftermath +aftermost +afternoon +aged +ageless +agency +agenda +agent +aggregate +aghast +agile +agility +aging +agnostic +agonize +agonizing +agony +agreeable +agreeably +agreed +agreeing +agreement +aground +ahead +ahoy +aide +aids +aim +ajar +alabaster +alarm +albatross +album +alfalfa +algebra +algorithm +alias +alibi +alienable +alienate +aliens +alike +alive +alkaline +alkalize +almanac +almighty +almost +aloe +aloft +aloha +alone +alongside +aloof +alphabet +alright +although +altitude +alto +aluminum +alumni +always +amaretto +amaze +amazingly +amber +ambiance +ambiguity +ambiguous +ambition +ambitious +ambulance +ambush +amendable +amendment +amends +amenity +amiable +amicably +amid +amigo +amino +amiss +ammonia +ammonium +amnesty +amniotic +among +amount +amperage +ample +amplifier +amplify +amply +amuck +amulet +amusable +amused +amusement +amuser +amusing +anaconda +anaerobic +anagram +anatomist +anatomy +anchor +anchovy +ancient +android +anemia +anemic +aneurism +anew +angelfish +angelic +anger +angled +angler +angles +angling +angrily +angriness +anguished +angular +animal +animate +animating +animation +animator +anime +animosity +ankle +annex +annotate +announcer +annoying +annually +annuity +anointer +another +answering +antacid +antarctic +anteater +antelope +antennae +anthem +anthill +anthology +antibody +antics +antidote +antihero +antiquely +antiques +antiquity +antirust +antitoxic +antitrust +antiviral +antivirus +antler +antonym +antsy +anvil +anybody +anyhow +anymore +anyone +anyplace +anything +anytime +anyway +anywhere +aorta +apache +apostle +appealing +appear +appease +appeasing +appendage +appendix +appetite +appetizer +applaud +applause +apple +appliance +applicant +applied +apply +appointee +appraisal +appraiser +apprehend +approach +approval +approve +apricot +april +apron +aptitude +aptly +aqua +aqueduct +arbitrary +arbitrate +ardently +area +arena +arguable +arguably +argue +arise +armadillo +armband +armchair +armed +armful +armhole +arming +armless +armoire +armored +armory +armrest +army +aroma +arose +around +arousal +arrange +array +arrest +arrival +arrive +arrogance +arrogant +arson +art +ascend +ascension +ascent +ascertain +ashamed +ashen +ashes +ashy +aside +askew +asleep +asparagus +aspect +aspirate +aspire +aspirin +astonish +astound +astride +astrology +astronaut +astronomy +astute +atlantic +atlas +atom +atonable +atop +atrium +atrocious +atrophy +attach +attain +attempt +attendant +attendee +attention +attentive +attest +attic +attire +attitude +attractor +attribute +atypical +auction +audacious +audacity +audible +audibly +audience +audio +audition +augmented +august +authentic +author +autism +autistic +autograph +automaker +automated +automatic +autopilot +available +avalanche +avatar +avenge +avenging +avenue +average +aversion +avert +aviation +aviator +avid +avoid +await +awaken +award +aware +awhile +awkward +awning +awoke +awry +axis +babble +babbling +babied +baboon +backache +backboard +backboned +backdrop +backed +backer +backfield +backfire +backhand +backing +backlands +backlash +backless +backlight +backlit +backlog +backpack +backpedal +backrest +backroom +backshift +backside +backslid +backspace +backspin +backstab +backstage +backtalk +backtrack +backup +backward +backwash +backwater +backyard +bacon +bacteria +bacterium +badass +badge +badland +badly +badness +baffle +baffling +bagel +bagful +baggage +bagged +baggie +bagginess +bagging +baggy +bagpipe +baguette +baked +bakery +bakeshop +baking +balance +balancing +balcony +balmy +balsamic +bamboo +banana +banish +banister +banjo +bankable +bankbook +banked +banker +banking +banknote +bankroll +banner +bannister +banshee +banter +barbecue +barbed +barbell +barber +barcode +barge +bargraph +barista +baritone +barley +barmaid +barman +barn +barometer +barrack +barracuda +barrel +barrette +barricade +barrier +barstool +bartender +barterer +bash +basically +basics +basil +basin +basis +basket +batboy +batch +bath +baton +bats +battalion +battered +battering +battery +batting +battle +bauble +bazooka +blabber +bladder +blade +blah +blame +blaming +blanching +blandness +blank +blaspheme +blasphemy +blast +blatancy +blatantly +blazer +blazing +bleach +bleak +bleep +blemish +blend +bless +blighted +blimp +bling +blinked +blinker +blinking +blinks +blip +blissful +blitz +blizzard +bloated +bloating +blob +blog +bloomers +blooming +blooper +blot +blouse +blubber +bluff +bluish +blunderer +blunt +blurb +blurred +blurry +blurt +blush +blustery +boaster +boastful +boasting +boat +bobbed +bobbing +bobble +bobcat +bobsled +bobtail +bodacious +body +bogged +boggle +bogus +boil +bok +bolster +bolt +bonanza +bonded +bonding +bondless +boned +bonehead +boneless +bonelike +boney +bonfire +bonnet +bonsai +bonus +bony +boogeyman +boogieman +book +boondocks +booted +booth +bootie +booting +bootlace +bootleg +boots +boozy +borax +boring +borough +borrower +borrowing +boss +botanical +botanist +botany +botch +both +bottle +bottling +bottom +bounce +bouncing +bouncy +bounding +boundless +bountiful +bovine +boxcar +boxer +boxing +boxlike +boxy +breach +breath +breeches +breeching +breeder +breeding +breeze +breezy +brethren +brewery +brewing +briar +bribe +brick +bride +bridged +brigade +bright +brilliant +brim +bring +brink +brisket +briskly +briskness +bristle +brittle +broadband +broadcast +broaden +broadly +broadness +broadside +broadways +broiler +broiling +broken +broker +bronchial +bronco +bronze +bronzing +brook +broom +brought +browbeat +brownnose +browse +browsing +bruising +brunch +brunette +brunt +brush +brussels +brute +brutishly +bubble +bubbling +bubbly +buccaneer +bucked +bucket +buckle +buckshot +buckskin +bucktooth +buckwheat +buddhism +buddhist +budding +buddy +budget +buffalo +buffed +buffer +buffing +buffoon +buggy +bulb +bulge +bulginess +bulgur +bulk +bulldog +bulldozer +bullfight +bullfrog +bullhorn +bullion +bullish +bullpen +bullring +bullseye +bullwhip +bully +bunch +bundle +bungee +bunion +bunkbed +bunkhouse +bunkmate +bunny +bunt +busboy +bush +busily +busload +bust +busybody +buzz +cabana +cabbage +cabbie +cabdriver +cable +caboose +cache +cackle +cacti +cactus +caddie +caddy +cadet +cadillac +cadmium +cage +cahoots +cake +calamari +calamity +calcium +calculate +calculus +caliber +calibrate +calm +caloric +calorie +calzone +camcorder +cameo +camera +camisole +camper +campfire +camping +campsite +campus +canal +canary +cancel +candied +candle +candy +cane +canine +canister +cannabis +canned +canning +cannon +cannot +canola +canon +canopener +canopy +canteen +canyon +capable +capably +capacity +cape +capillary +capital +capitol +capped +capricorn +capsize +capsule +caption +captivate +captive +captivity +capture +caramel +carat +caravan +carbon +cardboard +carded +cardiac +cardigan +cardinal +cardstock +carefully +caregiver +careless +caress +caretaker +cargo +caring +carless +carload +carmaker +carnage +carnation +carnival +carnivore +carol +carpenter +carpentry +carpool +carport +carried +carrot +carrousel +carry +cartel +cartload +carton +cartoon +cartridge +cartwheel +carve +carving +carwash +cascade +case +cash +casing +casino +casket +cassette +casually +casualty +catacomb +catalog +catalyst +catalyze +catapult +cataract +catatonic +catcall +catchable +catcher +catching +catchy +caterer +catering +catfight +catfish +cathedral +cathouse +catlike +catnap +catnip +catsup +cattail +cattishly +cattle +catty +catwalk +caucasian +caucus +causal +causation +cause +causing +cauterize +caution +cautious +cavalier +cavalry +caviar +cavity +cedar +celery +celestial +celibacy +celibate +celtic +cement +census +ceramics +ceremony +certainly +certainty +certified +certify +cesarean +cesspool +chafe +chaffing +chain +chair +chalice +challenge +chamber +chamomile +champion +chance +change +channel +chant +chaos +chaperone +chaplain +chapped +chaps +chapter +character +charbroil +charcoal +charger +charging +chariot +charity +charm +charred +charter +charting +chase +chasing +chaste +chastise +chastity +chatroom +chatter +chatting +chatty +cheating +cheddar +cheek +cheer +cheese +cheesy +chef +chemicals +chemist +chemo +cherisher +cherub +chess +chest +chevron +chevy +chewable +chewer +chewing +chewy +chief +chihuahua +childcare +childhood +childish +childless +childlike +chili +chill +chimp +chip +chirping +chirpy +chitchat +chivalry +chive +chloride +chlorine +choice +chokehold +choking +chomp +chooser +choosing +choosy +chop +chosen +chowder +chowtime +chrome +chubby +chuck +chug +chummy +chump +chunk +churn +chute +cider +cilantro +cinch +cinema +cinnamon +circle +circling +circular +circulate +circus +citable +citadel +citation +citizen +citric +citrus +city +civic +civil +clad +claim +clambake +clammy +clamor +clamp +clamshell +clang +clanking +clapped +clapper +clapping +clarify +clarinet +clarity +clash +clasp +class +clatter +clause +clavicle +claw +clay +clean +clear +cleat +cleaver +cleft +clench +clergyman +clerical +clerk +clever +clicker +client +climate +climatic +cling +clinic +clinking +clip +clique +cloak +clobber +clock +clone +cloning +closable +closure +clothes +clothing +cloud +clover +clubbed +clubbing +clubhouse +clump +clumsily +clumsy +clunky +clustered +clutch +clutter +coach +coagulant +coastal +coaster +coasting +coastland +coastline +coat +coauthor +cobalt +cobbler +cobweb +cocoa +coconut +cod +coeditor +coerce +coexist +coffee +cofounder +cognition +cognitive +cogwheel +coherence +coherent +cohesive +coil +coke +cola +cold +coleslaw +coliseum +collage +collapse +collar +collected +collector +collide +collie +collision +colonial +colonist +colonize +colony +colossal +colt +coma +come +comfort +comfy +comic +coming +comma +commence +commend +comment +commerce +commode +commodity +commodore +common +commotion +commute +commuting +compacted +compacter +compactly +compactor +companion +company +compare +compel +compile +comply +component +composed +composer +composite +compost +composure +compound +compress +comprised +computer +computing +comrade +concave +conceal +conceded +concept +concerned +concert +conch +concierge +concise +conclude +concrete +concur +condense +condiment +condition +condone +conducive +conductor +conduit +cone +confess +confetti +confidant +confident +confider +confiding +configure +confined +confining +confirm +conflict +conform +confound +confront +confused +confusing +confusion +congenial +congested +congrats +congress +conical +conjoined +conjure +conjuror +connected +connector +consensus +consent +console +consoling +consonant +constable +constant +constrain +constrict +construct +consult +consumer +consuming +contact +container +contempt +contend +contented +contently +contents +contest +context +contort +contour +contrite +control +contusion +convene +convent +copartner +cope +copied +copier +copilot +coping +copious +copper +copy +coral +cork +cornball +cornbread +corncob +cornea +corned +corner +cornfield +cornflake +cornhusk +cornmeal +cornstalk +corny +coronary +coroner +corporal +corporate +corral +correct +corridor +corrode +corroding +corrosive +corsage +corset +cortex +cosigner +cosmetics +cosmic +cosmos +cosponsor +cost +cottage +cotton +couch +cough +could +countable +countdown +counting +countless +country +county +courier +covenant +cover +coveted +coveting +coyness +cozily +coziness +cozy +crabbing +crabgrass +crablike +crabmeat +cradle +cradling +crafter +craftily +craftsman +craftwork +crafty +cramp +cranberry +crane +cranial +cranium +crank +crate +crave +craving +crawfish +crawlers +crawling +crayfish +crayon +crazed +crazily +craziness +crazy +creamed +creamer +creamlike +crease +creasing +creatable +create +creation +creative +creature +credible +credibly +credit +creed +creme +creole +crepe +crept +crescent +crested +cresting +crestless +crevice +crewless +crewman +crewmate +crib +cricket +cried +crier +crimp +crimson +cringe +cringing +crinkle +crinkly +crisped +crisping +crisply +crispness +crispy +criteria +critter +croak +crock +crook +croon +crop +cross +crouch +crouton +crowbar +crowd +crown +crucial +crudely +crudeness +cruelly +cruelness +cruelty +crumb +crummiest +crummy +crumpet +crumpled +cruncher +crunching +crunchy +crusader +crushable +crushed +crusher +crushing +crust +crux +crying +cryptic +crystal +cubbyhole +cube +cubical +cubicle +cucumber +cuddle +cuddly +cufflink +culinary +culminate +culpable +culprit +cultivate +cultural +culture +cupbearer +cupcake +cupid +cupped +cupping +curable +curator +curdle +cure +curfew +curing +curled +curler +curliness +curling +curly +curry +curse +cursive +cursor +curtain +curtly +curtsy +curvature +curve +curvy +cushy +cusp +cussed +custard +custodian +custody +customary +customer +customize +customs +cut +cycle +cyclic +cycling +cyclist +cylinder +cymbal +cytoplasm +cytoplast +dab +dad +daffodil +dagger +daily +daintily +dainty +dairy +daisy +dallying +dance +dancing +dandelion +dander +dandruff +dandy +danger +dangle +dangling +daredevil +dares +daringly +darkened +darkening +darkish +darkness +darkroom +darling +darn +dart +darwinism +dash +dastardly +data +datebook +dating +daughter +daunting +dawdler +dawn +daybed +daybreak +daycare +daydream +daylight +daylong +dayroom +daytime +dazzler +dazzling +deacon +deafening +deafness +dealer +dealing +dealmaker +dealt +dean +debatable +debate +debating +debit +debrief +debtless +debtor +debug +debunk +decade +decaf +decal +decathlon +decay +deceased +deceit +deceiver +deceiving +december +decency +decent +deception +deceptive +decibel +decidable +decimal +decimeter +decipher +deck +declared +decline +decode +decompose +decorated +decorator +decoy +decrease +decree +dedicate +dedicator +deduce +deduct +deed +deem +deepen +deeply +deepness +deface +defacing +defame +default +defeat +defection +defective +defendant +defender +defense +defensive +deferral +deferred +defiance +defiant +defile +defiling +define +definite +deflate +deflation +deflator +deflected +deflector +defog +deforest +defraud +defrost +deftly +defuse +defy +degraded +degrading +degrease +degree +dehydrate +deity +dejected +delay +delegate +delegator +delete +deletion +delicacy +delicate +delicious +delighted +delirious +delirium +deliverer +delivery +delouse +delta +deluge +delusion +deluxe +demanding +demeaning +demeanor +demise +democracy +democrat +demote +demotion +demystify +denatured +deniable +denial +denim +denote +dense +density +dental +dentist +denture +deny +deodorant +deodorize +departed +departure +depict +deplete +depletion +deplored +deploy +deport +depose +depraved +depravity +deprecate +depress +deprive +depth +deputize +deputy +derail +deranged +derby +derived +desecrate +deserve +deserving +designate +designed +designer +designing +deskbound +desktop +deskwork +desolate +despair +despise +despite +destiny +destitute +destruct +detached +detail +detection +detective +detector +detention +detergent +detest +detonate +detonator +detoxify +detract +deuce +devalue +deviancy +deviant +deviate +deviation +deviator +device +devious +devotedly +devotee +devotion +devourer +devouring +devoutly +dexterity +dexterous +diabetes +diabetic +diabolic +diagnoses +diagnosis +diagram +dial +diameter +diaper +diaphragm +diary +dice +dicing +dictate +dictation +dictator +difficult +diffused +diffuser +diffusion +diffusive +dig +dilation +diligence +diligent +dill +dilute +dime +diminish +dimly +dimmed +dimmer +dimness +dimple +diner +dingbat +dinghy +dinginess +dingo +dingy +dining +dinner +diocese +dioxide +diploma +dipped +dipper +dipping +directed +direction +directive +directly +directory +direness +dirtiness +disabled +disagree +disallow +disarm +disarray +disaster +disband +disbelief +disburse +discard +discern +discharge +disclose +discolor +discount +discourse +discover +discuss +disdain +disengage +disfigure +disgrace +dish +disinfect +disjoin +disk +dislike +disliking +dislocate +dislodge +disloyal +dismantle +dismay +dismiss +dismount +disobey +disorder +disown +disparate +disparity +dispatch +dispense +dispersal +dispersed +disperser +displace +display +displease +disposal +dispose +disprove +dispute +disregard +disrupt +dissuade +distance +distant +distaste +distill +distinct +distort +distract +distress +district +distrust +ditch +ditto +ditzy +dividable +divided +dividend +dividers +dividing +divinely +diving +divinity +divisible +divisibly +division +divisive +divorcee +dizziness +dizzy +doable +docile +dock +doctrine +document +dodge +dodgy +doily +doing +dole +dollar +dollhouse +dollop +dolly +dolphin +domain +domelike +domestic +dominion +dominoes +donated +donation +donator +donor +donut +doodle +doorbell +doorframe +doorknob +doorman +doormat +doornail +doorpost +doorstep +doorstop +doorway +doozy +dork +dormitory +dorsal +dosage +dose +dotted +doubling +douche +dove +down +dowry +doze +drab +dragging +dragonfly +dragonish +dragster +drainable +drainage +drained +drainer +drainpipe +dramatic +dramatize +drank +drapery +drastic +draw +dreaded +dreadful +dreadlock +dreamboat +dreamily +dreamland +dreamless +dreamlike +dreamt +dreamy +drearily +dreary +drench +dress +drew +dribble +dried +drier +drift +driller +drilling +drinkable +drinking +dripping +drippy +drivable +driven +driver +driveway +driving +drizzle +drizzly +drone +drool +droop +drop-down +dropbox +dropkick +droplet +dropout +dropper +drove +drown +drowsily +drudge +drum +dry +dubbed +dubiously +duchess +duckbill +ducking +duckling +ducktail +ducky +duct +dude +duffel +dugout +duh +duke +duller +dullness +duly +dumping +dumpling +dumpster +duo +dupe +duplex +duplicate +duplicity +durable +durably +duration +duress +during +dusk +dust +dutiful +duty +duvet +dwarf +dweeb +dwelled +dweller +dwelling +dwindle +dwindling +dynamic +dynamite +dynasty +dyslexia +dyslexic +each +eagle +earache +eardrum +earflap +earful +earlobe +early +earmark +earmuff +earphone +earpiece +earplugs +earring +earshot +earthen +earthlike +earthling +earthly +earthworm +earthy +earwig +easeful +easel +easiest +easily +easiness +easing +eastbound +eastcoast +easter +eastward +eatable +eaten +eatery +eating +eats +ebay +ebony +ebook +ecard +eccentric +echo +eclair +eclipse +ecologist +ecology +economic +economist +economy +ecosphere +ecosystem +edge +edginess +edging +edgy +edition +editor +educated +education +educator +eel +effective +effects +efficient +effort +eggbeater +egging +eggnog +eggplant +eggshell +egomaniac +egotism +egotistic +either +eject +elaborate +elastic +elated +elbow +eldercare +elderly +eldest +electable +election +elective +elephant +elevate +elevating +elevation +elevator +eleven +elf +eligible +eligibly +eliminate +elite +elitism +elixir +elk +ellipse +elliptic +elm +elongated +elope +eloquence +eloquent +elsewhere +elude +elusive +elves +email +embargo +embark +embassy +embattled +embellish +ember +embezzle +emblaze +emblem +embody +embolism +emboss +embroider +emcee +emerald +emergency +emission +emit +emote +emoticon +emotion +empathic +empathy +emperor +emphases +emphasis +emphasize +emphatic +empirical +employed +employee +employer +emporium +empower +emptier +emptiness +empty +emu +enable +enactment +enamel +enchanted +enchilada +encircle +enclose +enclosure +encode +encore +encounter +encourage +encroach +encrust +encrypt +endanger +endeared +endearing +ended +ending +endless +endnote +endocrine +endorphin +endorse +endowment +endpoint +endurable +endurance +enduring +energetic +energize +energy +enforced +enforcer +engaged +engaging +engine +engorge +engraved +engraver +engraving +engross +engulf +enhance +enigmatic +enjoyable +enjoyably +enjoyer +enjoying +enjoyment +enlarged +enlarging +enlighten +enlisted +enquirer +enrage +enrich +enroll +enslave +ensnare +ensure +entail +entangled +entering +entertain +enticing +entire +entitle +entity +entomb +entourage +entrap +entree +entrench +entrust +entryway +entwine +enunciate +envelope +enviable +enviably +envious +envision +envoy +envy +enzyme +epic +epidemic +epidermal +epidermis +epidural +epilepsy +epileptic +epilogue +epiphany +episode +equal +equate +equation +equator +equinox +equipment +equity +equivocal +eradicate +erasable +erased +eraser +erasure +ergonomic +errand +errant +erratic +error +erupt +escalate +escalator +escapable +escapade +escapist +escargot +eskimo +esophagus +espionage +espresso +esquire +essay +essence +essential +establish +estate +esteemed +estimate +estimator +estranged +estrogen +etching +eternal +eternity +ethanol +ether +ethically +ethics +euphemism +evacuate +evacuee +evade +evaluate +evaluator +evaporate +evasion +evasive +even +everglade +evergreen +everybody +everyday +everyone +evict +evidence +evident +evil +evoke +evolution +evolve +exact +exalted +example +excavate +excavator +exceeding +exception +excess +exchange +excitable +exciting +exclaim +exclude +excluding +exclusion +exclusive +excretion +excretory +excursion +excusable +excusably +excuse +exemplary +exemplify +exemption +exerciser +exert +exes +exfoliate +exhale +exhaust +exhume +exile +existing +exit +exodus +exonerate +exorcism +exorcist +expand +expanse +expansion +expansive +expectant +expedited +expediter +expel +expend +expenses +expensive +expert +expire +expiring +explain +expletive +explicit +explode +exploit +explore +exploring +exponent +exporter +exposable +expose +exposure +express +expulsion +exquisite +extended +extending +extent +extenuate +exterior +external +extinct +extortion +extradite +extras +extrovert +extrude +extruding +exuberant +fable +fabric +fabulous +facebook +facecloth +facedown +faceless +facelift +faceplate +faceted +facial +facility +facing +facsimile +faction +factoid +factor +factsheet +factual +faculty +fade +fading +failing +falcon +fall +false +falsify +fame +familiar +family +famine +famished +fanatic +fancied +fanciness +fancy +fanfare +fang +fanning +fantasize +fantastic +fantasy +fascism +fastball +faster +fasting +fastness +faucet +favorable +favorably +favored +favoring +favorite +fax +feast +federal +fedora +feeble +feed +feel +feisty +feline +felt-tip +feminine +feminism +feminist +feminize +femur +fence +fencing +fender +ferment +fernlike +ferocious +ferocity +ferret +ferris +ferry +fervor +fester +festival +festive +festivity +fetal +fetch +fever +fiber +fiction +fiddle +fiddling +fidelity +fidgeting +fidgety +fifteen +fifth +fiftieth +fifty +figment +figure +figurine +filing +filled +filler +filling +film +filter +filth +filtrate +finale +finalist +finalize +finally +finance +financial +finch +fineness +finer +finicky +finished +finisher +finishing +finite +finless +finlike +fiscally +fit +five +flaccid +flagman +flagpole +flagship +flagstick +flagstone +flail +flakily +flaky +flame +flammable +flanked +flanking +flannels +flap +flaring +flashback +flashbulb +flashcard +flashily +flashing +flashy +flask +flatbed +flatfoot +flatly +flatness +flatten +flattered +flatterer +flattery +flattop +flatware +flatworm +flavored +flavorful +flavoring +flaxseed +fled +fleshed +fleshy +flick +flier +flight +flinch +fling +flint +flip +flirt +float +flock +flogging +flop +floral +florist +floss +flounder +flyable +flyaway +flyer +flying +flyover +flypaper +foam +foe +fog +foil +folic +folk +follicle +follow +fondling +fondly +fondness +fondue +font +food +fool +footage +football +footbath +footboard +footer +footgear +foothill +foothold +footing +footless +footman +footnote +footpad +footpath +footprint +footrest +footsie +footsore +footwear +footwork +fossil +foster +founder +founding +fountain +fox +foyer +fraction +fracture +fragile +fragility +fragment +fragrance +fragrant +frail +frame +framing +frantic +fraternal +frayed +fraying +frays +freckled +freckles +freebase +freebee +freebie +freedom +freefall +freehand +freeing +freeload +freely +freemason +freeness +freestyle +freeware +freeway +freewill +freezable +freezing +freight +french +frenzied +frenzy +frequency +frequent +fresh +fretful +fretted +friction +friday +fridge +fried +friend +frighten +frightful +frigidity +frigidly +frill +fringe +frisbee +frisk +fritter +frivolous +frolic +from +front +frostbite +frosted +frostily +frosting +frostlike +frosty +froth +frown +frozen +fructose +frugality +frugally +fruit +frustrate +frying +gab +gaffe +gag +gainfully +gaining +gains +gala +gallantly +galleria +gallery +galley +gallon +gallows +gallstone +galore +galvanize +gambling +game +gaming +gamma +gander +gangly +gangrene +gangway +gap +garage +garbage +garden +gargle +garland +garlic +garment +garnet +garnish +garter +gas +gatherer +gathering +gating +gauging +gauntlet +gauze +gave +gawk +gazing +gear +gecko +geek +geiger +gem +gender +generic +generous +genetics +genre +gentile +gentleman +gently +gents +geography +geologic +geologist +geology +geometric +geometry +geranium +gerbil +geriatric +germicide +germinate +germless +germproof +gestate +gestation +gesture +getaway +getting +getup +giant +gibberish +giblet +giddily +giddiness +giddy +gift +gigabyte +gigahertz +gigantic +giggle +giggling +giggly +gigolo +gilled +gills +gimmick +girdle +giveaway +given +giver +giving +gizmo +gizzard +glacial +glacier +glade +gladiator +gladly +glamorous +glamour +glance +glancing +glandular +glare +glaring +glass +glaucoma +glazing +gleaming +gleeful +glider +gliding +glimmer +glimpse +glisten +glitch +glitter +glitzy +gloater +gloating +gloomily +gloomy +glorified +glorifier +glorify +glorious +glory +gloss +glove +glowing +glowworm +glucose +glue +gluten +glutinous +glutton +gnarly +gnat +goal +goatskin +goes +goggles +going +goldfish +goldmine +goldsmith +golf +goliath +gonad +gondola +gone +gong +good +gooey +goofball +goofiness +goofy +google +goon +gopher +gore +gorged +gorgeous +gory +gosling +gossip +gothic +gotten +gout +gown +grab +graceful +graceless +gracious +gradation +graded +grader +gradient +grading +gradually +graduate +graffiti +grafted +grafting +grain +granddad +grandkid +grandly +grandma +grandpa +grandson +granite +granny +granola +grant +granular +grape +graph +grapple +grappling +grasp +grass +gratified +gratify +grating +gratitude +gratuity +gravel +graveness +graves +graveyard +gravitate +gravity +gravy +gray +grazing +greasily +greedily +greedless +greedy +green +greeter +greeting +grew +greyhound +grid +grief +grievance +grieving +grievous +grill +grimace +grimacing +grime +griminess +grimy +grinch +grinning +grip +gristle +grit +groggily +groggy +groin +groom +groove +grooving +groovy +grope +ground +grouped +grout +grove +grower +growing +growl +grub +grudge +grudging +grueling +gruffly +grumble +grumbling +grumbly +grumpily +grunge +grunt +guacamole +guidable +guidance +guide +guiding +guileless +guise +gulf +gullible +gully +gulp +gumball +gumdrop +gumminess +gumming +gummy +gurgle +gurgling +guru +gush +gusto +gusty +gutless +guts +gutter +guy +guzzler +gyration +habitable +habitant +habitat +habitual +hacked +hacker +hacking +hacksaw +had +haggler +haiku +half +halogen +halt +halved +halves +hamburger +hamlet +hammock +hamper +hamster +hamstring +handbag +handball +handbook +handbrake +handcart +handclap +handclasp +handcraft +handcuff +handed +handful +handgrip +handgun +handheld +handiness +handiwork +handlebar +handled +handler +handling +handmade +handoff +handpick +handprint +handrail +handsaw +handset +handsfree +handshake +handstand +handwash +handwork +handwoven +handwrite +handyman +hangnail +hangout +hangover +hangup +hankering +hankie +hanky +haphazard +happening +happier +happiest +happily +happiness +happy +harbor +hardcopy +hardcore +hardcover +harddisk +hardened +hardener +hardening +hardhat +hardhead +hardiness +hardly +hardness +hardship +hardware +hardwired +hardwood +hardy +harmful +harmless +harmonica +harmonics +harmonize +harmony +harness +harpist +harsh +harvest +hash +hassle +haste +hastily +hastiness +hasty +hatbox +hatchback +hatchery +hatchet +hatching +hatchling +hate +hatless +hatred +haunt +haven +hazard +hazelnut +hazily +haziness +hazing +hazy +headache +headband +headboard +headcount +headdress +headed +header +headfirst +headgear +heading +headlamp +headless +headlock +headphone +headpiece +headrest +headroom +headscarf +headset +headsman +headstand +headstone +headway +headwear +heap +heat +heave +heavily +heaviness +heaving +hedge +hedging +heftiness +hefty +helium +helmet +helper +helpful +helping +helpless +helpline +hemlock +hemstitch +hence +henchman +henna +herald +herbal +herbicide +herbs +heritage +hermit +heroics +heroism +herring +herself +hertz +hesitancy +hesitant +hesitate +hexagon +hexagram +hubcap +huddle +huddling +huff +hug +hula +hulk +hull +human +humble +humbling +humbly +humid +humiliate +humility +humming +hummus +humongous +humorist +humorless +humorous +humpback +humped +humvee +hunchback +hundredth +hunger +hungrily +hungry +hunk +hunter +hunting +huntress +huntsman +hurdle +hurled +hurler +hurling +hurray +hurricane +hurried +hurry +hurt +husband +hush +husked +huskiness +hut +hybrid +hydrant +hydrated +hydration +hydrogen +hydroxide +hyperlink +hypertext +hyphen +hypnoses +hypnosis +hypnotic +hypnotism +hypnotist +hypnotize +hypocrisy +hypocrite +ibuprofen +ice +iciness +icing +icky +icon +icy +idealism +idealist +idealize +ideally +idealness +identical +identify +identity +ideology +idiocy +idiom +idly +igloo +ignition +ignore +iguana +illicitly +illusion +illusive +image +imaginary +imagines +imaging +imbecile +imitate +imitation +immature +immerse +immersion +imminent +immobile +immodest +immorally +immortal +immovable +immovably +immunity +immunize +impaired +impale +impart +impatient +impeach +impeding +impending +imperfect +imperial +impish +implant +implement +implicate +implicit +implode +implosion +implosive +imply +impolite +important +importer +impose +imposing +impotence +impotency +impotent +impound +imprecise +imprint +imprison +impromptu +improper +improve +improving +improvise +imprudent +impulse +impulsive +impure +impurity +iodine +iodize +ion +ipad +iphone +ipod +irate +irk +iron +irregular +irrigate +irritable +irritably +irritant +irritate +islamic +islamist +isolated +isolating +isolation +isotope +issue +issuing +italicize +italics +item +itinerary +itunes +ivory +ivy +jab +jackal +jacket +jackknife +jackpot +jailbird +jailbreak +jailer +jailhouse +jalapeno +jam +janitor +january +jargon +jarring +jasmine +jaundice +jaunt +java +jawed +jawless +jawline +jaws +jaybird +jaywalker +jazz +jeep +jeeringly +jellied +jelly +jersey +jester +jet +jiffy +jigsaw +jimmy +jingle +jingling +jinx +jitters +jittery +job +jockey +jockstrap +jogger +jogging +john +joining +jokester +jokingly +jolliness +jolly +jolt +jot +jovial +joyfully +joylessly +joyous +joyride +joystick +jubilance +jubilant +judge +judgingly +judicial +judiciary +judo +juggle +juggling +jugular +juice +juiciness +juicy +jujitsu +jukebox +july +jumble +jumbo +jump +junction +juncture +june +junior +juniper +junkie +junkman +junkyard +jurist +juror +jury +justice +justifier +justify +justly +justness +juvenile +kabob +kangaroo +karaoke +karate +karma +kebab +keenly +keenness +keep +keg +kelp +kennel +kept +kerchief +kerosene +kettle +kick +kiln +kilobyte +kilogram +kilometer +kilowatt +kilt +kimono +kindle +kindling +kindly +kindness +kindred +kinetic +kinfolk +king +kinship +kinsman +kinswoman +kissable +kisser +kissing +kitchen +kite +kitten +kitty +kiwi +kleenex +knapsack +knee +knelt +knickers +knoll +koala +kooky +kosher +krypton +kudos +kung +labored +laborer +laboring +laborious +labrador +ladder +ladies +ladle +ladybug +ladylike +lagged +lagging +lagoon +lair +lake +lance +landed +landfall +landfill +landing +landlady +landless +landline +landlord +landmark +landmass +landmine +landowner +landscape +landside +landslide +language +lankiness +lanky +lantern +lapdog +lapel +lapped +lapping +laptop +lard +large +lark +lash +lasso +last +latch +late +lather +latitude +latrine +latter +latticed +launch +launder +laundry +laurel +lavender +lavish +laxative +lazily +laziness +lazy +lecturer +left +legacy +legal +legend +legged +leggings +legible +legibly +legislate +lego +legroom +legume +legwarmer +legwork +lemon +lend +length +lens +lent +leotard +lesser +letdown +lethargic +lethargy +letter +lettuce +level +leverage +levers +levitate +levitator +liability +liable +liberty +librarian +library +licking +licorice +lid +life +lifter +lifting +liftoff +ligament +likely +likeness +likewise +liking +lilac +lilly +lily +limb +limeade +limelight +limes +limit +limping +limpness +line +lingo +linguini +linguist +lining +linked +linoleum +linseed +lint +lion +lip +liquefy +liqueur +liquid +lisp +list +litigate +litigator +litmus +litter +little +livable +lived +lively +liver +livestock +lividly +living +lizard +lubricant +lubricate +lucid +luckily +luckiness +luckless +lucrative +ludicrous +lugged +lukewarm +lullaby +lumber +luminance +luminous +lumpiness +lumping +lumpish +lunacy +lunar +lunchbox +luncheon +lunchroom +lunchtime +lung +lurch +lure +luridness +lurk +lushly +lushness +luster +lustfully +lustily +lustiness +lustrous +lusty +luxurious +luxury +lying +lyrically +lyricism +lyricist +lyrics +macarena +macaroni +macaw +mace +machine +machinist +magazine +magenta +maggot +magical +magician +magma +magnesium +magnetic +magnetism +magnetize +magnifier +magnify +magnitude +magnolia +mahogany +maimed +majestic +majesty +majorette +majority +makeover +maker +makeshift +making +malformed +malt +mama +mammal +mammary +mammogram +manager +managing +manatee +mandarin +mandate +mandatory +mandolin +manger +mangle +mango +mangy +manhandle +manhole +manhood +manhunt +manicotti +manicure +manifesto +manila +mankind +manlike +manliness +manly +manmade +manned +mannish +manor +manpower +mantis +mantra +manual +many +map +marathon +marauding +marbled +marbles +marbling +march +mardi +margarine +margarita +margin +marigold +marina +marine +marital +maritime +marlin +marmalade +maroon +married +marrow +marry +marshland +marshy +marsupial +marvelous +marxism +mascot +masculine +mashed +mashing +massager +masses +massive +mastiff +matador +matchbook +matchbox +matcher +matching +matchless +material +maternal +maternity +math +mating +matriarch +matrimony +matrix +matron +matted +matter +maturely +maturing +maturity +mauve +maverick +maximize +maximum +maybe +mayday +mayflower +moaner +moaning +mobile +mobility +mobilize +mobster +mocha +mocker +mockup +modified +modify +modular +modulator +module +moisten +moistness +moisture +molar +molasses +mold +molecular +molecule +molehill +mollusk +mom +monastery +monday +monetary +monetize +moneybags +moneyless +moneywise +mongoose +mongrel +monitor +monkhood +monogamy +monogram +monologue +monopoly +monorail +monotone +monotype +monoxide +monsieur +monsoon +monstrous +monthly +monument +moocher +moodiness +moody +mooing +moonbeam +mooned +moonlight +moonlike +moonlit +moonrise +moonscape +moonshine +moonstone +moonwalk +mop +morale +morality +morally +morbidity +morbidly +morphine +morphing +morse +mortality +mortally +mortician +mortified +mortify +mortuary +mosaic +mossy +most +mothball +mothproof +motion +motivate +motivator +motive +motocross +motor +motto +mountable +mountain +mounted +mounting +mourner +mournful +mouse +mousiness +moustache +mousy +mouth +movable +move +movie +moving +mower +mowing +much +muck +mud +mug +mulberry +mulch +mule +mulled +mullets +multiple +multiply +multitask +multitude +mumble +mumbling +mumbo +mummified +mummify +mummy +mumps +munchkin +mundane +municipal +muppet +mural +murkiness +murky +murmuring +muscular +museum +mushily +mushiness +mushroom +mushy +music +musket +muskiness +musky +mustang +mustard +muster +mustiness +musty +mutable +mutate +mutation +mute +mutilated +mutilator +mutiny +mutt +mutual +muzzle +myself +myspace +mystified +mystify +myth +nacho +nag +nail +name +naming +nanny +nanometer +nape +napkin +napped +napping +nappy +narrow +nastily +nastiness +national +native +nativity +natural +nature +naturist +nautical +navigate +navigator +navy +nearby +nearest +nearly +nearness +neatly +neatness +nebula +nebulizer +nectar +negate +negation +negative +neglector +negligee +negligent +negotiate +nemeses +nemesis +neon +nephew +nerd +nervous +nervy +nest +net +neurology +neuron +neurosis +neurotic +neuter +neutron +never +next +nibble +nickname +nicotine +niece +nifty +nimble +nimbly +nineteen +ninetieth +ninja +nintendo +ninth +nuclear +nuclei +nucleus +nugget +nullify +number +numbing +numbly +numbness +numeral +numerate +numerator +numeric +numerous +nuptials +nursery +nursing +nurture +nutcase +nutlike +nutmeg +nutrient +nutshell +nuttiness +nutty +nuzzle +nylon +oaf +oak +oasis +oat +obedience +obedient +obituary +object +obligate +obliged +oblivion +oblivious +oblong +obnoxious +oboe +obscure +obscurity +observant +observer +observing +obsessed +obsession +obsessive +obsolete +obstacle +obstinate +obstruct +obtain +obtrusive +obtuse +obvious +occultist +occupancy +occupant +occupier +occupy +ocean +ocelot +octagon +octane +october +octopus +ogle +oil +oink +ointment +okay +old +olive +olympics +omega +omen +ominous +omission +omit +omnivore +onboard +oncoming +ongoing +onion +online +onlooker +only +onscreen +onset +onshore +onslaught +onstage +onto +onward +onyx +oops +ooze +oozy +opacity +opal +open +operable +operate +operating +operation +operative +operator +opium +opossum +opponent +oppose +opposing +opposite +oppressed +oppressor +opt +opulently +osmosis +other +otter +ouch +ought +ounce +outage +outback +outbid +outboard +outbound +outbreak +outburst +outcast +outclass +outcome +outdated +outdoors +outer +outfield +outfit +outflank +outgoing +outgrow +outhouse +outing +outlast +outlet +outline +outlook +outlying +outmatch +outmost +outnumber +outplayed +outpost +outpour +output +outrage +outrank +outreach +outright +outscore +outsell +outshine +outshoot +outsider +outskirts +outsmart +outsource +outspoken +outtakes +outthink +outward +outweigh +outwit +oval +ovary +oven +overact +overall +overarch +overbid +overbill +overbite +overblown +overboard +overbook +overbuilt +overcast +overcoat +overcome +overcook +overcrowd +overdraft +overdrawn +overdress +overdrive +overdue +overeager +overeater +overexert +overfed +overfeed +overfill +overflow +overfull +overgrown +overhand +overhang +overhaul +overhead +overhear +overheat +overhung +overjoyed +overkill +overlabor +overlaid +overlap +overlay +overload +overlook +overlord +overlying +overnight +overpass +overpay +overplant +overplay +overpower +overprice +overrate +overreach +overreact +override +overripe +overrule +overrun +overshoot +overshot +oversight +oversized +oversleep +oversold +overspend +overstate +overstay +overstep +overstock +overstuff +oversweet +overtake +overthrow +overtime +overtly +overtone +overture +overturn +overuse +overvalue +overview +overwrite +owl +oxford +oxidant +oxidation +oxidize +oxidizing +oxygen +oxymoron +oyster +ozone +paced +pacemaker +pacific +pacifier +pacifism +pacifist +pacify +padded +padding +paddle +paddling +padlock +pagan +pager +paging +pajamas +palace +palatable +palm +palpable +palpitate +paltry +pampered +pamperer +pampers +pamphlet +panama +pancake +pancreas +panda +pandemic +pang +panhandle +panic +panning +panorama +panoramic +panther +pantomime +pantry +pants +pantyhose +paparazzi +papaya +paper +paprika +papyrus +parabola +parachute +parade +paradox +paragraph +parakeet +paralegal +paralyses +paralysis +paralyze +paramedic +parameter +paramount +parasail +parasite +parasitic +parcel +parched +parchment +pardon +parish +parka +parking +parkway +parlor +parmesan +parole +parrot +parsley +parsnip +partake +parted +parting +partition +partly +partner +partridge +party +passable +passably +passage +passcode +passenger +passerby +passing +passion +passive +passivism +passover +passport +password +pasta +pasted +pastel +pastime +pastor +pastrami +pasture +pasty +patchwork +patchy +paternal +paternity +path +patience +patient +patio +patriarch +patriot +patrol +patronage +patronize +pauper +pavement +paver +pavestone +pavilion +paving +pawing +payable +payback +paycheck +payday +payee +payer +paying +payment +payphone +payroll +pebble +pebbly +pecan +pectin +peculiar +peddling +pediatric +pedicure +pedigree +pedometer +pegboard +pelican +pellet +pelt +pelvis +penalize +penalty +pencil +pendant +pending +penholder +penknife +pennant +penniless +penny +penpal +pension +pentagon +pentagram +pep +perceive +percent +perch +percolate +perennial +perfected +perfectly +perfume +periscope +perish +perjurer +perjury +perkiness +perky +perm +peroxide +perpetual +perplexed +persecute +persevere +persuaded +persuader +pesky +peso +pessimism +pessimist +pester +pesticide +petal +petite +petition +petri +petroleum +petted +petticoat +pettiness +petty +petunia +phantom +phobia +phoenix +phonebook +phoney +phonics +phoniness +phony +phosphate +photo +phrase +phrasing +placard +placate +placidly +plank +planner +plant +plasma +plaster +plastic +plated +platform +plating +platinum +platonic +platter +platypus +plausible +plausibly +playable +playback +player +playful +playgroup +playhouse +playing +playlist +playmaker +playmate +playoff +playpen +playroom +playset +plaything +playtime +plaza +pleading +pleat +pledge +plentiful +plenty +plethora +plexiglas +pliable +plod +plop +plot +plow +ploy +pluck +plug +plunder +plunging +plural +plus +plutonium +plywood +poach +pod +poem +poet +pogo +pointed +pointer +pointing +pointless +pointy +poise +poison +poker +poking +polar +police +policy +polio +polish +politely +polka +polo +polyester +polygon +polygraph +polymer +poncho +pond +pony +popcorn +pope +poplar +popper +poppy +popsicle +populace +popular +populate +porcupine +pork +porous +porridge +portable +portal +portfolio +porthole +portion +portly +portside +poser +posh +posing +possible +possibly +possum +postage +postal +postbox +postcard +posted +poster +posting +postnasal +posture +postwar +pouch +pounce +pouncing +pound +pouring +pout +powdered +powdering +powdery +power +powwow +pox +praising +prance +prancing +pranker +prankish +prankster +prayer +praying +preacher +preaching +preachy +preamble +precinct +precise +precision +precook +precut +predator +predefine +predict +preface +prefix +preflight +preformed +pregame +pregnancy +pregnant +preheated +prelaunch +prelaw +prelude +premiere +premises +premium +prenatal +preoccupy +preorder +prepaid +prepay +preplan +preppy +preschool +prescribe +preseason +preset +preshow +president +presoak +press +presume +presuming +preteen +pretended +pretender +pretense +pretext +pretty +pretzel +prevail +prevalent +prevent +preview +previous +prewar +prewashed +prideful +pried +primal +primarily +primary +primate +primer +primp +princess +print +prior +prism +prison +prissy +pristine +privacy +private +privatize +prize +proactive +probable +probably +probation +probe +probing +probiotic +problem +procedure +process +proclaim +procreate +procurer +prodigal +prodigy +produce +product +profane +profanity +professed +professor +profile +profound +profusely +progeny +prognosis +program +progress +projector +prologue +prolonged +promenade +prominent +promoter +promotion +prompter +promptly +prone +prong +pronounce +pronto +proofing +proofread +proofs +propeller +properly +property +proponent +proposal +propose +props +prorate +protector +protegee +proton +prototype +protozoan +protract +protrude +proud +provable +proved +proven +provided +provider +providing +province +proving +provoke +provoking +provolone +prowess +prowler +prowling +proximity +proxy +prozac +prude +prudishly +prune +pruning +pry +psychic +public +publisher +pucker +pueblo +pug +pull +pulmonary +pulp +pulsate +pulse +pulverize +puma +pumice +pummel +punch +punctual +punctuate +punctured +pungent +punisher +punk +pupil +puppet +puppy +purchase +pureblood +purebred +purely +pureness +purgatory +purge +purging +purifier +purify +purist +puritan +purity +purple +purplish +purposely +purr +purse +pursuable +pursuant +pursuit +purveyor +pushcart +pushchair +pusher +pushiness +pushing +pushover +pushpin +pushup +pushy +putdown +putt +puzzle +puzzling +pyramid +pyromania +python +quack +quadrant +quail +quaintly +quake +quaking +qualified +qualifier +qualify +quality +qualm +quantum +quarrel +quarry +quartered +quarterly +quarters +quartet +quench +query +quicken +quickly +quickness +quicksand +quickstep +quiet +quill +quilt +quintet +quintuple +quirk +quit +quiver +quizzical +quotable +quotation +quote +rabid +race +racing +racism +rack +racoon +radar +radial +radiance +radiantly +radiated +radiation +radiator +radio +radish +raffle +raft +rage +ragged +raging +ragweed +raider +railcar +railing +railroad +railway +raisin +rake +raking +rally +ramble +rambling +ramp +ramrod +ranch +rancidity +random +ranged +ranger +ranging +ranked +ranking +ransack +ranting +rants +rare +rarity +rascal +rash +rasping +ravage +raven +ravine +raving +ravioli +ravishing +reabsorb +reach +reacquire +reaction +reactive +reactor +reaffirm +ream +reanalyze +reappear +reapply +reappoint +reapprove +rearrange +rearview +reason +reassign +reassure +reattach +reawake +rebalance +rebate +rebel +rebirth +reboot +reborn +rebound +rebuff +rebuild +rebuilt +reburial +rebuttal +recall +recant +recapture +recast +recede +recent +recess +recharger +recipient +recital +recite +reckless +reclaim +recliner +reclining +recluse +reclusive +recognize +recoil +recollect +recolor +reconcile +reconfirm +reconvene +recopy +record +recount +recoup +recovery +recreate +rectal +rectangle +rectified +rectify +recycled +recycler +recycling +reemerge +reenact +reenter +reentry +reexamine +referable +referee +reference +refill +refinance +refined +refinery +refining +refinish +reflected +reflector +reflex +reflux +refocus +refold +reforest +reformat +reformed +reformer +reformist +refract +refrain +refreeze +refresh +refried +refueling +refund +refurbish +refurnish +refusal +refuse +refusing +refutable +refute +regain +regalia +regally +reggae +regime +region +register +registrar +registry +regress +regretful +regroup +regular +regulate +regulator +rehab +reheat +rehire +rehydrate +reimburse +reissue +reiterate +rejoice +rejoicing +rejoin +rekindle +relapse +relapsing +relatable +related +relation +relative +relax +relay +relearn +release +relenting +reliable +reliably +reliance +reliant +relic +relieve +relieving +relight +relish +relive +reload +relocate +relock +reluctant +rely +remake +remark +remarry +rematch +remedial +remedy +remember +reminder +remindful +remission +remix +remnant +remodeler +remold +remorse +remote +removable +removal +removed +remover +removing +rename +renderer +rendering +rendition +renegade +renewable +renewably +renewal +renewed +renounce +renovate +renovator +rentable +rental +rented +renter +reoccupy +reoccur +reopen +reorder +repackage +repacking +repaint +repair +repave +repaying +repayment +repeal +repeated +repeater +repent +rephrase +replace +replay +replica +reply +reporter +repose +repossess +repost +repressed +reprimand +reprint +reprise +reproach +reprocess +reproduce +reprogram +reps +reptile +reptilian +repugnant +repulsion +repulsive +repurpose +reputable +reputably +request +require +requisite +reroute +rerun +resale +resample +rescuer +reseal +research +reselect +reseller +resemble +resend +resent +reset +reshape +reshoot +reshuffle +residence +residency +resident +residual +residue +resigned +resilient +resistant +resisting +resize +resolute +resolved +resonant +resonate +resort +resource +respect +resubmit +result +resume +resupply +resurface +resurrect +retail +retainer +retaining +retake +retaliate +retention +rethink +retinal +retired +retiree +retiring +retold +retool +retorted +retouch +retrace +retract +retrain +retread +retreat +retrial +retrieval +retriever +retry +return +retying +retype +reunion +reunite +reusable +reuse +reveal +reveler +revenge +revenue +reverb +revered +reverence +reverend +reversal +reverse +reversing +reversion +revert +revisable +revise +revision +revisit +revivable +revival +reviver +reviving +revocable +revoke +revolt +revolver +revolving +reward +rewash +rewind +rewire +reword +rework +rewrap +rewrite +rhyme +ribbon +ribcage +rice +riches +richly +richness +rickety +ricotta +riddance +ridden +ride +riding +rifling +rift +rigging +rigid +rigor +rimless +rimmed +rind +rink +rinse +rinsing +riot +ripcord +ripeness +ripening +ripping +ripple +rippling +riptide +rise +rising +risk +risotto +ritalin +ritzy +rival +riverbank +riverbed +riverboat +riverside +riveter +riveting +roamer +roaming +roast +robbing +robe +robin +robotics +robust +rockband +rocker +rocket +rockfish +rockiness +rocking +rocklike +rockslide +rockstar +rocky +rogue +roman +romp +rope +roping +roster +rosy +rotten +rotting +rotunda +roulette +rounding +roundish +roundness +roundup +roundworm +routine +routing +rover +roving +royal +rubbed +rubber +rubbing +rubble +rubdown +ruby +ruckus +rudder +rug +ruined +rule +rumble +rumbling +rummage +rumor +runaround +rundown +runner +running +runny +runt +runway +rupture +rural +ruse +rush +rust +rut +sabbath +sabotage +sacrament +sacred +sacrifice +sadden +saddlebag +saddled +saddling +sadly +sadness +safari +safeguard +safehouse +safely +safeness +saffron +saga +sage +sagging +saggy +said +saint +sake +salad +salami +salaried +salary +saline +salon +saloon +salsa +salt +salutary +salute +salvage +salvaging +salvation +same +sample +sampling +sanction +sanctity +sanctuary +sandal +sandbag +sandbank +sandbar +sandblast +sandbox +sanded +sandfish +sanding +sandlot +sandpaper +sandpit +sandstone +sandstorm +sandworm +sandy +sanitary +sanitizer +sank +santa +sapling +sappiness +sappy +sarcasm +sarcastic +sardine +sash +sasquatch +sassy +satchel +satiable +satin +satirical +satisfied +satisfy +saturate +saturday +sauciness +saucy +sauna +savage +savanna +saved +savings +savior +savor +saxophone +say +scabbed +scabby +scalded +scalding +scale +scaling +scallion +scallop +scalping +scam +scandal +scanner +scanning +scant +scapegoat +scarce +scarcity +scarecrow +scared +scarf +scarily +scariness +scarring +scary +scavenger +scenic +schedule +schematic +scheme +scheming +schilling +schnapps +scholar +science +scientist +scion +scoff +scolding +scone +scoop +scooter +scope +scorch +scorebook +scorecard +scored +scoreless +scorer +scoring +scorn +scorpion +scotch +scoundrel +scoured +scouring +scouting +scouts +scowling +scrabble +scraggly +scrambled +scrambler +scrap +scratch +scrawny +screen +scribble +scribe +scribing +scrimmage +script +scroll +scrooge +scrounger +scrubbed +scrubber +scruffy +scrunch +scrutiny +scuba +scuff +sculptor +sculpture +scurvy +scuttle +secluded +secluding +seclusion +second +secrecy +secret +sectional +sector +secular +securely +security +sedan +sedate +sedation +sedative +sediment +seduce +seducing +segment +seismic +seizing +seldom +selected +selection +selective +selector +self +seltzer +semantic +semester +semicolon +semifinal +seminar +semisoft +semisweet +senate +senator +send +senior +senorita +sensation +sensitive +sensitize +sensually +sensuous +sepia +september +septic +septum +sequel +sequence +sequester +series +sermon +serotonin +serpent +serrated +serve +service +serving +sesame +sessions +setback +setting +settle +settling +setup +sevenfold +seventeen +seventh +seventy +severity +shabby +shack +shaded +shadily +shadiness +shading +shadow +shady +shaft +shakable +shakily +shakiness +shaking +shaky +shale +shallot +shallow +shame +shampoo +shamrock +shank +shanty +shape +shaping +share +sharpener +sharper +sharpie +sharply +sharpness +shawl +sheath +shed +sheep +sheet +shelf +shell +shelter +shelve +shelving +sherry +shield +shifter +shifting +shiftless +shifty +shimmer +shimmy +shindig +shine +shingle +shininess +shining +shiny +ship +shirt +shivering +shock +shone +shoplift +shopper +shopping +shoptalk +shore +shortage +shortcake +shortcut +shorten +shorter +shorthand +shortlist +shortly +shortness +shorts +shortwave +shorty +shout +shove +showbiz +showcase +showdown +shower +showgirl +showing +showman +shown +showoff +showpiece +showplace +showroom +showy +shrank +shrapnel +shredder +shredding +shrewdly +shriek +shrill +shrimp +shrine +shrink +shrivel +shrouded +shrubbery +shrubs +shrug +shrunk +shucking +shudder +shuffle +shuffling +shun +shush +shut +shy +siamese +siberian +sibling +siding +sierra +siesta +sift +sighing +silenced +silencer +silent +silica +silicon +silk +silliness +silly +silo +silt +silver +similarly +simile +simmering +simple +simplify +simply +sincere +sincerity +singer +singing +single +singular +sinister +sinless +sinner +sinuous +sip +siren +sister +sitcom +sitter +sitting +situated +situation +sixfold +sixteen +sixth +sixties +sixtieth +sixtyfold +sizable +sizably +size +sizing +sizzle +sizzling +skater +skating +skedaddle +skeletal +skeleton +skeptic +sketch +skewed +skewer +skid +skied +skier +skies +skiing +skilled +skillet +skillful +skimmed +skimmer +skimming +skimpily +skincare +skinhead +skinless +skinning +skinny +skintight +skipper +skipping +skirmish +skirt +skittle +skydiver +skylight +skyline +skype +skyrocket +skyward +slab +slacked +slacker +slacking +slackness +slacks +slain +slam +slander +slang +slapping +slapstick +slashed +slashing +slate +slather +slaw +sled +sleek +sleep +sleet +sleeve +slept +sliceable +sliced +slicer +slicing +slick +slider +slideshow +sliding +slighted +slighting +slightly +slimness +slimy +slinging +slingshot +slinky +slip +slit +sliver +slobbery +slogan +sloped +sloping +sloppily +sloppy +slot +slouching +slouchy +sludge +slug +slum +slurp +slush +sly +small +smartly +smartness +smasher +smashing +smashup +smell +smelting +smile +smilingly +smirk +smite +smith +smitten +smock +smog +smoked +smokeless +smokiness +smoking +smoky +smolder +smooth +smother +smudge +smudgy +smuggler +smuggling +smugly +smugness +snack +snagged +snaking +snap +snare +snarl +snazzy +sneak +sneer +sneeze +sneezing +snide +sniff +snippet +snipping +snitch +snooper +snooze +snore +snoring +snorkel +snort +snout +snowbird +snowboard +snowbound +snowcap +snowdrift +snowdrop +snowfall +snowfield +snowflake +snowiness +snowless +snowman +snowplow +snowshoe +snowstorm +snowsuit +snowy +snub +snuff +snuggle +snugly +snugness +speak +spearfish +spearhead +spearman +spearmint +species +specimen +specked +speckled +specks +spectacle +spectator +spectrum +speculate +speech +speed +spellbind +speller +spelling +spendable +spender +spending +spent +spew +sphere +spherical +sphinx +spider +spied +spiffy +spill +spilt +spinach +spinal +spindle +spinner +spinning +spinout +spinster +spiny +spiral +spirited +spiritism +spirits +spiritual +splashed +splashing +splashy +splatter +spleen +splendid +splendor +splice +splicing +splinter +splotchy +splurge +spoilage +spoiled +spoiler +spoiling +spoils +spoken +spokesman +sponge +spongy +sponsor +spoof +spookily +spooky +spool +spoon +spore +sporting +sports +sporty +spotless +spotlight +spotted +spotter +spotting +spotty +spousal +spouse +spout +sprain +sprang +sprawl +spray +spree +sprig +spring +sprinkled +sprinkler +sprint +sprite +sprout +spruce +sprung +spry +spud +spur +sputter +spyglass +squabble +squad +squall +squander +squash +squatted +squatter +squatting +squeak +squealer +squealing +squeamish +squeegee +squeeze +squeezing +squid +squiggle +squiggly +squint +squire +squirt +squishier +squishy +stability +stabilize +stable +stack +stadium +staff +stage +staging +stagnant +stagnate +stainable +stained +staining +stainless +stalemate +staleness +stalling +stallion +stamina +stammer +stamp +stand +stank +staple +stapling +starboard +starch +stardom +stardust +starfish +stargazer +staring +stark +starless +starlet +starlight +starlit +starring +starry +starship +starter +starting +startle +startling +startup +starved +starving +stash +state +static +statistic +statue +stature +status +statute +statutory +staunch +stays +steadfast +steadier +steadily +steadying +steam +steed +steep +steerable +steering +steersman +stegosaur +stellar +stem +stench +stencil +step +stereo +sterile +sterility +sterilize +sterling +sternness +sternum +stew +stick +stiffen +stiffly +stiffness +stifle +stifling +stillness +stilt +stimulant +stimulate +stimuli +stimulus +stinger +stingily +stinging +stingray +stingy +stinking +stinky +stipend +stipulate +stir +stitch +stock +stoic +stoke +stole +stomp +stonewall +stoneware +stonework +stoning +stony +stood +stooge +stool +stoop +stoplight +stoppable +stoppage +stopped +stopper +stopping +stopwatch +storable +storage +storeroom +storewide +storm +stout +stove +stowaway +stowing +straddle +straggler +strained +strainer +straining +strangely +stranger +strangle +strategic +strategy +stratus +straw +stray +streak +stream +street +strength +strenuous +strep +stress +stretch +strewn +stricken +strict +stride +strife +strike +striking +strive +striving +strobe +strode +stroller +strongbox +strongly +strongman +struck +structure +strudel +struggle +strum +strung +strut +stubbed +stubble +stubbly +stubborn +stucco +stuck +student +studied +studio +study +stuffed +stuffing +stuffy +stumble +stumbling +stump +stung +stunned +stunner +stunning +stunt +stupor +sturdily +sturdy +styling +stylishly +stylist +stylized +stylus +suave +subarctic +subatomic +subdivide +subdued +subduing +subfloor +subgroup +subheader +subject +sublease +sublet +sublevel +sublime +submarine +submerge +submersed +submitter +subpanel +subpar +subplot +subprime +subscribe +subscript +subsector +subside +subsiding +subsidize +subsidy +subsoil +subsonic +substance +subsystem +subtext +subtitle +subtly +subtotal +subtract +subtype +suburb +subway +subwoofer +subzero +succulent +such +suction +sudden +sudoku +suds +sufferer +suffering +suffice +suffix +suffocate +suffrage +sugar +suggest +suing +suitable +suitably +suitcase +suitor +sulfate +sulfide +sulfite +sulfur +sulk +sullen +sulphate +sulphuric +sultry +superbowl +superglue +superhero +superior +superjet +superman +supermom +supernova +supervise +supper +supplier +supply +support +supremacy +supreme +surcharge +surely +sureness +surface +surfacing +surfboard +surfer +surgery +surgical +surging +surname +surpass +surplus +surprise +surreal +surrender +surrogate +surround +survey +survival +survive +surviving +survivor +sushi +suspect +suspend +suspense +sustained +sustainer +swab +swaddling +swagger +swampland +swan +swapping +swarm +sway +swear +sweat +sweep +swell +swept +swerve +swifter +swiftly +swiftness +swimmable +swimmer +swimming +swimsuit +swimwear +swinger +swinging +swipe +swirl +switch +swivel +swizzle +swooned +swoop +swoosh +swore +sworn +swung +sycamore +sympathy +symphonic +symphony +symptom +synapse +syndrome +synergy +synopses +synopsis +synthesis +synthetic +syrup +system +t-shirt +tabasco +tabby +tableful +tables +tablet +tableware +tabloid +tackiness +tacking +tackle +tackling +tacky +taco +tactful +tactical +tactics +tactile +tactless +tadpole +taekwondo +tag +tainted +take +taking +talcum +talisman +tall +talon +tamale +tameness +tamer +tamper +tank +tanned +tannery +tanning +tantrum +tapeless +tapered +tapering +tapestry +tapioca +tapping +taps +tarantula +target +tarmac +tarnish +tarot +tartar +tartly +tartness +task +tassel +taste +tastiness +tasting +tasty +tattered +tattle +tattling +tattoo +taunt +tavern +thank +that +thaw +theater +theatrics +thee +theft +theme +theology +theorize +thermal +thermos +thesaurus +these +thesis +thespian +thicken +thicket +thickness +thieving +thievish +thigh +thimble +thing +think +thinly +thinner +thinness +thinning +thirstily +thirsting +thirsty +thirteen +thirty +thong +thorn +those +thousand +thrash +thread +threaten +threefold +thrift +thrill +thrive +thriving +throat +throbbing +throng +throttle +throwaway +throwback +thrower +throwing +thud +thumb +thumping +thursday +thus +thwarting +thyself +tiara +tibia +tidal +tidbit +tidiness +tidings +tidy +tiger +tighten +tightly +tightness +tightrope +tightwad +tigress +tile +tiling +till +tilt +timid +timing +timothy +tinderbox +tinfoil +tingle +tingling +tingly +tinker +tinkling +tinsel +tinsmith +tint +tinwork +tiny +tipoff +tipped +tipper +tipping +tiptoeing +tiptop +tiring +tissue +trace +tracing +track +traction +tractor +trade +trading +tradition +traffic +tragedy +trailing +trailside +train +traitor +trance +tranquil +transfer +transform +translate +transpire +transport +transpose +trapdoor +trapeze +trapezoid +trapped +trapper +trapping +traps +trash +travel +traverse +travesty +tray +treachery +treading +treadmill +treason +treat +treble +tree +trekker +tremble +trembling +tremor +trench +trend +trespass +triage +trial +triangle +tribesman +tribunal +tribune +tributary +tribute +triceps +trickery +trickily +tricking +trickle +trickster +tricky +tricolor +tricycle +trident +tried +trifle +trifocals +trillion +trilogy +trimester +trimmer +trimming +trimness +trinity +trio +tripod +tripping +triumph +trivial +trodden +trolling +trombone +trophy +tropical +tropics +trouble +troubling +trough +trousers +trout +trowel +truce +truck +truffle +trump +trunks +trustable +trustee +trustful +trusting +trustless +truth +try +tubby +tubeless +tubular +tucking +tuesday +tug +tuition +tulip +tumble +tumbling +tummy +turban +turbine +turbofan +turbojet +turbulent +turf +turkey +turmoil +turret +turtle +tusk +tutor +tutu +tux +tweak +tweed +tweet +tweezers +twelve +twentieth +twenty +twerp +twice +twiddle +twiddling +twig +twilight +twine +twins +twirl +twistable +twisted +twister +twisting +twisty +twitch +twitter +tycoon +tying +tyke +udder +ultimate +ultimatum +ultra +umbilical +umbrella +umpire +unabashed +unable +unadorned +unadvised +unafraid +unaired +unaligned +unaltered +unarmored +unashamed +unaudited +unawake +unaware +unbaked +unbalance +unbeaten +unbend +unbent +unbiased +unbitten +unblended +unblessed +unblock +unbolted +unbounded +unboxed +unbraided +unbridle +unbroken +unbuckled +unbundle +unburned +unbutton +uncanny +uncapped +uncaring +uncertain +unchain +unchanged +uncharted +uncheck +uncivil +unclad +unclaimed +unclamped +unclasp +uncle +unclip +uncloak +unclog +unclothed +uncoated +uncoiled +uncolored +uncombed +uncommon +uncooked +uncork +uncorrupt +uncounted +uncouple +uncouth +uncover +uncross +uncrown +uncrushed +uncured +uncurious +uncurled +uncut +undamaged +undated +undaunted +undead +undecided +undefined +underage +underarm +undercoat +undercook +undercut +underdog +underdone +underfed +underfeed +underfoot +undergo +undergrad +underhand +underline +underling +undermine +undermost +underpaid +underpass +underpay +underrate +undertake +undertone +undertook +undertow +underuse +underwear +underwent +underwire +undesired +undiluted +undivided +undocked +undoing +undone +undrafted +undress +undrilled +undusted +undying +unearned +unearth +unease +uneasily +uneasy +uneatable +uneaten +unedited +unelected +unending +unengaged +unenvied +unequal +unethical +uneven +unexpired +unexposed +unfailing +unfair +unfasten +unfazed +unfeeling +unfiled +unfilled +unfitted +unfitting +unfixable +unfixed +unflawed +unfocused +unfold +unfounded +unframed +unfreeze +unfrosted +unfrozen +unfunded +unglazed +ungloved +unglue +ungodly +ungraded +ungreased +unguarded +unguided +unhappily +unhappy +unharmed +unhealthy +unheard +unhearing +unheated +unhelpful +unhidden +unhinge +unhitched +unholy +unhook +unicorn +unicycle +unified +unifier +uniformed +uniformly +unify +unimpeded +uninjured +uninstall +uninsured +uninvited +union +uniquely +unisexual +unison +unissued +unit +universal +universe +unjustly +unkempt +unkind +unknotted +unknowing +unknown +unlaced +unlatch +unlawful +unleaded +unlearned +unleash +unless +unleveled +unlighted +unlikable +unlimited +unlined +unlinked +unlisted +unlit +unlivable +unloaded +unloader +unlocked +unlocking +unlovable +unloved +unlovely +unloving +unluckily +unlucky +unmade +unmanaged +unmanned +unmapped +unmarked +unmasked +unmasking +unmatched +unmindful +unmixable +unmixed +unmolded +unmoral +unmovable +unmoved +unmoving +unnamable +unnamed +unnatural +unneeded +unnerve +unnerving +unnoticed +unopened +unopposed +unpack +unpadded +unpaid +unpainted +unpaired +unpaved +unpeeled +unpicked +unpiloted +unpinned +unplanned +unplanted +unpleased +unpledged +unplowed +unplug +unpopular +unproven +unquote +unranked +unrated +unraveled +unreached +unread +unreal +unreeling +unrefined +unrelated +unrented +unrest +unretired +unrevised +unrigged +unripe +unrivaled +unroasted +unrobed +unroll +unruffled +unruly +unrushed +unsaddle +unsafe +unsaid +unsalted +unsaved +unsavory +unscathed +unscented +unscrew +unsealed +unseated +unsecured +unseeing +unseemly +unseen +unselect +unselfish +unsent +unsettled +unshackle +unshaken +unshaved +unshaven +unsheathe +unshipped +unsightly +unsigned +unskilled +unsliced +unsmooth +unsnap +unsocial +unsoiled +unsold +unsolved +unsorted +unspoiled +unspoken +unstable +unstaffed +unstamped +unsteady +unsterile +unstirred +unstitch +unstopped +unstuck +unstuffed +unstylish +unsubtle +unsubtly +unsuited +unsure +unsworn +untagged +untainted +untaken +untamed +untangled +untapped +untaxed +unthawed +unthread +untidy +untie +until +untimed +untimely +untitled +untoasted +untold +untouched +untracked +untrained +untreated +untried +untrimmed +untrue +untruth +unturned +untwist +untying +unusable +unused +unusual +unvalued +unvaried +unvarying +unveiled +unveiling +unvented +unviable +unvisited +unvocal +unwanted +unwarlike +unwary +unwashed +unwatched +unweave +unwed +unwelcome +unwell +unwieldy +unwilling +unwind +unwired +unwitting +unwomanly +unworldly +unworn +unworried +unworthy +unwound +unwoven +unwrapped +unwritten +unzip +upbeat +upchuck +upcoming +upcountry +update +upfront +upgrade +upheaval +upheld +uphill +uphold +uplifted +uplifting +upload +upon +upper +upright +uprising +upriver +uproar +uproot +upscale +upside +upstage +upstairs +upstart +upstate +upstream +upstroke +upswing +uptake +uptight +uptown +upturned +upward +upwind +uranium +urban +urchin +urethane +urgency +urgent +urging +urologist +urology +usable +usage +useable +used +uselessly +user +usher +usual +utensil +utility +utilize +utmost +utopia +utter +vacancy +vacant +vacate +vacation +vagabond +vagrancy +vagrantly +vaguely +vagueness +valiant +valid +valium +valley +valuables +value +vanilla +vanish +vanity +vanquish +vantage +vaporizer +variable +variably +varied +variety +various +varmint +varnish +varsity +varying +vascular +vaseline +vastly +vastness +veal +vegan +veggie +vehicular +velcro +velocity +velvet +vendetta +vending +vendor +veneering +vengeful +venomous +ventricle +venture +venue +venus +verbalize +verbally +verbose +verdict +verify +verse +version +versus +vertebrae +vertical +vertigo +very +vessel +vest +veteran +veto +vexingly +viability +viable +vibes +vice +vicinity +victory +video +viewable +viewer +viewing +viewless +viewpoint +vigorous +village +villain +vindicate +vineyard +vintage +violate +violation +violator +violet +violin +viper +viral +virtual +virtuous +virus +visa +viscosity +viscous +viselike +visible +visibly +vision +visiting +visitor +visor +vista +vitality +vitalize +vitally +vitamins +vivacious +vividly +vividness +vixen +vocalist +vocalize +vocally +vocation +voice +voicing +void +volatile +volley +voltage +volumes +voter +voting +voucher +vowed +vowel +voyage +wackiness +wad +wafer +waffle +waged +wager +wages +waggle +wagon +wake +waking +walk +walmart +walnut +walrus +waltz +wand +wannabe +wanted +wanting +wasabi +washable +washbasin +washboard +washbowl +washcloth +washday +washed +washer +washhouse +washing +washout +washroom +washstand +washtub +wasp +wasting +watch +water +waviness +waving +wavy +whacking +whacky +wham +wharf +wheat +whenever +whiff +whimsical +whinny +whiny +whisking +whoever +whole +whomever +whoopee +whooping +whoops +why +wick +widely +widen +widget +widow +width +wieldable +wielder +wife +wifi +wikipedia +wildcard +wildcat +wilder +wildfire +wildfowl +wildland +wildlife +wildly +wildness +willed +willfully +willing +willow +willpower +wilt +wimp +wince +wincing +wind +wing +winking +winner +winnings +winter +wipe +wired +wireless +wiring +wiry +wisdom +wise +wish +wisplike +wispy +wistful +wizard +wobble +wobbling +wobbly +wok +wolf +wolverine +womanhood +womankind +womanless +womanlike +womanly +womb +woof +wooing +wool +woozy +word +work +worried +worrier +worrisome +worry +worsening +worshiper +worst +wound +woven +wow +wrangle +wrath +wreath +wreckage +wrecker +wrecking +wrench +wriggle +wriggly +wrinkle +wrinkly +wrist +writing +written +wrongdoer +wronged +wrongful +wrongly +wrongness +wrought +xbox +xerox +yahoo +yam +yanking +yapping +yard +yarn +yeah +yearbook +yearling +yearly +yearning +yeast +yelling +yelp +yen +yesterday +yiddish +yield +yin +yippee +yo-yo +yodel +yoga +yogurt +yonder +yoyo +yummy +zap +zealous +zebra +zen +zeppelin +zero +zestfully +zesty +zigzagged +zipfile +zipping +zippy +zips +zit +zodiac +zombie +zone +zoning +zookeeper +zoologist +zoology +zoom diff --git a/tests/live/conftest.py b/tests/live/conftest.py new file mode 100644 index 0000000..9504a11 --- /dev/null +++ b/tests/live/conftest.py @@ -0,0 +1,86 @@ +"""Fixtures for the session-stream web app suite. + +Everything is standard library and everything is in-process: a real +`LiveServer` on a loopback ephemeral port, fed by a stub capture, with plain +`http.client` for the requests. No compositor, no `grim`, no broker. + +The doubles (`StubCapture`, `read_parts`, `PHRASE`) live in `live_helpers.py`, not +here, so the tests import them by a plain module name -- the same shape as +`tests/unit`. +""" + +from __future__ import annotations + +import contextlib +import http.client +import pathlib +import sys + +import pytest + +_HERE = pathlib.Path(__file__).resolve().parent +_SRC = _HERE.parents[1] / "src" +for _path in (str(_HERE), str(_SRC)): + if _path not in sys.path: + sys.path.insert(0, _path) + +from fake_jpeg import jpeg_bytes # noqa: E402 +from live_helpers import PHRASE, StubCapture # noqa: E402 +from hhstream_live.publisher import StreamRegistry, StreamSpec # noqa: E402 +from hhstream_live.server import LiveServer, RateLimiter # noqa: E402 + + +@pytest.fixture +def jpg(): + return jpeg_bytes + + +@pytest.fixture +def registry(): + reg = StreamRegistry() + yield reg + reg.stop_all() + + +@pytest.fixture +def serve(registry): + """`with serve(phrase=..., capture=..., fps=...) as (server, capture)`.""" + servers: list[LiveServer] = [] + + @contextlib.contextmanager + def _serve(*, phrase: str = PHRASE, capture: StubCapture | None = None, + fps: int = 5, label: str = "demo", limit: int = 30): + capture = capture or StubCapture() + spec = StreamSpec(phrase=phrase, output="hh-demo", fps=fps, session="demo", + label=label) + registry.add(spec, capture) + server = LiveServer(registry, host="127.0.0.1", port=0, + limiter=RateLimiter(limit=limit)) + servers.append(server) + server.start() + assert server.ready.wait(5) + try: + yield server, capture + finally: + server.stop() + + yield _serve + for server in servers: + with contextlib.suppress(Exception): + server.stop() + + +@pytest.fixture +def http_conn(): + """Open an HTTP connection to a running server; closed at teardown.""" + connections: list[http.client.HTTPConnection] = [] + + def _connect(server: LiveServer, timeout: float = 10.0) -> http.client.HTTPConnection: + conn = http.client.HTTPConnection("127.0.0.1", server.port, timeout=timeout) + connections.append(conn) + return conn + + yield _connect + for conn in connections: + with contextlib.suppress(Exception): + conn.close() diff --git a/tests/live/fake_jpeg.py b/tests/live/fake_jpeg.py new file mode 100644 index 0000000..8738a6f --- /dev/null +++ b/tests/live/fake_jpeg.py @@ -0,0 +1,31 @@ +"""A tiny real JPEG, for tests that need decodable image bytes. + +The frame server's tests use a fake that writes a PNG (zlib is in the standard +library). MJPEG needs *JPEG* parts, and there is no JPEG encoder in the stdlib, +so the smallest honest thing is one real 1x1 JPEG embedded as base64 and reused. + +`jpeg_bytes(marker)` rewrites one *quantisation-table* byte, so two frames differ +byte-for-byte (a test can prove "a new frame arrived") while the image stays +valid: the EOI marker, the headers and the scan data are untouched. +""" + +from __future__ import annotations + +import base64 + +#: A 1x1 JPEG. Real enough that `hhstream.capture.image_size` reports (1, 1). +_ONE_PIXEL = ( + "/9j/4AAQSkZJRgABAQEAYABgAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRof" + "Hh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/wAALCAABAAEBAREA/8QAFAAB" + "AAAAAAAAAAAAAAAAAAAACf/EABQQAQAAAAAAAAAAAAAAAAAAAAD/2gAIAQEAAD8AVN//2Q==" +) +#: The start of the quantisation table: `FFDB`, length `0043`, precision/id `00`. +_DQT = b"\xff\xdb\x00C\x00" + + +def jpeg_bytes(marker: int = 0) -> bytes: + """The 1x1 JPEG with one table byte varied by `marker` (0..249).""" + data = bytearray(base64.b64decode(_ONE_PIXEL)) + start = data.index(_DQT) + len(_DQT) + data[start + 63] = (marker % 250) + 1 # a nonzero quantiser keeps it decodable + return bytes(data) diff --git a/tests/live/live_helpers.py b/tests/live/live_helpers.py new file mode 100644 index 0000000..6cb45cf --- /dev/null +++ b/tests/live/live_helpers.py @@ -0,0 +1,82 @@ +"""Shared doubles for the session-stream web app suite. + +Kept out of `conftest.py` on purpose: the rest of the suite imports its doubles +from a plain module (`tests/unit/helpers.py`), and `import conftest` by name is +fragile across directories. +""" + +from __future__ import annotations + +import pathlib +import sys +import threading +import time + +_HERE = pathlib.Path(__file__).resolve().parent +_SRC = _HERE.parents[1] / "src" +for _path in (str(_HERE), str(_SRC)): + if _path not in sys.path: + sys.path.insert(0, _path) + +from fake_jpeg import jpeg_bytes # noqa: E402 + +#: The one phrase the suite enables. Shape matters, not the words. +PHRASE = "amber-otter-lantern-harbor-7k2qp" + + +class StubCapture: + """A `Capture` under the test's control: counts grabs, can be told to fail.""" + + def __init__(self, frames=None, fail_first: int = 0, delay: float = 0.0): + self.frames = list(frames) if frames else [jpeg_bytes(i) for i in range(1, 50)] + self.fail_first = fail_first + self.delay = delay + self.calls = 0 + self._lock = threading.Lock() + + def grab(self) -> bytes: + with self._lock: + self.calls += 1 + call = self.calls + if self.delay: + time.sleep(self.delay) + if call <= self.fail_first: + raise RuntimeError(f"stub capture failing on purpose (call {call})") + return self.frames[(call - 1 - self.fail_first) % len(self.frames)] + + +def read_parts(resp, count: int, timeout: float = 10.0) -> list[bytes]: + """Read up to `count` JPEG parts out of a multipart MJPEG response.""" + from hhstream_live.server import BOUNDARY + + boundary = ("--" + BOUNDARY).encode("ascii") + parts: list[bytes] = [] + buf = b"" + deadline = time.monotonic() + timeout + while len(parts) < count and time.monotonic() < deadline: + chunk = resp.read(4096) + if not chunk: + break + buf += chunk + while True: + start = buf.find(boundary) + if start < 0: + break + header_end = buf.find(b"\r\n\r\n", start) + if header_end < 0: + break + header = buf[start:header_end].decode("latin-1") + length = None + for line in header.splitlines(): + if line.lower().startswith("content-length:"): + length = int(line.split(":", 1)[1].strip()) + if length is None: + break + body_start = header_end + 4 + if len(buf) < body_start + length: + break + parts.append(buf[body_start:body_start + length]) + buf = buf[body_start + length + 2:] + if len(parts) >= count: + break + return parts[:count] diff --git a/tests/live/test_live_cli.py b/tests/live/test_live_cli.py new file mode 100644 index 0000000..a4ce249 --- /dev/null +++ b/tests/live/test_live_cli.py @@ -0,0 +1,50 @@ +"""The command line: it aims a capture at an output, and it can mint a phrase.""" + +from __future__ import annotations + +import io +from contextlib import redirect_stdout + +import pytest +from hhstream_live.__main__ import DEFAULT_CAPTURE, capture_argv, main, parse_stream + + +def test_capture_command_is_aimed_by_inserting_the_output_before_stdout(): + assert capture_argv(DEFAULT_CAPTURE, "hh-demo") == [ + "grim", "-c", "-t", "jpeg", "-q", "80", "-o", "hh-demo", "-", + ] + + +def test_capture_command_without_a_trailing_dash_gets_the_output_appended(): + assert capture_argv(["grim", "-t", "jpeg"], "hh-x") == [ + "grim", "-t", "jpeg", "-o", "hh-x", + ] + + +def test_parse_stream_reads_a_pair_and_normalizes_the_phrase(): + assert parse_stream("Amber-Otter-Lantern-Harbor-7K2QP=hh-demo") == ( + "amber-otter-lantern-harbor-7k2qp", "hh-demo", + ) + + +@pytest.mark.parametrize("bad", ["no-equals-sign", "=hh-demo", "phrase=", "ab=hh"]) +def test_parse_stream_rejects_a_bad_pair(bad): + import argparse + + with pytest.raises(argparse.ArgumentTypeError): + parse_stream(bad) + + +def test_the_phrase_subcommand_prints_a_phrase(): + out = io.StringIO() + with redirect_stdout(out): + assert main(["phrase"]) == 0 + printed = out.getvalue().strip() + assert printed.count("-") == 4 # four words + tail + + +def test_the_phrase_subcommand_can_print_several(): + out = io.StringIO() + with redirect_stdout(out): + main(["phrase", "--count", "3"]) + assert len(out.getvalue().split()) == 3 diff --git a/tests/live/test_live_phrase.py b/tests/live/test_live_phrase.py new file mode 100644 index 0000000..0a4df90 --- /dev/null +++ b/tests/live/test_live_phrase.py @@ -0,0 +1,63 @@ +"""The phrase: shape, entropy, and the rejection of anything unusable.""" + +from __future__ import annotations + +import re + +import pytest +from hhstream_live import phrase + +SHAPE = re.compile(r"^[a-z]+(-[a-z]+){3}-[a-z0-9]{5}$") + + +def test_generated_phrase_has_the_agreed_shape(): + for _ in range(50): + generated = phrase.generate() + assert SHAPE.match(generated), generated + head, _, tail = generated.rpartition("-") + assert len(tail) == 5 + assert all(word in phrase.words() for word in head.split("-")) + + +def test_entropy_is_what_the_plan_claims(): + # Four EFF-large words plus a five-character base-36 tail, ~77.5 bits. + assert phrase.entropy_bits() > 75 + + +def test_wordlist_is_the_eff_large_list(): + words = phrase.words() + assert len(words) == 7776 + assert len(set(words)) == 7776 + assert "abacus" in words + + +def test_generate_is_not_reproducible_from_a_seed(): + # SystemRandom/secrets: seeding the module RNG must not pin the output. + import random + + random.seed(1234) + first = phrase.generate() + random.seed(1234) + second = phrase.generate() + assert first != second + + +def test_normalize_lowercases_and_trims(): + assert phrase.normalize(" Amber-Otter-7K2 ") == "amber-otter-7k2" + + +def test_validate_accepts_a_good_phrase_and_returns_the_normalized_form(): + assert phrase.validate("Amber-Otter-Lantern-7K2QP") == "amber-otter-lantern-7k2qp" + + +@pytest.mark.parametrize("bad", ["", "single", "two-tokens", "has space-phrase-x", + "dash--double-phrase", "-leading-hyphen", + "trailing-hyphen-", "under_score-in-phrase"]) +def test_validate_rejects_a_phrase_that_is_not_the_shape(bad): + with pytest.raises(ValueError): + phrase.validate(bad) + + +def test_validate_rejects_an_overlong_phrase(): + with pytest.raises(ValueError): + phrase.validate("-".join(["averyverylongword"] * 8)) diff --git a/tests/live/test_live_publisher.py b/tests/live/test_live_publisher.py new file mode 100644 index 0000000..24150d5 --- /dev/null +++ b/tests/live/test_live_publisher.py @@ -0,0 +1,125 @@ +"""The publisher: no capture while unwatched, one slot, and giving up when dead.""" + +from __future__ import annotations + +import time + +import pytest +from live_helpers import PHRASE, StubCapture +from hhstream_live.publisher import (FPS_CAP, FAILURE_LIMIT, Stream, StreamRegistry, + StreamSpec) + + +def make_stream(capture, *, fps=10, phrase=PHRASE): + return Stream(StreamSpec(phrase=phrase, output="hh-demo", fps=fps), capture) + + +@pytest.mark.parametrize("bad", [0, -1, True, 1.5, FPS_CAP + 1]) +def test_fps_must_be_a_positive_int_within_the_cap(bad): + with pytest.raises(ValueError): + make_stream(StubCapture(), fps=bad) + + +def test_no_capture_while_nobody_is_watching(): + """An enabled stream costs nothing until a request arrives.""" + capture = StubCapture() + stream = make_stream(capture) + stream.start() + try: + time.sleep(0.4) + assert capture.calls == 0 + finally: + stream.stop() + + +def test_capture_starts_when_a_viewer_arrives_and_stops_when_it_leaves(): + capture = StubCapture() + stream = make_stream(capture) + stream.start() + try: + stream.add_viewer() + frame, seq = stream.wait_frame(0, timeout=5) + assert frame and seq >= 1 + stream.remove_viewer() + time.sleep(0.3) + settled = capture.calls + time.sleep(0.4) + assert capture.calls == settled # idle again + finally: + stream.stop() + + +def test_snapshot_is_none_before_anyone_watches_and_bytes_after(): + stream = make_stream(StubCapture()) + stream.start() + try: + assert stream.snapshot() is None + stream.add_viewer() + frame, _ = stream.wait_frame(0, timeout=5) + stream.remove_viewer() + assert frame is not None + assert stream.snapshot() == frame + finally: + stream.stop() + + +def test_a_slow_viewer_does_not_queue_frames(): + """One slot: a viewer that sleeps sees the latest frame, never a backlog.""" + stream = make_stream(StubCapture(delay=0.01), fps=10) + stream.start() + try: + stream.add_viewer() + first, seq = stream.wait_frame(0, timeout=5) + time.sleep(0.3) # let many frames go by unsent + later, seq2 = stream.wait_frame(seq, timeout=5) + assert later is not None + # seq jumped by more than one: the frames in between were dropped, not + # buffered for us. + assert seq2 > seq + 1 + finally: + stream.stop() + + +def test_a_dead_capture_gives_up_and_closes_viewers(): + capture = StubCapture(fail_first=10_000) # never succeeds + stream = make_stream(capture, fps=10) + stream.start() + try: + stream.add_viewer() + frame, _ = stream.wait_frame(0, timeout=10) + assert frame is None + assert stream.stopped + assert capture.calls >= FAILURE_LIMIT + finally: + stream.stop() + + +# ---- registry -------------------------------------------------------------- + +def test_registry_adds_and_finds_a_phrase_case_insensitively(): + registry = StreamRegistry() + try: + registry.add(StreamSpec(phrase=PHRASE, output="hh-demo"), StubCapture()) + assert registry.get("Amber-Otter-Lantern-Harbor-7K2QP") is not None + assert registry.get("nope-not-here-at-all") is None + finally: + registry.stop_all() + + +def test_registry_refuses_a_duplicate_phrase(): + registry = StreamRegistry() + try: + registry.add(StreamSpec(phrase=PHRASE, output="hh-demo"), StubCapture()) + with pytest.raises(ValueError): + registry.add(StreamSpec(phrase=PHRASE, output="hh-other"), StubCapture()) + finally: + registry.stop_all() + + +def test_registry_remove_stops_the_stream(): + registry = StreamRegistry() + stream = registry.add(StreamSpec(phrase=PHRASE, output="hh-demo"), StubCapture()) + assert registry.remove(PHRASE) is stream + assert registry.get(PHRASE) is None + assert stream.stopped + assert registry.remove(PHRASE) is None diff --git a/tests/live/test_live_server.py b/tests/live/test_live_server.py new file mode 100644 index 0000000..d959666 --- /dev/null +++ b/tests/live/test_live_server.py @@ -0,0 +1,165 @@ +"""The web app's obligations: the router, the pages, and MJPEG over one port.""" + +from __future__ import annotations + +import threading +import time + +from live_helpers import PHRASE, StubCapture, read_parts +from hhstream_live.server import BOUNDARY + +WRONG = "no-such-phrase-here-00000" + + +def get(conn, path): + conn.request("GET", path) + resp = conn.getresponse() + body = resp.read() + return resp.status, resp, body + + +def test_landing_page_is_a_form_and_names_no_phrase(serve, http_conn): + with serve() as (server, _capture): + conn = http_conn(server) + status, resp, body = get(conn, "/") + assert status == 200 + assert resp.getheader("Content-Type").startswith("text/html") + assert PHRASE not in body.decode() + assert "hh-demo" not in body.decode() + assert b"phrase" in body.lower() + + +def test_healthz(serve, http_conn): + with serve() as (server, _): + status, _, body = get(http_conn(server), "/healthz") + assert status == 200 and body.strip() == b"ok" + + +def test_unknown_phrase_is_404_with_an_empty_body(serve, http_conn): + with serve() as (server, _): + status, resp, body = get(http_conn(server), f"/{WRONG}/") + assert status == 404 + assert body == b"" + assert WRONG not in resp.getheader("Content-Type", "") + + +def test_a_wrong_subpath_under_a_valid_phrase_is_404(serve, http_conn): + with serve() as (server, _): + status, _, body = get(http_conn(server), f"/{PHRASE}/secrets.txt") + assert status == 404 and body == b"" + + +def test_viewer_page_names_the_output_and_points_at_the_stream(serve, http_conn): + with serve() as (server, _): + status, _, body = get(http_conn(server), f"/{PHRASE}/") + assert status == 200 + page = body.decode() + assert f"/{PHRASE}/stream.mjpg" in page + assert "hh-demo" in page + + +def test_phrase_is_case_insensitive(serve, http_conn): + with serve() as (server, _): + status, _, _ = get(http_conn(server), f"/{PHRASE.upper()}/") + assert status == 200 + + +def test_status_json_shape(serve, http_conn): + import json + + with serve() as (server, _): + status, resp, body = get(http_conn(server), f"/{PHRASE}/status.json") + assert status == 200 + assert resp.getheader("Content-Type").startswith("application/json") + doc = json.loads(body) + assert doc["session"] == "demo" + assert doc["output"] == "hh-demo" + assert doc["fps"] == 5 + assert doc["viewers"] == 0 + assert doc["stopped"] is False + + +def test_snapshot_is_503_until_someone_has_watched(serve, http_conn): + with serve() as (server, _): + status, _, body = get(http_conn(server), f"/{PHRASE}/frame.jpg") + assert status == 503 and body == b"" + + +def test_stream_is_mjpeg_and_delivers_valid_jpeg_parts(serve, http_conn): + with serve(fps=10) as (server, capture): + conn = http_conn(server) + conn.request("GET", f"/{PHRASE}/stream.mjpg") + resp = conn.getresponse() + assert resp.status == 200 + assert resp.getheader("Content-Type") == ( + f"multipart/x-mixed-replace; boundary={BOUNDARY}" + ) + parts = read_parts(resp, 2) + assert len(parts) == 2 + for part in parts: + assert part.startswith(b"\xff\xd8") # JPEG SOI + assert part.endswith(b"\xff\xd9") # JPEG EOI + conn.close() + + +def test_snapshot_works_after_a_viewer_has_pulled_a_frame(serve, http_conn): + with serve(fps=10) as (server, _): + stream_conn = http_conn(server) + stream_conn.request("GET", f"/{PHRASE}/stream.mjpg") + resp = stream_conn.getresponse() + assert read_parts(resp, 1) + stream_conn.close() + time.sleep(0.05) + + status, ctype, body = get(http_conn(server), f"/{PHRASE}/frame.jpg") + assert status == 200 + assert ctype.getheader("Content-Type") == "image/jpeg" + assert body.startswith(b"\xff\xd8") + + +def test_two_viewers_both_receive_frames(serve, http_conn): + with serve(fps=10) as (server, _): + results: dict[str, object] = {} + + def watch(name: str) -> None: + conn = http_conn(server) + conn.request("GET", f"/{PHRASE}/stream.mjpg") + resp = conn.getresponse() + results[name] = read_parts(resp, 1) + conn.close() + + threads = [threading.Thread(target=watch, args=(n,)) for n in ("a", "b")] + for t in threads: + t.start() + for t in threads: + t.join(timeout=10) + assert results["a"] and results["b"] + + +def test_no_capture_until_a_request_arrives(serve): + with serve() as (server, capture): + time.sleep(0.4) + assert capture.calls == 0 + import http.client + + conn = http.client.HTTPConnection("127.0.0.1", server.port, timeout=10) + conn.request("GET", f"/{PHRASE}/stream.mjpg") + resp = conn.getresponse() + assert read_parts(resp, 1) + conn.close() + assert capture.calls >= 1 + + +def test_repeated_wrong_phrases_are_rate_limited(serve, http_conn): + with serve(limit=3) as (server, _): + conn = http_conn(server) + seen = [get(conn, f"/{WRONG}-{i}/")[0] for i in range(3)] + assert seen == [404, 404, 404] + assert get(conn, f"/{WRONG}-9/")[0] == 429 + + +def test_the_server_binds_and_reports_its_port(serve): + with serve() as (server, _): + assert server.ready.is_set() + assert server.port > 0 + assert server.url_for(PHRASE) == f"http://127.0.0.1:{server.port}/{PHRASE}/" From 52e3786bfb33a181ab1a08c69843ed0848b3b5fa Mon Sep 17 00:00:00 2001 From: Hermenix Date: Tue, 6 Oct 2026 16:19:32 +0000 Subject: [PATCH 2/8] fix(live): a dead output stops the stream even after a good frame The failure cap read `consecutive_failures >= FAILURE_LIMIT and captured == 0`, so it could only ever fire on a cold start. An output removed while someone was watching -- grim answering "unknown output" to every grab from then on -- kept the loop retrying twice a second for as long as the viewer held the page open. That is the busy loop the cap exists to stop, so the count is now of failures in a row and nothing else. Giving up stays sticky: a removed output does not come back, and re-enabling the session is how it returns. While here, the capture thread is named for its output and the give-up is logged by output. The phrase is the stream's only credential, and faulthandler, `threading.excepthook` and any sampling profiler all print thread names. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01URhyHGQnmKH9zBXxoznfQY --- src/hhstream_live/publisher.py | 29 ++++++++++----- tests/live/live_helpers.py | 14 ++++++-- tests/live/test_live_publisher.py | 59 +++++++++++++++++++++++++++++++ 3 files changed, 92 insertions(+), 10 deletions(-) diff --git a/src/hhstream_live/publisher.py b/src/hhstream_live/publisher.py index 081c2e2..ed56809 100644 --- a/src/hhstream_live/publisher.py +++ b/src/hhstream_live/publisher.py @@ -31,9 +31,12 @@ log = logging.getLogger("hhstream.live") #: the box is also running the compositor and the main publisher. DEFAULT_FPS = 5 FPS_CAP = 10 -#: After this many capture failures with no successful frame, a stream gives up -#: and closes its viewers. `grim` against a removed output ("unknown output") -#: fails every time, and retrying forever would be a busy loop on a dead session. +#: After this many *consecutive* capture failures a stream gives up and closes +#: its viewers. `grim` against a removed output ("unknown output") fails every +#: time, and retrying forever would be a busy loop on a dead session -- which is +#: as true after a thousand good frames as it is from a cold start, so the count +#: is of failures in a row, not of failures-before-the-first-frame. A successful +#: grab resets it; giving up does not reset, because a dead output stays dead. FAILURE_LIMIT = 10 @@ -95,8 +98,11 @@ class Stream: # ---- lifecycle --------------------------------------------------- def start(self) -> None: + # Named for the output, never the phrase: faulthandler, the default + # `threading.excepthook` and any profiler that samples the process all + # print thread names, and the phrase is this stream's only credential. self._thread = threading.Thread( - target=self._run, name=f"hhstream-live-{self.spec.phrase}", daemon=True + target=self._run, name=f"hhstream-live-{self.spec.output}", daemon=True ) self._thread.start() @@ -129,15 +135,22 @@ class Stream: with self._cond: self._failures += 1 self._consecutive_failures += 1 - give_up = self._consecutive_failures >= FAILURE_LIMIT and self._captured == 0 + give_up = self._consecutive_failures >= FAILURE_LIMIT if give_up: + # Sticky on purpose: a stream that reached the cap is + # not re-armed by `add_viewer`, because the case this + # guards -- an output that was removed -- does not come + # back. Re-enabling the session is how it comes back. self._stopped = True self._cond.notify_all() if give_up: - log.warning("stream %s: giving up after %d capture failures: %s", - self.spec.phrase, FAILURE_LIMIT, exc) + # By output, not by phrase: a log line is the easiest place + # for a credential to escape to. + log.warning("stream on %s: giving up after %d consecutive " + "capture failures: %s", + self.spec.output, FAILURE_LIMIT, exc) return - log.debug("stream %s: capture failed: %s", self.spec.phrase, exc) + log.debug("stream on %s: capture failed: %s", self.spec.output, exc) if self._sleep(0.5): return continue diff --git a/tests/live/live_helpers.py b/tests/live/live_helpers.py index 6cb45cf..2992690 100644 --- a/tests/live/live_helpers.py +++ b/tests/live/live_helpers.py @@ -25,11 +25,19 @@ PHRASE = "amber-otter-lantern-harbor-7k2qp" class StubCapture: - """A `Capture` under the test's control: counts grabs, can be told to fail.""" + """A `Capture` under the test's control: counts grabs, can be told to fail. - def __init__(self, frames=None, fail_first: int = 0, delay: float = 0.0): + `fail_first` is a capture that is broken from the start (a cold start that + never works); `fail_after` is one that works and then stops -- the output + was removed while someone was watching, which is grim's "unknown output" on + every grab from then on. + """ + + def __init__(self, frames=None, fail_first: int = 0, delay: float = 0.0, + fail_after: int | None = None): self.frames = list(frames) if frames else [jpeg_bytes(i) for i in range(1, 50)] self.fail_first = fail_first + self.fail_after = fail_after self.delay = delay self.calls = 0 self._lock = threading.Lock() @@ -42,6 +50,8 @@ class StubCapture: time.sleep(self.delay) if call <= self.fail_first: raise RuntimeError(f"stub capture failing on purpose (call {call})") + if self.fail_after is not None and call > self.fail_after: + raise RuntimeError(f"stub capture: unknown output (call {call})") return self.frames[(call - 1 - self.fail_first) % len(self.frames)] diff --git a/tests/live/test_live_publisher.py b/tests/live/test_live_publisher.py index 24150d5..9807d9b 100644 --- a/tests/live/test_live_publisher.py +++ b/tests/live/test_live_publisher.py @@ -2,6 +2,7 @@ from __future__ import annotations +import logging import time import pytest @@ -94,6 +95,64 @@ def test_a_dead_capture_gives_up_and_closes_viewers(): stream.stop() +def test_a_capture_that_dies_after_a_good_frame_also_gives_up(): + """The cap counts failures *in a row*, not failures before the first frame. + + An output removed while someone is watching fails every grab from then on. + A give-up rule that only applied to a cold start would leave this stream + retrying twice a second for as long as the viewer held the page open -- + exactly the busy loop on a dead session the cap exists to stop. + """ + capture = StubCapture(fail_after=1) # one good frame, then the output is gone + stream = make_stream(capture, fps=10) + stream.start() + try: + stream.add_viewer() # and the viewer stays attached for the whole test + frame, seq = stream.wait_frame(0, timeout=5) + assert frame is not None and seq >= 1 + + # Poll through the viewer's own wait, so a `wait_frame` that hangs is a + # failure here rather than a test that quietly waits for the deadline. + deadline = time.monotonic() + 15 + while not stream.stopped and time.monotonic() < deadline: + assert stream.wait_frame(seq, timeout=1.0) == (None, seq) + assert stream.stopped, "a dead output must stop the stream, frame or no frame" + + # And the viewer in the loop is released at once rather than on timeout. + started = time.monotonic() + assert stream.wait_frame(seq, timeout=10.0) == (None, seq) + assert time.monotonic() - started < 1.0 + assert capture.calls >= FAILURE_LIMIT + finally: + stream.stop() + + +def test_giving_up_never_writes_the_phrase_to_the_log(caplog): + """A log line is the easiest place for the stream's one credential to leak. + + faulthandler and `threading.excepthook` print thread names, so the capture + thread is checked here too. + """ + capture = StubCapture(fail_first=10_000) + stream = make_stream(capture, fps=10) + with caplog.at_level(logging.DEBUG, logger="hhstream.live"): + stream.start() + thread_name = stream._thread.name + try: + stream.add_viewer() + assert stream.wait_frame(0, timeout=15) == (None, 0) + assert stream.stopped + finally: + stream.stop() + + messages = [record.getMessage() for record in caplog.records] + assert messages, "the give-up must be logged at all" + assert any("hh-demo" in message for message in messages) + for message in messages: + assert PHRASE not in message, message + assert PHRASE not in thread_name and "hh-demo" in thread_name + + # ---- registry -------------------------------------------------------------- def test_registry_adds_and_finds_a_phrase_case_insensitively(): From 19338af17cd7f2fddb0b4a041ff9e5d5d2a5a255 Mon Sep 17 00:00:00 2001 From: Hermenix Date: Tue, 6 Oct 2026 16:19:41 +0000 Subject: [PATCH 3/8] fix(live): a phrase is four plain words and a tail, and nothing weaker Two holes in the one credential a stream has. The wordlist has four hyphenated entries (`drop-down`, `felt-tip`, `t-shirt`, `yo-yo`). A phrase drawn from one is six tokens -- `abacus-drop-down-lantern-harbor-7k2qp` -- and a listener cannot hear where the tail starts. `words()` now keeps only `^[a-z]+$`; 7772 words costs 0.003 bits, so the ~77.5-bit figure is unchanged. The shape test asserted over 50 draws, which caught this about a tenth of the time; the property is now asserted over the whole list. `validate` accepted `a-b-c`. It is what an operator-supplied `--phrase` is checked against, so everything it accepts is a password someone can choose for a LAN-facing stream. The floor is now exactly what `generate` produces: four `[a-z]{3,}` words and a five-character `[a-z0-9]` tail. The router still looks phrases up without validating, so an unknown phrase is the same empty 404 it was. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01URhyHGQnmKH9zBXxoznfQY --- .../plans/2026-10-04-session-stream-webapp.md | 20 ++++--- src/hhstream_live/phrase.py | 55 +++++++++++++------ tests/live/test_live_phrase.py | 50 +++++++++++++++-- 3 files changed, 97 insertions(+), 28 deletions(-) diff --git a/docs/plans/2026-10-04-session-stream-webapp.md b/docs/plans/2026-10-04-session-stream-webapp.md index d86516c..2796523 100644 --- a/docs/plans/2026-10-04-session-stream-webapp.md +++ b/docs/plans/2026-10-04-session-stream-webapp.md @@ -85,14 +85,20 @@ session record so it survives a broker restart and a re-enable. - Shape: `----`, e.g. `amber-otter-lantern-harbor-7k2qp`. -- Words come from a **vendored EFF large wordlist** (7776 words, ~70 KB, in the +- Words come from a **vendored EFF large wordlist** (7776 lines, ~70 KB, in the repo — no network at runtime, no dependency) and the tail is five characters - of `[a-z0-9]`. Four words is `4 · log2(7776) ≈ 51.7` bits and the tail adds - `5 · log2(36) ≈ 25.8`, for **≈ 77.5 bits** — comfortably past anything a LAN - request rate can enumerate, and the operator's stated shape ("5 with a random - 5 character string for the last word"). -- `--phrase` on enable sets your own (checked against the shape); `disable` - then `enable` **rotates** it. A rotation is one record write. + of `[a-z0-9]`. Four of those lines are hyphenated (`drop-down`, `felt-tip`, + `t-shirt`, `yo-yo`) and are dropped on load, because a hyphen is the phrase's + own separator and such a word would make a six-token phrase nobody could hear + the shape of; **7772 words** are usable. Four words is `4 · log2(7772) ≈ 51.7` + bits and the tail adds `5 · log2(36) ≈ 25.8`, for **≈ 77.5 bits** — + comfortably past anything a LAN request rate can enumerate, and the operator's + stated shape ("5 with a random 5 character string for the last word"). +- `--phrase` on enable sets your own, and it is held to **exactly the generated + shape** (four `[a-z]{3,}` words plus a five-character `[a-z0-9]` tail): the + phrase is the stream's only credential, so anything a `--phrase` may be is a + password an operator can choose, and a looser check would accept `a-b-c`. + `disable` then `enable` **rotates** it. A rotation is one record write. - The phrase is the only credential. It is in the URL, in cleartext on the LAN when the LAN is opted into — the threat model is "another device on the LAN can watch if it can guess the phrase", not "the phrase is hidden from the diff --git a/src/hhstream_live/phrase.py b/src/hhstream_live/phrase.py index f5a7c03..be94a3d 100644 --- a/src/hhstream_live/phrase.py +++ b/src/hhstream_live/phrase.py @@ -1,7 +1,7 @@ """The phrase: a stream's only credential, shaped to be said out loud. A phrase is ``word-word-word-word-xxxxx``: four words off the EFF large -wordlist and a five-character base-36 tail. That is ``4*log2(7776) + +wordlist and a five-character base-36 tail. That is ``4*log2(7772) + 5*log2(36)`` ~= **77.5 bits** of entropy, against a word set small enough to be a file in the repository and a tail short enough to type on a phone. @@ -11,8 +11,13 @@ base64 string is not. The job is identical (it *is* the password to the stream); only the shape is chosen for a human. The list is vendored (`wordlist.txt`, 7776 lines) so a stream never needs the -network to start. `generate` uses `SystemRandom`/`secrets`, so the phrase is -not reproducible from the process seed. +network to start. Four of those lines are hyphenated (`drop-down`, `felt-tip`, +`t-shirt`, `yo-yo`) and are dropped on load: a hyphen is the phrase's own +separator, so a phrase built from one would be six tokens and a listener could +not hear where the tail starts. 7772 words costs 0.003 bits. + +`generate` uses `SystemRandom`/`secrets`, so the phrase is not reproducible +from the process seed. """ from __future__ import annotations @@ -24,18 +29,29 @@ import secrets from pathlib import Path #: Four words plus the tail. The plan's answer to "how many"; four is -#: `4 * log2(7776)` ~= 51.7 bits on its own, and the tail pushes it past 77. +#: `4 * log2(7772)` ~= 51.7 bits on its own, and the tail pushes it past 77. WORD_COUNT = 4 SUFFIX_LEN = 5 #: Lowercase letters and digits: what a person can retype without a shift key, #: and 36 symbols is still `5 * log2(36)` ~= 25.8 bits. SUFFIX_ALPHABET = "abcdefghijklmnopqrstuvwxyz0123456789" -#: A phrase is one or more hyphen-joined tokens of lowercase letters/digits, -#: at least three of them (so a caller-supplied phrase cannot be a single word -#: or a passcode-free stub). Generated phrases are five tokens; this is the -#: looser shape a `--phrase` must satisfy. -_SHAPE = re.compile(r"^[a-z0-9]+(?:-[a-z0-9]+){2,}$") +#: The smallest token a word may be. The wordlist's own floor is 3, so this +#: rejects nothing `generate` can produce and everything an operator might type +#: as a stand-in ("a-b-c-d-12345"). +MIN_WORD_LEN = 3 +#: A phrase is exactly what `generate` makes: `WORD_COUNT` alphabetic words of +#: at least `MIN_WORD_LEN` characters, then a `SUFFIX_LEN` tail of `[a-z0-9]`. +#: An operator-supplied `--phrase` is a password for a LAN-facing stream, so it +#: is held to the generated shape rather than to a looser "looks like a phrase" +#: pattern -- a looser one accepts `a-b-c`, and nothing stops an operator from +#: typing that. +_SHAPE = re.compile( + rf"^[a-z]{{{MIN_WORD_LEN},}}(?:-[a-z]{{{MIN_WORD_LEN},}}){{{WORD_COUNT - 1}}}" + rf"-[a-z0-9]{{{SUFFIX_LEN}}}$" +) +#: Only the words whose letters are the whole line; see the module docstring. +_WORD_SHAPE = re.compile(r"^[a-z]+$") MAX_LENGTH = 96 _WORDLIST = Path(__file__).with_name("wordlist.txt") @@ -43,7 +59,11 @@ _cache: tuple[str, ...] | None = None def words() -> tuple[str, ...]: - """The wordlist, read once and remembered. + """The usable wordlist, read once and remembered. + + Only plain `[a-z]+` lines survive: the EFF list contains four hyphenated + entries and the hyphen is the phrase's separator, so one of them would make + a six-token phrase nobody could read out correctly. Raises `RuntimeError` on an empty file rather than returning nothing: a generator that picks from an empty list would raise a confusing `IndexError` @@ -52,9 +72,10 @@ def words() -> tuple[str, ...]: global _cache if _cache is None: text = _WORDLIST.read_text(encoding="utf-8") - loaded = tuple(word.strip() for word in text.splitlines() if word.strip()) + stripped = (line.strip() for line in text.splitlines()) + loaded = tuple(word for word in stripped if _WORD_SHAPE.match(word)) if not loaded: - raise RuntimeError(f"wordlist {_WORDLIST} is empty") + raise RuntimeError(f"wordlist {_WORDLIST} has no usable words") _cache = loaded return _cache @@ -91,12 +112,12 @@ def validate(phrase: str) -> str: raise ValueError(f"phrase is {len(normalized)} characters; the maximum is {MAX_LENGTH}") if not _SHAPE.match(normalized): raise ValueError( - "phrase must be lowercase words/digits joined by hyphens, at least " - "three tokens (e.g. four-words-and-a-tail), not " - f"{phrase!r}" + f"phrase must be {WORD_COUNT} lowercase words of at least " + f"{MIN_WORD_LEN} letters plus a {SUFFIX_LEN}-character [a-z0-9] " + f"tail (e.g. amber-otter-lantern-harbor-7k2qp), not {phrase!r}" ) return normalized -__all__ = ["WORD_COUNT", "SUFFIX_LEN", "generate", "normalize", "validate", - "entropy_bits", "words", "MAX_LENGTH"] +__all__ = ["WORD_COUNT", "SUFFIX_LEN", "MIN_WORD_LEN", "generate", "normalize", + "validate", "entropy_bits", "words", "MAX_LENGTH"] diff --git a/tests/live/test_live_phrase.py b/tests/live/test_live_phrase.py index 0a4df90..2c17b30 100644 --- a/tests/live/test_live_phrase.py +++ b/tests/live/test_live_phrase.py @@ -8,6 +8,7 @@ import pytest from hhstream_live import phrase SHAPE = re.compile(r"^[a-z]+(-[a-z]+){3}-[a-z0-9]{5}$") +WORD = re.compile(r"^[a-z]+$") def test_generated_phrase_has_the_agreed_shape(): @@ -24,11 +25,24 @@ def test_entropy_is_what_the_plan_claims(): assert phrase.entropy_bits() > 75 -def test_wordlist_is_the_eff_large_list(): +def test_wordlist_is_the_eff_large_list_minus_its_hyphenated_entries(): words = phrase.words() - assert len(words) == 7776 - assert len(set(words)) == 7776 + # The file is the EFF large list, 7776 lines; four of them are hyphenated + # and the loader drops them, because a hyphen is the phrase's separator. + assert len(words) == 7772 + assert len(set(words)) == 7772 assert "abacus" in words + assert "drop-down" not in words + + +def test_every_loaded_word_is_plain_lowercase_letters(): + """Not a sampling test: one hyphenated word makes a six-token phrase. + + `test_generated_phrase_has_the_agreed_shape` would only catch that by luck + -- four draws out of 7776 hit a hyphenated entry about 0.2% of the time -- + so the property is asserted over the whole list instead. + """ + assert all(WORD.match(word) for word in phrase.words()) def test_generate_is_not_reproducible_from_a_seed(): @@ -47,7 +61,15 @@ def test_normalize_lowercases_and_trims(): def test_validate_accepts_a_good_phrase_and_returns_the_normalized_form(): - assert phrase.validate("Amber-Otter-Lantern-7K2QP") == "amber-otter-lantern-7k2qp" + assert phrase.validate("Amber-Otter-Lantern-Harbor-7K2QP") == ( + "amber-otter-lantern-harbor-7k2qp" + ) + + +def test_validate_accepts_everything_generate_produces(): + for _ in range(50): + generated = phrase.generate() + assert phrase.validate(generated) == generated @pytest.mark.parametrize("bad", ["", "single", "two-tokens", "has space-phrase-x", @@ -58,6 +80,26 @@ def test_validate_rejects_a_phrase_that_is_not_the_shape(bad): phrase.validate(bad) +@pytest.mark.parametrize("weak", [ + "a-b-c", # the reviewer's example: three letters + "ab-cd-ef-gh-12345", # four "words" of two letters + "amber-otter-lantern-7k2qp", # three words, not four + "amber-otter-lantern-harbor", # four words, no tail + "amber-otter-lantern-harbor-7k2", # a three-character tail + "amber-otter-lantern-harbor-7k2qpz", # a six-character tail + "amber-otter-lantern-harbor-hill-7k2qp", # five words + "amber-0tter-lantern-harbor-7k2qp", # a digit inside a word +]) +def test_validate_rejects_a_phrase_weaker_than_the_generated_shape(weak): + """`--phrase` is a password for a LAN-facing stream, not a label. + + Anything `validate` accepts is something an operator can set, so the floor + has to be the shape `generate` produces -- otherwise `a-b-c` opens a stream. + """ + with pytest.raises(ValueError): + phrase.validate(weak) + + def test_validate_rejects_an_overlong_phrase(): with pytest.raises(ValueError): phrase.validate("-".join(["averyverylongword"] * 8)) From a9299682d2ad07e8914b9da7a263da3cb25b5005 Mon Sep 17 00:00:00 2001 From: Hermenix Date: Tue, 6 Oct 2026 16:19:50 +0000 Subject: [PATCH 4/8] fix(live): bound connections, a delimited stream, and a quiet access log Four things the HTTP side owed. A connection had no read timeout, so a client that connected and sent nothing held a handler thread in `readline` for ever -- and on the MJPEG route it also held a viewer slot, which keeps grim running for nobody. Handlers now carry a 15 s header timeout, cleared once the MJPEG headers are out because from there the wait is on frames, not on the client. TCP keepalive covers the other half: a black-holed viewer that never closes. The MJPEG response had no `Content-Length`, no chunked encoding and no `Connection: close` -- `close_connection = True` was set after `end_headers()`, so it never became a header. An intermediary was entitled to buffer the whole thing waiting for an EOF that never comes. `log_message` formatted the request line, which is the phrase, at debug level; so did the client-error line. Both now log the method, the status and the peer, never the path. `RateLimiter` pruned only the key it was asked about, so every source address that ever missed stayed in the table for the life of the process -- a leak any LAN host could drive. The sweep now covers the table. Finally, `LiveServer` says where it is really bound: `address` still returns something dialable, and `describe_bind` is what names `0.0.0.0` as all interfaces. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01URhyHGQnmKH9zBXxoznfQY --- src/hhstream_live/server.py | 105 +++++++++++++++++++++++++++----- tests/live/test_live_server.py | 106 ++++++++++++++++++++++++++++++++- 2 files changed, 196 insertions(+), 15 deletions(-) diff --git a/src/hhstream_live/server.py b/src/hhstream_live/server.py index bfff9c5..8ebc9e9 100644 --- a/src/hhstream_live/server.py +++ b/src/hhstream_live/server.py @@ -22,6 +22,7 @@ nothing a dependency would make easier. from __future__ import annotations import logging +import socket import threading import time from collections import deque @@ -43,6 +44,11 @@ DEFAULT_HOST = "127.0.0.1" #: Wrong-phrase lookups allowed per client per window before 429. RATE_LIMIT = 30 RATE_WINDOW_S = 60.0 +#: How long a connection may stay silent before it is dropped. It covers the +#: request line and the headers only: a client that connects and says nothing +#: would otherwise hold a handler thread in `readline` for ever, and on the +#: MJPEG route it would hold a viewer slot -- which keeps `grim` running. +HEADER_TIMEOUT_S = 15.0 class RateLimiter: @@ -51,6 +57,10 @@ class RateLimiter: A phrase carries ~77 bits, so guessing is already hopeless; this is here so that a LAN attacker cannot even *try* at line speed, and so the attempt is visible in one place. Only misses are counted. + + The table holds only clients that missed inside the current window: the key + is a client address, which on a LAN is the attacker's to choose, so a table + that only ever grew would be a slow memory leak driven from the network. """ def __init__(self, limit: int = RATE_LIMIT, window: float = RATE_WINDOW_S, @@ -63,16 +73,29 @@ class RateLimiter: def allow(self, key: str) -> bool: now = self._clock() + cutoff = now - self.window with self._lock: + # Prune every key, not just this one. Pruning only `key` would leave + # a client that missed once and never came back in the table for the + # life of the process; the sweep is over the clients seen in the + # last window, which is what the limit already bounds. + for seen, hits in list(self._hits.items()): + while hits and hits[0] < cutoff: + hits.popleft() + if not hits: + del self._hits[seen] hits = self._hits.setdefault(key, deque()) - cutoff = now - self.window - while hits and hits[0] < cutoff: - hits.popleft() if len(hits) >= self.limit: return False hits.append(now) return True + @property + def tracked(self) -> int: + """How many clients the table is holding. For the leak test and status.""" + with self._lock: + return len(self._hits) + def reset(self) -> None: with self._lock: self._hits.clear() @@ -93,6 +116,9 @@ class _Handler(BaseHTTPRequestHandler): protocol_version = "HTTP/1.1" server_version = "hhstream-live" sys_version = "" + #: `StreamRequestHandler.setup` applies this to the socket. Without it a + #: client that opens a connection and sends nothing pins a handler thread. + timeout = HEADER_TIMEOUT_S # ---- plumbing ---------------------------------------------------- @@ -100,8 +126,31 @@ class _Handler(BaseHTTPRequestHandler): def registry(self) -> StreamRegistry: return self.server.registry # type: ignore[attr-defined] + def setup(self) -> None: + super().setup() + # Keepalive is for the other half of the same problem: a viewer whose + # network black-holes never closes, so `sendall` blocks and the viewer + # count never drops -- which keeps `grim` capturing for nobody. The + # kernel's probes end that connection instead. + try: + self.connection.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1) + except OSError as exc: # a platform without it is not worth failing over + log.debug("could not enable TCP keepalive: %s", exc) + def log_message(self, format: str, *args) -> None: # noqa: A002 - base signature - log.debug("%s %s", self.address_string(), format % args) + """Log that something happened, never *what was asked for*. + + The base class formats `self.requestline` into almost every message it + emits, and the request line contains the phrase -- the stream's only + credential. So the format string and its arguments are dropped here and + the method is logged on its own; `log_request` adds the status. + """ + log.debug("%s %s", self.address_string(), self.command or "-") + + def log_request(self, code="-", size="-") -> None: + if isinstance(code, HTTPStatus): + code = code.value + log.debug("%s %s -> %s", self.address_string(), self.command or "-", code) def _send(self, status: HTTPStatus, body: bytes, content_type: str, headers: dict[str, str] | None = None) -> None: @@ -128,7 +177,8 @@ class _Handler(BaseHTTPRequestHandler): except (BrokenPipeError, ConnectionResetError): pass # the viewer went away mid-write; that is not an error except OSError as exc: - log.debug("client error on %s: %s", path, exc) + # Not the path: it carries the phrase. + log.debug("client error from %s: %s", self.address_string(), exc) def _route(self, path: str) -> None: if path in ("", "/"): @@ -191,12 +241,19 @@ class _Handler(BaseHTTPRequestHandler): self.send_header("Content-Type", f"multipart/x-mixed-replace; boundary={BOUNDARY}") self.send_header("Cache-Control", "no-store") self.send_header("X-Content-Type-Options", "nosniff") + # The connection *is* the stream: it has no `Content-Length` and is not + # chunked, so "the response ended" can only mean "the connection ended". + # HTTP/1.1 keeps a connection alive by default, and an intermediary that + # believed that would be entitled to buffer this response until an EOF + # that never comes. Saying so has to happen before `end_headers`, or it + # is not a header at all -- `send_header` also sets `close_connection`. + self.send_header("Connection", "close") self.end_headers() + # The header timeout has done its job; from here the handler waits on + # frames, not on the client, and a quiet minute is normal. + self.connection.settimeout(None) stream.add_viewer() - # The connection is the stream: after this handler returns there is no - # next request to read, so it must not be kept alive. - self.close_connection = True last_seq = 0 try: while True: @@ -225,6 +282,9 @@ class _Handler(BaseHTTPRequestHandler): class LiveServer: """Owns the listener and its lifetime. One instance, many streams.""" + #: Binds that mean "every interface" rather than one address of its own. + WILDCARDS = ("", "0.0.0.0", "::") + def __init__(self, registry: StreamRegistry, *, host: str = DEFAULT_HOST, port: int = DEFAULT_PORT, limiter: RateLimiter | None = None) -> None: self.registry = registry @@ -242,12 +302,28 @@ class LiveServer: return self.requested_port return self._httpd.server_address[1] + @property + def binds_everywhere(self) -> bool: + return self.host in self.WILDCARDS + @property def address(self) -> str: - advertised = self.host - if advertised in ("0.0.0.0", "::"): - advertised = "127.0.0.1" - return advertised + """A host a browser can dial. A wildcard bind has none of its own, so + the loopback name stands in; `describe_bind` is what tells the truth + about the reach.""" + return "127.0.0.1" if self.binds_everywhere else self.host + + def describe_bind(self) -> str: + """One line naming the real bind, and saying when it is every interface. + + `url_for` has to print something dialable, which for `0.0.0.0` is a lie + of omission -- the stream is on the LAN, not on loopback. This is where + that is said out loud. + """ + where = f"{self.host or '0.0.0.0'}:{self.port}" + if self.binds_everywhere: + return f"bound to {where} -- all interfaces, reachable from the LAN" + return f"bound to {where}" def url_for(self, phrase: str) -> str: return f"http://{self.address}:{self.port}/{phrase}/" @@ -264,7 +340,7 @@ class LiveServer: name="hhstream-live-http", daemon=True) self._thread.start() self.ready.set() - log.info("listening on http://%s:%d", self.host, self.port) + log.info("listening, %s", self.describe_bind()) def stop(self) -> None: if self._httpd is not None: @@ -277,4 +353,5 @@ class LiveServer: self.registry.stop_all() -__all__ = ["LiveServer", "RateLimiter", "DEFAULT_PORT", "DEFAULT_HOST", "BOUNDARY"] +__all__ = ["LiveServer", "RateLimiter", "DEFAULT_PORT", "DEFAULT_HOST", "BOUNDARY", + "HEADER_TIMEOUT_S"] diff --git a/tests/live/test_live_server.py b/tests/live/test_live_server.py index d959666..b80872d 100644 --- a/tests/live/test_live_server.py +++ b/tests/live/test_live_server.py @@ -2,11 +2,13 @@ from __future__ import annotations +import logging +import socket import threading import time from live_helpers import PHRASE, StubCapture, read_parts -from hhstream_live.server import BOUNDARY +from hhstream_live.server import BOUNDARY, HEADER_TIMEOUT_S, LiveServer, RateLimiter WRONG = "no-such-phrase-here-00000" @@ -94,6 +96,10 @@ def test_stream_is_mjpeg_and_delivers_valid_jpeg_parts(serve, http_conn): assert resp.getheader("Content-Type") == ( f"multipart/x-mixed-replace; boundary={BOUNDARY}" ) + # The stream has no length and is not chunked, so the only thing that + # can delimit it is the close -- which an intermediary is only entitled + # to expect if the response says so. + assert resp.getheader("Connection") == "close" parts = read_parts(resp, 2) assert len(parts) == 2 for part in parts: @@ -150,6 +156,104 @@ def test_no_capture_until_a_request_arrives(serve): assert capture.calls >= 1 +def test_a_silent_client_does_not_pin_a_handler_thread(serve): + """A connection that never sends a request line must not hold a thread. + + The socket timeout is the only thing that ends it; without one the handler + sits in `readline` for the life of the process. + """ + assert HEADER_TIMEOUT_S <= 30 # a cap that never fires is not a cap + + with serve() as (server, _): + before = threading.active_count() + sock = socket.create_connection(("127.0.0.1", server.port), timeout=5) + try: + time.sleep(0.2) + assert threading.active_count() > before # a handler is waiting on us + sock.settimeout(HEADER_TIMEOUT_S + 10) + started = time.monotonic() + # The server closes on timeout, which arrives here as EOF. + assert sock.recv(4096) == b"" + waited = time.monotonic() - started + assert waited <= HEADER_TIMEOUT_S + 5, waited + finally: + sock.close() + + +def test_a_viewer_that_drops_mid_stream_releases_its_slot(serve, http_conn): + """A dropped viewer must bring `viewers` back to 0, or grim keeps running.""" + import json + + with serve(fps=10) as (server, _): + sock = socket.create_connection(("127.0.0.1", server.port), timeout=10) + sock.sendall(f"GET /{PHRASE}/stream.mjpg HTTP/1.1\r\n" + f"Host: 127.0.0.1\r\n\r\n".encode("ascii")) + assert sock.recv(4096) # headers, and at least the start of a part + + def viewers() -> int: + return json.loads(get(http_conn(server), f"/{PHRASE}/status.json")[2])["viewers"] + + # The headers can reach us a moment before the handler counts itself in. + deadline = time.monotonic() + 10 + while viewers() == 0 and time.monotonic() < deadline: + time.sleep(0.05) + assert viewers() == 1 + + sock.close() # the phone walked out of range + + deadline = time.monotonic() + 10 + while viewers() and time.monotonic() < deadline: + time.sleep(0.1) + assert viewers() == 0 + + +def test_the_access_log_never_carries_the_phrase(serve, http_conn, caplog): + """Debug logging is on in development, and the request line is the phrase.""" + with caplog.at_level(logging.DEBUG, logger="hhstream.live"): + with serve() as (server, _): + assert get(http_conn(server), f"/{PHRASE}/status.json")[0] == 200 + assert get(http_conn(server), f"/{WRONG}/")[0] == 404 + + messages = [record.getMessage() for record in caplog.records] + assert messages, "something must be logged, or this proves nothing" + for message in messages: + assert PHRASE not in message, message + assert WRONG not in message, message + + +def test_the_server_says_when_it_is_bound_to_every_interface(registry): + """`url_for` must print something dialable; the bind line tells the truth.""" + loopback = LiveServer(registry, host="127.0.0.1", port=9988) + assert loopback.describe_bind() == "bound to 127.0.0.1:9988" + assert not loopback.binds_everywhere + + everywhere = LiveServer(registry, host="0.0.0.0", port=9988) + assert everywhere.binds_everywhere + assert everywhere.address == "127.0.0.1" # still dialable + described = everywhere.describe_bind() + assert "0.0.0.0:9988" in described and "all interfaces" in described + + +def test_the_rate_limiter_counts_each_client_separately_and_forgets_them(): + """One noisy client must not spend another's budget, and must not be kept. + + `_hits` is keyed by client address, so a table that only grew would be a + memory leak any LAN host could drive. + """ + now = [0.0] + limiter = RateLimiter(limit=2, window=60.0, clock=lambda: now[0]) + + assert limiter.allow("10.0.0.1") + assert limiter.allow("10.0.0.1") + assert not limiter.allow("10.0.0.1") # that client is out of budget + assert limiter.allow("10.0.0.2") # this one is not + assert limiter.tracked == 2 + + now[0] += 61.0 # both windows have passed + assert limiter.allow("10.0.0.1") + assert limiter.tracked == 1, "a client that went quiet is not kept" + + def test_repeated_wrong_phrases_are_rate_limited(serve, http_conn): with serve(limit=3) as (server, _): conn = http_conn(server) From bd3ca2e64451d1e0777d5532f8990d082b9fc155 Mon Sep 17 00:00:00 2001 From: Hermenix Date: Tue, 6 Oct 2026 16:20:00 +0000 Subject: [PATCH 5/8] fix(live): `hhstream-live` with no subcommand serves, and SIGTERM stops it `cmd_serve` read `args.capture_command`, which only exists on the `serve` subparser, so a bare `hhstream-live` died with an `AttributeError`. The word `serve` is now inserted before parsing rather than defaulted afterwards: the top-level parser would otherwise reject `hhstream-live --port 9999` for an unknown option before anything could be filled in. `--help` is left alone so it still shows the program's own help. `--log-level` gained a copy on each subparser, defaulting to `SUPPRESS` so that naming the subcommand does not reset it. SIGTERM had no handler, so the default action killed the process and `server.stop()` never ran -- the exit status was a signal, not a 0. The handler is installed before the listener is bound, so a supervisor's first signal is never the fatal one. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01URhyHGQnmKH9zBXxoznfQY --- src/hhstream_live/__main__.py | 101 +++++++++++++++++++++++++++------- tests/live/test_live_cli.py | 79 +++++++++++++++++++++++++- 2 files changed, 157 insertions(+), 23 deletions(-) diff --git a/src/hhstream_live/__main__.py b/src/hhstream_live/__main__.py index b0b3dd9..3c0bb80 100644 --- a/src/hhstream_live/__main__.py +++ b/src/hhstream_live/__main__.py @@ -17,12 +17,16 @@ from __future__ import annotations import argparse import logging +import signal import sys +import threading from . import phrase as phrase_mod from .publisher import DEFAULT_FPS, FPS_CAP, StreamRegistry, StreamSpec from .server import DEFAULT_HOST, DEFAULT_PORT, LiveServer +log = logging.getLogger("hhstream.live") + #: The session's own capture command, JPEG for the same reason the host config #: spells it out: a 1280x720 JPEG q80 is ~46 ms and the point is a video, not a #: slideshow. `-c` asks grim to include the cursor. @@ -53,13 +57,27 @@ def parse_stream(arg: str) -> tuple[str, str]: raise argparse.ArgumentTypeError(str(exc)) from exc +def _add_log_level(parser: argparse.ArgumentParser, default: str) -> None: + """Put `--log-level` on a parser. + + It is a global option, but it has to exist on the subparsers too: with no + subcommand the whole line is parsed as `serve ...`, and `serve` is then the + parser that sees it. `SUPPRESS` as the subparser default is what stops + `hhstream-live --log-level debug serve` from being reset to `info` -- a + subparser argument that defaults to `SUPPRESS` and is not supplied leaves + the namespace alone. + """ + parser.add_argument("--log-level", default=default, + choices=["debug", "info", "warning", "error"]) + + def build_parser() -> argparse.ArgumentParser: parser = argparse.ArgumentParser(prog="hhstream-live", description=__doc__) - parser.add_argument("--log-level", default="info", - choices=["debug", "info", "warning", "error"]) + _add_log_level(parser, "info") subs = parser.add_subparsers(dest="command") serve = subs.add_parser("serve", help="run the web app (default)") + _add_log_level(serve, argparse.SUPPRESS) serve.add_argument("--port", type=int, default=DEFAULT_PORT, help=f"the one port every stream shares (default {DEFAULT_PORT})") serve.add_argument("--bind", default=DEFAULT_HOST, @@ -76,11 +94,36 @@ def build_parser() -> argparse.ArgumentParser: "only shape that survives words starting with '-')") gen = subs.add_parser("phrase", help="print a fresh phrase and exit") + _add_log_level(gen, argparse.SUPPRESS) gen.add_argument("--count", type=int, default=1) return parser +#: The subcommands, named here because the default one is chosen before the +#: parser runs. See `parse_args`. +COMMANDS = ("serve", "phrase") + + +def parse_args(argv: list[str] | None = None) -> argparse.Namespace: + """Parse the command line, defaulting to `serve`. + + `serve` is the default, and its options live on its own subparser, so the + word has to be *inserted* rather than filled in afterwards: a namespace from + the top-level parser has no `capture_command`, no `port` and no `stream`, and + `cmd_serve` would die on the first attribute it read. Inserting it has to + happen before parsing, too -- `hhstream-live --port 9999` would otherwise be + rejected for an unknown `--port` before anything could be defaulted. + + `--help` is left alone so that a bare `hhstream-live --help` still shows the + program's own help rather than `serve`'s. + """ + raw = list(sys.argv[1:] if argv is None else argv) + if not any(word in COMMANDS or word in ("-h", "--help") for word in raw): + raw = ["serve", *raw] + return build_parser().parse_args(raw) + + def make_registry(streams: list[tuple[str, str]], capture_base: list[str], fps: int) -> StreamRegistry: from hhstream.capture import CommandCapture @@ -97,42 +140,58 @@ def cmd_serve(args: argparse.Namespace) -> int: registry = make_registry(args.stream, capture_base, args.fps) server = LiveServer(registry, host=args.bind, port=args.port) - print(f"landing page: http://{server.address}:{args.port}/", file=sys.stderr) - if not args.stream: - print("no --stream given; only the landing page will answer", file=sys.stderr) - server.start() # raises OSError loudly if the port is taken - for phrase in registry.phrases(): - print(server.url_for(phrase)) - sys.stdout.flush() + # A service is stopped with SIGTERM, not with Ctrl-C: without a handler the + # default action kills the process outright and `server.stop` never runs, so + # the listener and the capture threads go down with no ordering at all. + # Installed *before* the listener exists, so there is no window in which a + # supervisor's very first signal is the fatal one. + stopping = threading.Event() + previous = None + + def _request_stop(signum, frame) -> None: # noqa: ARG001 - handler signature + stopping.set() try: - import time + previous = signal.signal(signal.SIGTERM, _request_stop) + except ValueError: + # Not the main thread: an embedder's problem to solve, not ours. + log.debug("SIGTERM handler not installed (not the main thread)") - while True: - time.sleep(3600) + try: + server.start() # raises OSError loudly if the port is taken + # Printed after `start`, because a requested port of 0 only has an answer + # once it is bound. `describe_bind` names the real bind, so `0.0.0.0` is + # not reported as loopback just because that is the address you click. + print(server.describe_bind(), file=sys.stderr) + print(f"landing page: http://{server.address}:{server.port}/", file=sys.stderr) + if not args.stream: + print("no --stream given; only the landing page will answer", + file=sys.stderr) + for phrase in registry.phrases(): + print(server.url_for(phrase)) + sys.stdout.flush() + stopping.wait() + print("stopping", file=sys.stderr) except KeyboardInterrupt: print("stopping", file=sys.stderr) finally: + if previous is not None: + signal.signal(signal.SIGTERM, previous) server.stop() return 0 def main(argv: list[str] | None = None) -> int: - parser = build_parser() - args = parser.parse_args(argv) + args = parse_args(argv) logging.basicConfig( - level=getattr(logging, (args.log_level or "info").upper()), + level=getattr(logging, (getattr(args, "log_level", None) or "info").upper()), format="%(asctime)s %(levelname)s %(name)s: %(message)s", ) - command = args.command or "serve" - if command == "phrase": + if args.command == "phrase": for _ in range(args.count): print(phrase_mod.generate()) return 0 - if command == "serve": - return cmd_serve(args) - parser.error(f"unknown command {command!r}") # pragma: no cover - return 2 + return cmd_serve(args) if __name__ == "__main__": diff --git a/tests/live/test_live_cli.py b/tests/live/test_live_cli.py index a4ce249..f4ad0b6 100644 --- a/tests/live/test_live_cli.py +++ b/tests/live/test_live_cli.py @@ -3,10 +3,22 @@ from __future__ import annotations import io +import os +import pathlib +import re +import signal +import subprocess +import sys +import time from contextlib import redirect_stdout import pytest -from hhstream_live.__main__ import DEFAULT_CAPTURE, capture_argv, main, parse_stream +from hhstream_live.__main__ import (DEFAULT_CAPTURE, capture_argv, main, parse_args, + parse_stream) + +#: What `hhstream-live phrase` must print: four alphabetic words and a tail. +PHRASE_SHAPE = re.compile(r"^[a-z]+(?:-[a-z]+){3}-[a-z0-9]{5}$") +_SRC = pathlib.Path(__file__).resolve().parents[2] / "src" def test_capture_command_is_aimed_by_inserting_the_output_before_stdout(): @@ -40,7 +52,9 @@ def test_the_phrase_subcommand_prints_a_phrase(): with redirect_stdout(out): assert main(["phrase"]) == 0 printed = out.getvalue().strip() - assert printed.count("-") == 4 # four words + tail + # Not just `count("-") == 4`: a hyphenated word on the list would satisfy + # the count with five words and four hyphens in the wrong places. + assert PHRASE_SHAPE.match(printed), printed def test_the_phrase_subcommand_can_print_several(): @@ -48,3 +62,64 @@ def test_the_phrase_subcommand_can_print_several(): with redirect_stdout(out): main(["phrase", "--count", "3"]) assert len(out.getvalue().split()) == 3 + + +# ---- the default subcommand, and shutdown ---------------------------------- + +def test_a_bare_command_line_means_serve(): + """`hhstream-live` with no subcommand is `hhstream-live serve`. + + The serve options live on the `serve` subparser, so the namespace has to be + the subparser's; reading `capture_command` off the top-level one is the + `AttributeError` this guards. + """ + args = parse_args([]) + assert args.command == "serve" + assert args.capture_command is None + assert args.port and args.bind and args.stream == [] + assert args.log_level == "info" + + +def test_serve_options_work_without_naming_the_subcommand(): + args = parse_args(["--port", "0", "--bind", "0.0.0.0"]) + assert (args.command, args.port, args.bind) == ("serve", 0, "0.0.0.0") + + +def test_a_global_option_survives_the_subcommand(): + """`--log-level` is on both parsers; the subparser must not reset it.""" + assert parse_args(["--log-level", "debug"]).log_level == "debug" + assert parse_args(["--log-level", "debug", "serve"]).log_level == "debug" + assert parse_args(["--log-level", "debug", "phrase"]).log_level == "debug" + + +def test_a_bare_run_serves_and_sigterm_stops_it_cleanly(): + """The whole path: no subcommand, then the signal a service is stopped with. + + `SIGTERM`'s default action kills the process outright, so without a handler + `server.stop()` never runs and the exit status is a signal, not a 0. + """ + env = dict(os.environ, PYTHONPATH=os.pathsep.join( + [str(_SRC), os.environ.get("PYTHONPATH", "")]).rstrip(os.pathsep)) + proc = subprocess.Popen( + [sys.executable, "-m", "hhstream_live", "--port", "0"], + stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, env=env, + ) + try: + deadline = time.monotonic() + 20 + banner = "" + while "landing page:" not in banner and time.monotonic() < deadline: + line = proc.stderr.readline() + if not line: + break + banner += line + assert "landing page:" in banner, banner + assert "bound to 127.0.0.1:" in banner, banner + + proc.send_signal(signal.SIGTERM) + out, err = proc.communicate(timeout=20) + except BaseException: + proc.kill() + proc.communicate() + raise + assert proc.returncode == 0, (proc.returncode, banner, err) + assert "Traceback" not in err, err From db7aaf82764e1ccefcb0b8cfddf3ca887889c360 Mon Sep 17 00:00:00 2001 From: Hermenix Date: Tue, 6 Oct 2026 16:20:00 +0000 Subject: [PATCH 6/8] build(nix): prove the wordlist is actually installed `pythonImportsCheck` only imports, and `hhstream_live.phrase` reads its wordlist lazily, so a wheel that dropped `wordlist.txt` sailed past the check and would have failed at the first `enable` instead. The package now reads the list from the installed tree and mints a phrase through `validate`, which is the only place the packaged data is exercised -- `doCheck = false`, the suite runs in the flake's `checks`. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01URhyHGQnmKH9zBXxoznfQY --- nix/package.nix | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/nix/package.nix b/nix/package.nix index 67bd480..1736493 100644 --- a/nix/package.nix +++ b/nix/package.nix @@ -29,11 +29,27 @@ python3.pkgs.buildPythonApplication { "hhstream_pointer.ops" "hhstream_pointer._protocol.wlr_virtual_pointer_unstable_v1" # The web app: importing the server pulls in the router, the publisher and - # the pages. The phrase generator's wordlist is covered by the unit suite, - # which reads the *installed* package. + # the pages. "hhstream_live.server" ]; + # `pythonImportsCheck` only *imports*, and `hhstream_live.phrase` reads its + # wordlist lazily -- so a wheel that dropped `wordlist.txt` would sail past it + # and fail at the first `enable` instead. Read the list for real, against the + # installed tree, and mint a phrase so the generator and its validator are + # proven on the same words. `doCheck = false` (the suite runs in the flake's + # `checks`), so this is the only place the packaged data is exercised. + postFixup = '' + echo "checking the installed wordlist..." + PYTHONPATH="$out/${python3.sitePackages}:$PYTHONPATH" ${python3.interpreter} -c ' +import hhstream_live.phrase as phrase +count = len(phrase.words()) +assert count > 7000, f"wordlist has only {count} words; is wordlist.txt installed?" +assert phrase.validate(phrase.generate()) +print(f"wordlist ok: {count} words, ~{phrase.entropy_bits():.1f} bits a phrase") +' + ''; + meta = { description = "Publish a headless Hyprland session's frames on a Unix socket"; license = lib.licenses.mit; From 714190007036565e47b2885f4a2ce8846810b28e Mon Sep 17 00:00:00 2001 From: Hermenix Date: Sun, 11 Oct 2026 02:22:00 +0000 Subject: [PATCH 7/8] module: declare the stream's port, and the one hole it needs `sessions.stream { port, bind, openFirewall }` -- the piece of stage 3 that does not need the broker. It exists because a machine had to be able to *reach* a stream before the rest of the app is settled: the operator asked for the port to be open on the deployment machine, where the app still runs by hand, and a bare `networking.firewall.allowedTCPPorts` line in one machine's config is the wrong place for a fact about this flake. - `openFirewall` defaults false and is deliberately not implied by `sessions.enable`: a live view of the desktop is readable by anyone who can reach the port, the phrase in the URL is its only credential, and there is no TLS. It opens exactly one port -- the one the option names. - `bind` defaults to loopback and is *checked*: `openFirewall = true` beside a loopback bind is refused, because that is a hole with no service behind it. - `checks/vm-stream-firewall.nix` boots the module with a configured port that is not the module's default, puts listeners on that port and on a neighbour, and asserts from a second machine on the same VLAN that the peer reaches the configured port and is refused on the other -- while both answer on loopback inside the guest, so the refusal is the firewall's doing and not an empty socket. The peer is a second machine because a probe from the test driver proves nothing: the driver is not on the test VLAN. Nothing here turns a stream on -- no unit, no phrases, no `session stream` verbs, which stay stage 2+3 -- and docs/plans/2026-10-04-session-stream-webapp.md now records which part of its stage 3 landed and why out of order. --- README.dev.md | 3 + checks/default.nix | 7 + checks/vm-stream-firewall.nix | 243 ++++++++++++++++++ .../plans/2026-10-04-session-stream-webapp.md | 16 +- nix/module.nix | 88 +++++++ 5 files changed, 356 insertions(+), 1 deletion(-) create mode 100644 checks/vm-stream-firewall.nix diff --git a/README.dev.md b/README.dev.md index 38a4091..f345d03 100644 --- a/README.dev.md +++ b/README.dev.md @@ -48,6 +48,7 @@ and the pointer driver. | `vm-ipc` | the **compositor's IPC**: both sockets exist at the names `docs/CONTRACT-sessions.md` §5 freezes and fit inside `sun_path`, `hyprctl` answers, the broker's own `HyprlandIPC` reaches it, and a subscriber on the **event** socket receives an `openwindow` for a window the test creates | yes | | `vm-containers` | one `hh-*` output per session at the container mode, the application's window on it, a second window of the same application beside it, an unmatched window **quarantined**, `grim -o` capturing that container alone, and teardown removing the output and freeing its workspace slot | yes | | `vm-pointer` | the **lease and the driver**: a version-2 virtual pointer, a warp that lands where it was aimed *as the application reports it*, a click delivered as one press and one release, a contended acquire refused with BUSY, an expired lease swept with its held button lifted, and a broker restart that leaves the lock free, the bookmark intact and the driver reachable | yes | +| `vm-stream-firewall` | the stream's **port**: the guest's own running firewall accepts the configured port and not the module's default, a peer outside the guest reaches it, and is refused on a port beside it that was never opened | yes | All of them at once: @@ -69,6 +70,7 @@ nix build -L .#checks.x86_64-linux.vm-sessions nix build -L .#checks.x86_64-linux.vm-ipc nix build -L .#checks.x86_64-linux.vm-containers nix build -L .#checks.x86_64-linux.vm-pointer +nix build -L .#checks.x86_64-linux.vm-stream-firewall ``` `-L` is not optional in practice: without it, Nix hides the build log, and the @@ -219,6 +221,7 @@ checks/ vm-ipc.nix nixosTest, the compositor's request and event sockets vm-containers.nix nixosTest, one output and one workspace per session vm-pointer.nix nixosTest, the lease, the driver and a click that lands + vm-stream-firewall.nix nixosTest, the stream's port in the guest's firewall lib/test-tools.nix packages the tests/ helpers for the guest lib/session-tools.nix packages the two guest programs below lib/session-machine.nix the guest the three session checks share diff --git a/checks/default.nix b/checks/default.nix index 8f01f56..703b5f1 100644 --- a/checks/default.nix +++ b/checks/default.nix @@ -7,6 +7,7 @@ # vm-ipc -- the compositor's IPC sockets, and an event that arrives # vm-containers -- one output and one workspace per session # vm-pointer -- the lease, the driver, and a click the application saw +# vm-stream-firewall -- the stream's port in the guest's own running firewall # # What the guests need lives in checks/lib: `test-tools.nix` for the fake # capture and the contract asserter, `session-tools.nix` for the event @@ -53,4 +54,10 @@ in vm-containers = pkgs.callPackage ./vm-containers.nix sessionArgs; vm-pointer = pkgs.callPackage ./vm-pointer.nix sessionArgs; + + # Not a `sessionArgs` consumer: this one is about the module's own options, + # not about the broker or the compositor (see the file's header). + vm-stream-firewall = pkgs.callPackage ./vm-stream-firewall.nix { + inherit hhstream hhsModule testTools; + }; } diff --git a/checks/vm-stream-firewall.nix b/checks/vm-stream-firewall.nix new file mode 100644 index 0000000..8acc6d2 --- /dev/null +++ b/checks/vm-stream-firewall.nix @@ -0,0 +1,243 @@ +# checks..vm-stream-firewall -- the stream's port, and only that port. +# +# `sessions.stream.openFirewall` is the one part of the session-stream web app +# that a machine declares before the app has a unit (it is the piece of +# docs/plans/2026-10-04-session-stream-webapp.md §3 that does not need the +# broker), so it is the one part worth gating on its own. +# +# The option's effect is a firewall rule, and a rule is only a rule once the +# firewall has loaded it -- so this boots the module rather than evaluating it +# and reading `config.networking.firewall.allowedTCPPorts` back. A test that +# reads the option out of the evaluation proves the option is plumbed; this one +# asks the running firewall, and then asks it from *outside* the guest, which +# is the only place a firewall exists for. +# +# Four things about the shape are deliberate: +# +# * **A second machine is the peer.** Nothing outside a guest can reach it +# except another guest: the test driver runs on the host and is not on the +# test VLAN, so a probe from the driver fails whether the port is open or +# not. The peer is a bare machine on the same VLAN with no module on it -- +# a peer running the module would be testing the machine against itself. +# * The configured port is **not** the module's default (9123, with 9988 as +# the default it must ignore). The interesting failure is an option that +# opens its own default whatever it is set to, and a check that configures +# 9988 cannot see it. +# * A listener is put on the unopened neighbour too, so "refused" means the +# firewall refused rather than nothing being there to connect to. Both +# listeners answer on loopback inside the guest first. +# * The address the peer probes is read out of the guest instead of assumed, +# because a check that guesses an interface name is testing the guess -- and +# the guest carries two addresses (the qemu NAT one and the test VLAN's), +# only one of which the peer can reach. +{ + testers, + hhstream, + testTools, + hhsModule, + python3, + curl, + # The port the module is told to open; one beside it that it is not; and the + # module's own default, which must not appear at all. + configuredPort ? 9123, + unopenedPort ? 9124, + defaultPort ? 9988, +}: +testers.runNixOSTest { + name = "hhstream-vm-stream-firewall"; + + # Two guests on a two-core machine is slower than one; cap it so a hang is a + # failure rather than a wedged builder. + globalTimeout = 1800; + + nodes.machine = { + # gpu-session.nix is what makes the compositor able to start at all; this + # check is not about the compositor, but `sessions.stream` lives under a + # module that boots one, and a machine kept from starting its own units is + # not the machine anyone runs. + imports = [ hhsModule ./lib/gpu-session.nix ]; + + services.hyprland-headless-stream = { + enable = true; + # The module declares `package` with no default (see README.dev.md), so + # the test has to supply the flake's build. + package = hhstream; + frameRate = 30; + outputs = [ + { + width = 1280; + height = 720; + refresh = 30; + } + ]; + # The fake capture, as in vm-basic: the compositor's own frames are not + # this check's subject, and a real `grim` here would only add minutes of + # software rendering to it. + captureCommand = [ + "${testTools}/bin/hhs-fake-capture" + "--width" + "1280" + "--height" + "720" + ]; + + sessions.stream = { + port = configuredPort; + # Not a loopback address, because the module refuses that pairing with + # `openFirewall` on: opening a port nothing can reach is a hole with no + # service behind it (see the assertion in nix/module.nix). + bind = "0.0.0.0"; + openFirewall = true; + }; + }; + + # Two listeners, each bound on every interface: the configured port and a + # neighbour the module was never told about. They exist so that a refusal + # out here is the firewall's doing rather than an empty port. + systemd.services.probe-configured = { + wantedBy = [ "multi-user.target" ]; + serviceConfig.ExecStart = "${python3}/bin/python3 -m http.server ${toString configuredPort}"; + }; + + systemd.services.probe-unopened = { + wantedBy = [ "multi-user.target" ]; + serviceConfig.ExecStart = "${python3}/bin/python3 -m http.server ${toString unopenedPort}"; + }; + + environment.systemPackages = [ + # `curl` is the loopback probe; python3 is the listeners themselves. + python3 + curl + ]; + + virtualisation = { + memorySize = 2048; + cores = 2; + diskSize = 4096; + graphics = false; + # The VLAN the peer is on. Without it the two machines share nothing but + # the driver's console, and "outside" stops existing. + vlans = [ 1 ]; + }; + }; + + # The peer: a bare machine on the same VLAN, no module, no compositor. Its + # whole job is to fail to reach one port and succeed on another. + nodes.peer = { + environment.systemPackages = [ curl ]; + + virtualisation = { + memorySize = 1024; + cores = 1; + vlans = [ 1 ]; + }; + }; + + testScript = '' + CONFIGURED = ${toString configuredPort} + UNOPENED = ${toString unopenedPort} + DEFAULT = ${toString defaultPort} + + + def dump(label): + print(f"########## diagnostics: {label} ##########") + for cmd in ( + "systemctl --no-pager -l status firewall.service", + "systemctl --no-pager -l status probe-configured.service probe-unopened.service", + "nixos-firewall-tool show", + "ip -4 -o addr show", + ): + status, out = machine.execute(cmd, timeout=60) + print(out) + print("########## end diagnostics ##########") + + + def peer_reaches(address, port, timeout=8): + """Ask the peer to open the port: out there, a firewall is a firewall.""" + status, out = peer.execute( + f"curl -sS -f -o /dev/null --max-time {timeout} http://{address}:{port}/", + timeout=timeout + 10, + ) + return status == 0 + + + start_all() + + # Every global address the machine has, waited for and then read off the + # machine rather than assumed: a bare test machine carries two (the qemu NAT + # one and the test VLAN's), and only one of them is a path the peer can + # take. + # + # The wait is not decoration. Addresses appear a moment after `start_all()` + # returns, and reading too early yields an empty list -- which reads as "the + # peer can reach nothing" rather than as "asked a second too soon". + machine.wait_until_succeeds( + "ip -4 -o addr show scope global | grep -q ' inet '", timeout=90 + ) + addresses = [] + for line in machine.succeed("ip -4 -o addr show scope global").splitlines(): + # "2: eth0 inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic ..." + fields = line.split() + if len(fields) >= 4 and fields[2] == "inet": + addresses.append(fields[3].split("/")[0]) + print(f"the machine's addresses: {addresses}") + assert addresses, "the machine reported no global address to probe" + + with subtest("a machine with a stream declared is still a machine that boots"): + try: + machine.wait_for_unit("hyprland-headless.service", timeout=120) + machine.wait_for_unit("hyprland-headless-stream.service", timeout=120) + except Exception: + dump("the module's own units did not come up") + raise + + with subtest("the running firewall accepts the configured port, and not the default"): + try: + machine.wait_for_unit("firewall.service") + rules = machine.succeed("nixos-firewall-tool show") + print(rules) + except Exception: + dump("the firewall did not come up") + raise + assert f"dpt:{CONFIGURED}" in rules, ( + f"the firewall does not accept {CONFIGURED}: the option did not reach " + f"the rule set" + ) + assert f"dpt:{DEFAULT}" not in rules, ( + f"the firewall accepts {DEFAULT}: the module opened its own default " + f"port instead of the configured one" + ) + + with subtest("both probe listeners are up and answer inside the guest"): + try: + machine.wait_for_unit("probe-configured.service", timeout=60) + machine.wait_for_unit("probe-unopened.service", timeout=60) + for port in (CONFIGURED, UNOPENED): + machine.succeed(f"curl -sS -f -o /dev/null http://127.0.0.1:{port}/") + except Exception: + dump("a probe listener did not come up") + raise + + with subtest("the peer reaches the configured port"): + reached = [a for a in addresses if peer_reaches(a, CONFIGURED)] + print(f"the peer reached {CONFIGURED} on: {reached}") + if not reached: + dump(f"the peer could not reach {CONFIGURED} on any of {addresses}") + raise AssertionError( + f"{CONFIGURED} is accepted in the rule set but the peer cannot reach " + f"it on any address the machine has ({addresses})" + ) + + with subtest("and is refused on the port the module never opened"): + # On the same addresses that just answered for the configured port, so + # the refusal is the firewall's doing and not a missing route. + for address in reached: + if peer_reaches(address, UNOPENED): + dump(f"the peer reached {UNOPENED} on {address}") + raise AssertionError( + f"the peer reached {UNOPENED} on {address}, a port the module was " + f"never told to open: the firewall is letting through more than " + f"the option names" + ) + ''; +} diff --git a/docs/plans/2026-10-04-session-stream-webapp.md b/docs/plans/2026-10-04-session-stream-webapp.md index 2796523..7be79cc 100644 --- a/docs/plans/2026-10-04-session-stream-webapp.md +++ b/docs/plans/2026-10-04-session-stream-webapp.md @@ -178,4 +178,18 @@ one at a time; no browser in a VM check. **Stage 1 is built** (`src/hhstream_live/`, `tests/live/`, 51 unit tests): the router, the pages, the phrase and the publisher, runnable standalone via `hhstream-live serve --stream PHRASE=OUTPUT`. The broker verbs and -`/run/hhstream/live.sock` are Stage 2. \ No newline at end of file +`/run/hhstream/live.sock` are Stage 2. + +**Stage 3 has started, out of order and on purpose** (2026-10-11). A machine +had to be able to *reach* a stream before the rest of the app is settled: the +operator asked for the port to be open on the deployment machine, where the app +runs by hand, and a hand-written firewall line in one machine's config is the +wrong place for a fact about this flake. So `sessions.stream { port, bind, +openFirewall }` and the single rule it stands for are in `nix/module.nix`, with +`checks/vm-stream-firewall.nix` asking the *running* firewall — and a peer +outside the guest — whether the configured port is the one that opened. + +What that does **not** do: no unit, no phrases, no `session stream +enable|disable|list|status`, and nothing in the module turns a stream on. The +app is still started by hand, and the option's whole job is to describe where a +stream will be reachable from. \ No newline at end of file diff --git a/nix/module.nix b/nix/module.nix index 2d7446b..80b31e9 100644 --- a/nix/module.nix +++ b/nix/module.nix @@ -47,6 +47,11 @@ let controlSocket = "${brokerRuntimeDir}/control.sock"; pointerSocket = "${brokerRuntimeDir}/pointer.sock"; + # Addresses that mean "this listener answers on loopback only". The spelling + # `localhost` is included because it is what a person writes; it is not a + # resolution the module performs. + loopbackBinds = [ "127.0.0.1" "::1" "localhost" ]; + socketDir = builtins.dirOf cfg.socketPath; # Hyprland refuses `--socket` unless it is paired with `--wayland-fd` (that @@ -620,6 +625,63 @@ in { }; }; + # The session-stream web app — `hhstream-live`, the router that serves a + # live view of one session per phrase, over one port + # (`docs/plans/2026-10-04-session-stream-webapp.md`). + # + # Only the *reach* of it is declared here so far: the port every stream + # shares, the address it will bind, and whether that port is opened in + # the firewall. The unit that runs the app, the phrases the broker keeps + # in its records and the `session stream enable` verbs are later stages + # of that plan — and deliberately so: a stream is switched on per + # session, never for all of them, so nothing here turns one on. + stream = { + port = lib.mkOption { + type = lib.types.port; + default = 9988; + description = '' + The single TCP port every stream shares. + + One listener rather than one per session is what makes the phrase + speakable: there is one address to give a person, one thing to + firewall and one thing to stop. It is also why this is a fixed + number with no range to fall back on — a stream at a port nobody + was told about is worse than a stream that is down, so a taken port + fails loudly instead of moving. + ''; + }; + + bind = lib.mkOption { + type = lib.types.str; + default = "127.0.0.1"; + description = '' + The address the web app will bind. + + Loopback by default, where the reachable path is an ssh tunnel + (`ssh -L 9988:127.0.0.1:9988`) and nothing else. Set it to an + address a peer can reach (`"0.0.0.0"`) when + {option}`sessions.stream.openFirewall` is set: opening the port for + a listener that is not on it is a hole with no service behind it, + which is why that pairing is refused rather than allowed. + ''; + }; + + openFirewall = lib.mkOption { + type = lib.types.bool; + default = false; + description = '' + Open {option}`sessions.stream.port` in the firewall. + + Off by default, and deliberately not implied by + {option}`sessions.enable`: a live view of this desktop is readable + by anyone who can reach the port, and the phrase in the URL is its + only credential — no TLS, no second factor + (`docs/plans/2026-10-04-…` §4). Opening it is an explicit act, and + it opens exactly one port. + ''; + }; + }; + stateDir = lib.mkOption { type = lib.types.str; default = "/var/lib/hhstream"; @@ -695,6 +757,21 @@ in { so without the broker it would bind a socket nothing connects to. ''; } + { + assertion = + !cfg.sessions.stream.openFirewall + || !(lib.elem cfg.sessions.stream.bind loopbackBinds); + message = '' + services.hyprland-headless-stream.sessions.stream.openFirewall is true + while sessions.stream.bind is "${cfg.sessions.stream.bind}" — a loopback + address. The port would be open in the firewall with nothing answering on + it from anywhere but this machine, which is a hole rather than a feature. + + Set sessions.stream.bind to an address a peer can reach ("0.0.0.0"), or + leave the firewall closed and reach the stream through a tunnel + (`ssh -L 9988:127.0.0.1:9988`). + ''; + } { assertion = !cfg.sessions.enable || lib.hasAttr cfg.sessions.user config.users.users; message = '' @@ -757,6 +834,17 @@ in { }) ]; + # The stream's one port, in the firewall. + # + # `lib.optional`, not a conditional block: the option is one line and its + # effect is one port in the rule set, so there is nothing here that could + # open a second one. Everything else about the app — the phrase per + # session, the capture, the enable/disable verbs — is later stages of the + # plan; this is the only part of the stream that a machine declares before + # any of that exists. + networking.firewall.allowedTCPPorts = + lib.optional cfg.sessions.stream.openFirewall cfg.sessions.stream.port; + users.users.${cfg.user} = { isSystemUser = true; group = cfg.group; From 4fefb448acaefeea4f392d915cd1e4c82be7d4cb Mon Sep 17 00:00:00 2001 From: Hermenix Date: Sun, 11 Oct 2026 02:31:52 +0000 Subject: [PATCH 8/8] docs(todo): the capture binary needs an absolute path under a unit Found on the deployment machine on 2026-10-11, the first time `hhstream-live` was run **as a unit** rather than from a login shell: a systemd unit's PATH has no system profile, so the capture command `grim` is not found, ten times, and the stream gives up. It answers on its port, serves the landing page and reports `"stopped": true, "failures": 10` while showing nothing -- the quiet half of the failure is what makes it worth a file. The module already solves this for `sessions.apps` by requiring an absolute store path; the stream's unit has to do the same (or set PATH, which is what made the deployment run work). Recorded in `docs/TODO/` with the two fixes, and named as a risk on stage 3 of the plan, which is the stage that turns this from an experiment into the only path. --- docs/TODO/README.md | 1 + docs/TODO/capture-binary-and-the-unit-path.md | 56 +++++++++++++++++++ .../plans/2026-10-04-session-stream-webapp.md | 2 +- 3 files changed, 58 insertions(+), 1 deletion(-) create mode 100644 docs/TODO/capture-binary-and-the-unit-path.md diff --git a/docs/TODO/README.md b/docs/TODO/README.md index e371bbe..4a57b10 100644 --- a/docs/TODO/README.md +++ b/docs/TODO/README.md @@ -14,6 +14,7 @@ Each file answers three questions and stops: | [scroll-has-no-compositor-proof](scroll-has-no-compositor-proof.md) | medium | works live; unproven in a gate | | [reconcile-while-containers-are-off](reconcile-while-containers-are-off.md) | medium | partly fixed: discovery now survives a restart; teardown still lives inside the feature | | [kill-can-leave-the-app-alive](kill-can-leave-the-app-alive.md) | medium | found by counting windows, not sessions | +| [capture-binary-and-the-unit-path](capture-binary-and-the-unit-path.md) | medium | found the first time the app was run as a unit: a unit's `PATH` has no `grim` | Closed since this folder was written: diff --git a/docs/TODO/capture-binary-and-the-unit-path.md b/docs/TODO/capture-binary-and-the-unit-path.md new file mode 100644 index 0000000..c2a0a76 --- /dev/null +++ b/docs/TODO/capture-binary-and-the-unit-path.md @@ -0,0 +1,56 @@ +# The capture command needs an absolute path under a unit + +## The issue + +`hhstream-live` runs its capture command (`grim -c -t jpeg -q 80 -o -`) +by name, from the `PATH` it inherits. Under **systemd** that path is +`/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin` — the system +profile is deliberately not on it — so the capture fails, ten times, and the +stream gives up: + +``` +WARNING hhstream.live: stream on hh-chromium-yhvpms: giving up after 10 +consecutive capture failures: capture command ['grim', '-c', '-t', 'jpeg', +'-q', '80', '-o', 'hh-chromium-yhvpms', '-'] could not be run: +[Errno 2] No such file or directory: 'grim' +``` + +Measured on 2026-10-11 on the deployment machine, the first time the app was +run **as a unit** rather than from a login shell. A login shell has +`/run/current-system/sw/bin` on its `PATH`, which is why every run before this +one — the stage-1 work, the standalone `serve --stream` runs, the whole test +suite — never saw it. + +What makes it worth a file rather than a line: the failure is quiet in the +worst way. The unit starts, the port opens, the landing page answers, and +`status.json` reports a stream that is simply `"stopped": true` with +`"failures": 10`. A viewer sees an empty frame area and nothing else says why. +Only the unit's log names the reason, one line above the giving-up entry. + +## Ideas + +1. **An absolute binary, as the module already does elsewhere** *(recommended)*. + `sessions.apps` requires an absolute store path for exactly this reason — + "a unit's default `PATH` does not include the system profile", which is why + the allowlist spells `foot` as `${pkgs.foot}/bin/foot`. The stream's unit + should not inherit `grim` by name either: pass + `${pkgs.grim}/bin/grim` as the capture command from the module (or set + `PATH` on the unit, which is what made it work on the deployment machine). + The absolute path is also the stronger control: it names one exact binary + instead of whatever `PATH` resolves to at the moment the unit starts. +2. **Resolve early and fail loudly.** Have the app resolve the binary itself at + start-up and refuse to serve with a named error (`capture_binary_missing`) + if it is not there. A unit that dies at start is a five-second diagnosis; a + unit that serves an empty frame is an afternoon. +3. **Both.** The module passes the absolute path *and* the app checks it once + at start-up: the first is the fix, the second is what catches the next + deployment shape nobody has thought of yet. + +## Why it matters + +Stage 3's unit is the point at which running under systemd stops being an +experiment and becomes the only path every deployment takes — and a stream +that answers, reports a healthy port, and shows nothing is a worse failure than +one that does not start. Whichever idea is taken, the check for it belongs in +the same stage as the unit: a `vm-*` check that starts the unit and reads one +frame would have caught this on the day it was written. \ No newline at end of file diff --git a/docs/plans/2026-10-04-session-stream-webapp.md b/docs/plans/2026-10-04-session-stream-webapp.md index 7be79cc..e423e67 100644 --- a/docs/plans/2026-10-04-session-stream-webapp.md +++ b/docs/plans/2026-10-04-session-stream-webapp.md @@ -156,7 +156,7 @@ capture without one. | 0 | Contract text: `docs/CONTRACT-sessions.md` → **sessions-v3** (the four verbs, `/run/hhstream/live.sock`, `hhstream-live`, the single port, the viewer-touch exception, schema 4, error names). | S | Freezing the wrong default. This plan is the review. | | 1 | `hhstream-live`: the HTTP server, the router, the phrase (wordlist + generator), one output, injected capture, **unit tests**. No broker. | M | A phone that will not paint `multipart/x-mixed-replace`; the index leaking phrases when made public. | | 2 | Broker + CLI: the record field, schema 4, the four verbs, teardown order, viewer heartbeat, integration test on `127.0.0.1`. | L | A second capture on re-enable; `disable` racing `output remove`; a heartbeat that keeps a session alive with zero viewers. | -| 3 | Module: the unit, `/run/hhstream/live.sock`, `port` 9988, `bind` loopback, `fps` 5/cap 10, `enable` false, `openFirewall` false, `publicIndex` false. No host firewall edit here. | M | A second `RuntimeDirectory` on `/run/hhstream` deleting `control.sock`. | +| 3 | Module: the unit, `/run/hhstream/live.sock`, `port` 9988, `bind` loopback, `fps` 5/cap 10, `enable` false, `openFirewall` false, `publicIndex` false. No host firewall edit here. | M | A second `RuntimeDirectory` on `/run/hhstream` deleting `control.sock`; and the capture binary — a unit's `PATH` has no `grim`, so the stream serves an empty frame area without saying why (`docs/TODO/capture-binary-and-the-unit-path.md`). | | 4 | Host pass: enable a real session, open the URL in the machine's own Chromium, confirm pixels change without a reload; write down what was on screen. | S | An empty container looking like a broken stream. Confirm the window is on `hh-` first. | Test layers are v1's: `tests/unit` for the HTTP/router/phrase with a fake