Merge origin/main into feat/keyboard-input: keep both the stream and the

keyboard sides

origin/main brought the session live-stream feature (stage 1): the
`hhstream_live` router, its phrase generator and wordlist, the
`sessions.stream.*` module options with their loopback-vs-openFirewall
assertion and the one firewall port, the `vm-stream-firewall` check, and
`tests/live/`. This branch adds keyboard input: the `key`/`type` ops, the
uploaded XKB keymap, `sessions.keyboard.{layout,model,variant,options}`
and the four `--xkb-*` flags on the driver unit, `vm-keyboard` and
`vm-keyboard-de`, and their docs.

The two features are independent, so every conflict wanted both sides.
Four files, resolved as follows.

checks/default.nix -- both the header list and the attribute set take all
three new checks: `vm-keyboard`, `vm-keyboard-de` (the same file with
`keyboardLayout = "de"`, because the layout reaches the keymap only
through the driver unit's argv) and `vm-stream-firewall`, which stays a
non-`sessionArgs` consumer with main's comment saying why. Neither side's
check list is reduced; the flake now exposes ten.

nix/package.nix -- one `pythonImportsCheck` list with both sides' entries:
`hhstream_pointer.keymap` and the vendored
`virtual_keyboard_unstable_v1` from here, `hhstream_live.server` from
main. The surrounding hunks were not in conflict, so the branch's
xkbcommon dependency and main's wordlist `postFixup` both survive, and
both ran in this merge's `checks.unit` build (`wordlist ok: 7772 words`).

nix/module.nix -- the sole conflict was textual: both sides open a new
sub-option block immediately after `sessions.pointer`. Both blocks are
kept, `keyboard` then `stream`, each with its own closing. Everything
else in the file merged cleanly, so `loopbackBinds`, the
openFirewall-on-loopback assertion and
`networking.firewall.allowedTCPPorts` are present alongside the driver
unit's `--xkb-layout`/`--xkb-model`/`--xkb-variant`/`--xkb-options`.

README.dev.md -- the `checks/` layout listing keeps both entries in the
file's existing order, `vm-keyboard.nix` then `vm-stream-firewall.nix`.

Verified: `nix flake check --no-build` evaluates every output green, and
`nix build .#checks.x86_64-linux.unit` passes -- 977 passed, 2 skipped,
0 failed, over both sides' suites. The VM checks were not run (they boot
a software-rendered compositor and must run one at a time).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PjpHyir6hxyjdw2o1RwQmn
This commit is contained in:
HermenixandClaude Opus 5 committed 2026-10-11 02:57:55 +00:00
commit 5f94833a76
26 files changed
+10544 -13

No files matched your search

+15
View File
@@ -0,0 +1,15 @@
"""The per-session live web stream: one port, a router, a phrase per session.
Stage 1 of `docs/plans/2026-10-04-session-stream-webapp.md`: the web application
itself -- the HTTP router, the viewer/snapshot/status pages, the MJPEG
publisher, and the phrase generator. No broker wiring yet; the broker's verbs
and the `/run/hhstream/live.sock` control channel are a separate stage.
The design in one line: a session's stream exists only once it is *enabled*,
its key is a spoken phrase, and every enabled session is a route under one
listener rather than a port of its own.
"""
from __future__ import annotations
__all__ = ["phrase", "publisher", "server", "pages"]
+198
View File
@@ -0,0 +1,198 @@
"""``hhstream-live`` -- run the session-stream web app.
Stage 1 runs it **standalone**: the streams are named on the command line as
``--stream <phrase>=<output>``, which is enough to watch a real container and to
drive the unit tests. Stage 2 adds the broker's control socket
(``/run/hhstream/live.sock``), at which point ``--stream`` becomes the way a
human debugs it rather than the way the broker configures it.
hhstream-live serve --stream four-words-and-a-tail=hh-demo
hhstream-live phrase
The capture command is one list of words; the output is aimed by inserting
``-o <output>`` before the trailing ``-``, which is grim's "write to stdout".
"""
from __future__ import annotations
import argparse
import logging
import signal
import sys
import threading
from . import phrase as phrase_mod
from .publisher import DEFAULT_FPS, FPS_CAP, StreamRegistry, StreamSpec
from .server import DEFAULT_HOST, DEFAULT_PORT, LiveServer
log = logging.getLogger("hhstream.live")
#: The session's own capture command, JPEG for the same reason the host config
#: spells it out: a 1280x720 JPEG q80 is ~46 ms and the point is a video, not a
#: slideshow. `-c` asks grim to include the cursor.
DEFAULT_CAPTURE = ["grim", "-c", "-t", "jpeg", "-q", "80", "-"]
def capture_argv(base: list[str], output: str) -> list[str]:
"""Aim the base capture command at one output.
``grim`` writes to stdout when its last argument is ``-``; the output
selector goes just before it so the shape holds for any tool that keeps that
convention, and a base without a trailing ``-`` simply gets ``-o`` appended.
"""
base = list(base)
if base and base[-1] == "-":
return base[:-1] + ["-o", output, "-"]
return base + ["-o", output]
def parse_stream(arg: str) -> tuple[str, str]:
"""``<phrase>=<output>`` -> the pair, each side checked non-empty."""
phrase, sep, output = arg.partition("=")
if not sep or not phrase.strip() or not output.strip():
raise argparse.ArgumentTypeError(f"expected PHRASE=OUTPUT, got {arg!r}")
try:
return phrase_mod.validate(phrase), output.strip()
except ValueError as exc:
raise argparse.ArgumentTypeError(str(exc)) from exc
def _add_log_level(parser: argparse.ArgumentParser, default: str) -> None:
"""Put `--log-level` on a parser.
It is a global option, but it has to exist on the subparsers too: with no
subcommand the whole line is parsed as `serve ...`, and `serve` is then the
parser that sees it. `SUPPRESS` as the subparser default is what stops
`hhstream-live --log-level debug serve` from being reset to `info` -- a
subparser argument that defaults to `SUPPRESS` and is not supplied leaves
the namespace alone.
"""
parser.add_argument("--log-level", default=default,
choices=["debug", "info", "warning", "error"])
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(prog="hhstream-live", description=__doc__)
_add_log_level(parser, "info")
subs = parser.add_subparsers(dest="command")
serve = subs.add_parser("serve", help="run the web app (default)")
_add_log_level(serve, argparse.SUPPRESS)
serve.add_argument("--port", type=int, default=DEFAULT_PORT,
help=f"the one port every stream shares (default {DEFAULT_PORT})")
serve.add_argument("--bind", default=DEFAULT_HOST,
help=f"address to bind (default {DEFAULT_HOST}, loopback)")
serve.add_argument("--fps", type=int, default=DEFAULT_FPS,
help=f"capture rate for streams that do not set one "
f"(default {DEFAULT_FPS}, cap {FPS_CAP})")
serve.add_argument("--stream", action="append", default=[], type=parse_stream,
metavar="PHRASE=OUTPUT",
help="enable a stream on this output; repeatable")
serve.add_argument("--capture-command", action="append", default=None,
metavar="WORD",
help="capture argv word; repeat (a repeated flag is the "
"only shape that survives words starting with '-')")
gen = subs.add_parser("phrase", help="print a fresh phrase and exit")
_add_log_level(gen, argparse.SUPPRESS)
gen.add_argument("--count", type=int, default=1)
return parser
#: The subcommands, named here because the default one is chosen before the
#: parser runs. See `parse_args`.
COMMANDS = ("serve", "phrase")
def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
"""Parse the command line, defaulting to `serve`.
`serve` is the default, and its options live on its own subparser, so the
word has to be *inserted* rather than filled in afterwards: a namespace from
the top-level parser has no `capture_command`, no `port` and no `stream`, and
`cmd_serve` would die on the first attribute it read. Inserting it has to
happen before parsing, too -- `hhstream-live --port 9999` would otherwise be
rejected for an unknown `--port` before anything could be defaulted.
`--help` is left alone so that a bare `hhstream-live --help` still shows the
program's own help rather than `serve`'s.
"""
raw = list(sys.argv[1:] if argv is None else argv)
if not any(word in COMMANDS or word in ("-h", "--help") for word in raw):
raw = ["serve", *raw]
return build_parser().parse_args(raw)
def make_registry(streams: list[tuple[str, str]], capture_base: list[str],
fps: int) -> StreamRegistry:
from hhstream.capture import CommandCapture
registry = StreamRegistry()
for phrase, output in streams:
spec = StreamSpec(phrase=phrase, output=output, fps=fps, label=output)
registry.add(spec, CommandCapture(capture_argv(capture_base, output)))
return registry
def cmd_serve(args: argparse.Namespace) -> int:
capture_base = args.capture_command or DEFAULT_CAPTURE
registry = make_registry(args.stream, capture_base, args.fps)
server = LiveServer(registry, host=args.bind, port=args.port)
# A service is stopped with SIGTERM, not with Ctrl-C: without a handler the
# default action kills the process outright and `server.stop` never runs, so
# the listener and the capture threads go down with no ordering at all.
# Installed *before* the listener exists, so there is no window in which a
# supervisor's very first signal is the fatal one.
stopping = threading.Event()
previous = None
def _request_stop(signum, frame) -> None: # noqa: ARG001 - handler signature
stopping.set()
try:
previous = signal.signal(signal.SIGTERM, _request_stop)
except ValueError:
# Not the main thread: an embedder's problem to solve, not ours.
log.debug("SIGTERM handler not installed (not the main thread)")
try:
server.start() # raises OSError loudly if the port is taken
# Printed after `start`, because a requested port of 0 only has an answer
# once it is bound. `describe_bind` names the real bind, so `0.0.0.0` is
# not reported as loopback just because that is the address you click.
print(server.describe_bind(), file=sys.stderr)
print(f"landing page: http://{server.address}:{server.port}/", file=sys.stderr)
if not args.stream:
print("no --stream given; only the landing page will answer",
file=sys.stderr)
for phrase in registry.phrases():
print(server.url_for(phrase))
sys.stdout.flush()
stopping.wait()
print("stopping", file=sys.stderr)
except KeyboardInterrupt:
print("stopping", file=sys.stderr)
finally:
if previous is not None:
signal.signal(signal.SIGTERM, previous)
server.stop()
return 0
def main(argv: list[str] | None = None) -> int:
args = parse_args(argv)
logging.basicConfig(
level=getattr(logging, (getattr(args, "log_level", None) or "info").upper()),
format="%(asctime)s %(levelname)s %(name)s: %(message)s",
)
if args.command == "phrase":
for _ in range(args.count):
print(phrase_mod.generate())
return 0
return cmd_serve(args)
if __name__ == "__main__":
sys.exit(main())
+115
View File
@@ -0,0 +1,115 @@
"""The pages the router serves: the landing form and the per-session viewer.
Everything here is one self-contained HTML string. No CDN, no framework, no
build step -- "webapp" means a router and legible pages, not a toolchain. The
viewer page is the one that matters: it is an ``<img>`` pointed at the MJPEG
route, because a top-level ``multipart/x-mixed-replace`` document is unreliable
on phones while an image tag is not.
The landing page is a **form, never a list**. The phrase is the password to a
stream and is known only by whoever enabled it, so the root page must not
enumerate sessions or leak phrases; it just hands the typed phrase to
``/<phrase>/``.
"""
from __future__ import annotations
import html
from functools import lru_cache
_STYLE = """
:root{color-scheme:dark}
*{box-sizing:border-box}
body{margin:0;background:#0b0d12;color:#e6e8ee;font:15px/1.5 ui-sans-serif,system-ui,sans-serif}
main{max-width:960px;margin:0 auto;padding:24px}
h1{font-size:20px;margin:0 0 4px}
a{color:#7cc4ff}
.muted{color:#8b93a7}
.card{background:#141821;border:1px solid #232838;border-radius:12px;padding:20px}
img{width:100%;height:auto;display:block;border-radius:8px;background:#000}
input{width:100%;padding:12px 14px;margin:10px 0 14px;border-radius:8px;
border:1px solid #2c3346;background:#0e121b;color:inherit;font:inherit}
button{padding:11px 16px;border:0;border-radius:8px;background:#3b82f6;color:#fff;
font:inherit;cursor:pointer}
button:hover{background:#2f6fd6}
.badge{display:inline-flex;align-items:center;gap:6px;font-size:13px;color:#8b93a7}
.dot{width:8px;height:8px;border-radius:50%;background:#3ddc84;box-shadow:0 0 8px #3ddc84}
.row{display:flex;align-items:center;justify-content:space-between;gap:12px;margin-bottom:12px}
.tools{display:flex;gap:14px;font-size:13px;margin-top:12px}
"""
def _esc(text: str) -> str:
return html.escape(text, quote=True)
@lru_cache(maxsize=1)
def landing_page(app_name: str = "hhstream live") -> str:
"""The root page: a box to type a phrase into. No session list, ever."""
return f"""<!doctype html>
<html lang="en"><head>
<meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
<title>{_esc(app_name)}</title><style>{_STYLE}</style>
</head><body><main>
<h1>{_esc(app_name)}</h1>
<p class="muted">A live view is opened with the phrase printed when its session
was enabled.</p>
<div class="card">
<label for="p">Session phrase</label>
<input id="p" autocomplete="off" autocapitalize="off" spellcheck="false"
placeholder="four-words-and-a-tail">
<button id="go">Open stream</button>
</div>
<script>
const input = document.getElementById('p');
function open() {{
const phrase = input.value.trim().toLowerCase();
if (phrase) location.href = '/' + encodeURIComponent(phrase) + '/';
}}
document.getElementById('go').addEventListener('click', open);
input.addEventListener('keydown', e => {{ if (e.key === 'Enter') open(); }});
input.focus();
</script>
</main></body></html>
"""
def viewer_page(phrase: str, label: str, output: str, fps: int) -> str:
"""The page you watch: an `<img>` on the stream, plus a small live readout."""
base = f"/{phrase}"
return f"""<!doctype html>
<html lang="en"><head>
<meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
<title>{_esc(label)} — live</title><style>{_STYLE}</style>
</head><body><main>
<div class="row">
<div>
<h1>{_esc(label)}</h1>
<div class="muted">{_esc(output)} · {fps} fps</div>
</div>
<span class="badge"><span class="dot"></span><span id="v">live</span></span>
</div>
<div class="card">
<img id="stream" src="{base}/stream.mjpg" alt="live view of {_esc(label)}">
</div>
<div class="tools">
<a href="{base}/frame.jpg">snapshot</a>
<a href="{base}/status.json">status</a>
<a href="/">another phrase</a>
</div>
<script>
const out = document.getElementById('v');
async function tick() {{
try {{
const s = await (await fetch('{base}/status.json', {{cache:'no-store'}})).json();
out.textContent = s.viewers + (s.viewers === 1 ? ' viewer' : ' viewers') +
(s.last_frame_age_s == null ? '' : ' · ' + s.last_frame_age_s.toFixed(1) + 's ago');
}} catch (e) {{ out.textContent = 'offline'; }}
}}
tick(); setInterval(tick, 2000);
</script>
</main></body></html>
"""
__all__ = ["landing_page", "viewer_page"]
+123
View File
@@ -0,0 +1,123 @@
"""The phrase: a stream's only credential, shaped to be said out loud.
A phrase is ``word-word-word-word-xxxxx``: four words off the EFF large
wordlist and a five-character base-36 tail. That is ``4*log2(7772) +
5*log2(36)`` ~= **77.5 bits** of entropy, against a word set small enough to be
a file in the repository and a tail short enough to type on a phone.
Why words and not the ``secrets.token_urlsafe`` blob the first plan proposed:
the phrase is read out -- "open amber-otter-lantern-harbor-7k2qp" -- and a
base64 string is not. The job is identical (it *is* the password to the
stream); only the shape is chosen for a human.
The list is vendored (`wordlist.txt`, 7776 lines) so a stream never needs the
network to start. Four of those lines are hyphenated (`drop-down`, `felt-tip`,
`t-shirt`, `yo-yo`) and are dropped on load: a hyphen is the phrase's own
separator, so a phrase built from one would be six tokens and a listener could
not hear where the tail starts. 7772 words costs 0.003 bits.
`generate` uses `SystemRandom`/`secrets`, so the phrase is not reproducible
from the process seed.
"""
from __future__ import annotations
import math
import random
import re
import secrets
from pathlib import Path
#: Four words plus the tail. The plan's answer to "how many"; four is
#: `4 * log2(7772)` ~= 51.7 bits on its own, and the tail pushes it past 77.
WORD_COUNT = 4
SUFFIX_LEN = 5
#: Lowercase letters and digits: what a person can retype without a shift key,
#: and 36 symbols is still `5 * log2(36)` ~= 25.8 bits.
SUFFIX_ALPHABET = "abcdefghijklmnopqrstuvwxyz0123456789"
#: The smallest token a word may be. The wordlist's own floor is 3, so this
#: rejects nothing `generate` can produce and everything an operator might type
#: as a stand-in ("a-b-c-d-12345").
MIN_WORD_LEN = 3
#: A phrase is exactly what `generate` makes: `WORD_COUNT` alphabetic words of
#: at least `MIN_WORD_LEN` characters, then a `SUFFIX_LEN` tail of `[a-z0-9]`.
#: An operator-supplied `--phrase` is a password for a LAN-facing stream, so it
#: is held to the generated shape rather than to a looser "looks like a phrase"
#: pattern -- a looser one accepts `a-b-c`, and nothing stops an operator from
#: typing that.
_SHAPE = re.compile(
rf"^[a-z]{{{MIN_WORD_LEN},}}(?:-[a-z]{{{MIN_WORD_LEN},}}){{{WORD_COUNT - 1}}}"
rf"-[a-z0-9]{{{SUFFIX_LEN}}}$"
)
#: Only the words whose letters are the whole line; see the module docstring.
_WORD_SHAPE = re.compile(r"^[a-z]+$")
MAX_LENGTH = 96
_WORDLIST = Path(__file__).with_name("wordlist.txt")
_cache: tuple[str, ...] | None = None
def words() -> tuple[str, ...]:
"""The usable wordlist, read once and remembered.
Only plain `[a-z]+` lines survive: the EFF list contains four hyphenated
entries and the hyphen is the phrase's separator, so one of them would make
a six-token phrase nobody could read out correctly.
Raises `RuntimeError` on an empty file rather than returning nothing: a
generator that picks from an empty list would raise a confusing `IndexError`
later, and the empty file is a packaging bug worth naming here.
"""
global _cache
if _cache is None:
text = _WORDLIST.read_text(encoding="utf-8")
stripped = (line.strip() for line in text.splitlines())
loaded = tuple(word for word in stripped if _WORD_SHAPE.match(word))
if not loaded:
raise RuntimeError(f"wordlist {_WORDLIST} has no usable words")
_cache = loaded
return _cache
def entropy_bits(word_count: int = WORD_COUNT, suffix_len: int = SUFFIX_LEN) -> float:
"""How many bits a phrase of this shape carries, for the record."""
return word_count * math.log2(len(words())) + suffix_len * math.log2(len(SUFFIX_ALPHABET))
def generate(*, word_count: int = WORD_COUNT, suffix_len: int = SUFFIX_LEN) -> str:
"""A fresh phrase, drawn from the system entropy pool."""
if word_count < 1 or suffix_len < 0:
raise ValueError("word_count must be >= 1 and suffix_len >= 0")
rng = random.SystemRandom()
chosen = [rng.choice(words()) for _ in range(word_count)]
tail = "".join(secrets.choice(SUFFIX_ALPHABET) for _ in range(suffix_len))
return "-".join([*chosen, tail]) if tail else "-".join(chosen)
def normalize(phrase: str) -> str:
"""Fold a phrase to the one form the registry and the router compare on.
Lowercase and trimmed, because a phrase is typed by hand; the router also
accepts it without a trailing slash. Nothing else is folded -- a phrase with
an internal space is simply invalid, not silently rewritten.
"""
return phrase.strip().lower()
def validate(phrase: str) -> str:
"""Return the normalized phrase, or raise `ValueError` naming the shape."""
normalized = normalize(phrase)
if len(normalized) > MAX_LENGTH:
raise ValueError(f"phrase is {len(normalized)} characters; the maximum is {MAX_LENGTH}")
if not _SHAPE.match(normalized):
raise ValueError(
f"phrase must be {WORD_COUNT} lowercase words of at least "
f"{MIN_WORD_LEN} letters plus a {SUFFIX_LEN}-character [a-z0-9] "
f"tail (e.g. amber-otter-lantern-harbor-7k2qp), not {phrase!r}"
)
return normalized
__all__ = ["WORD_COUNT", "SUFFIX_LEN", "MIN_WORD_LEN", "generate", "normalize",
"validate", "entropy_bits", "words", "MAX_LENGTH"]
+277
View File
@@ -0,0 +1,277 @@
"""The publisher: one capture thread and one frame slot per enabled session.
This is the same shape as the frame server's inner loop (`hhstream/server.py`),
scaled from one output to many and given two rules the network side needs:
* **one slot, no queue.** A viewer reads the most recent frame when it is ready
for another one; a slow viewer misses frames instead of forcing a backlog, so
memory does not grow with viewers and one stalled phone cannot delay another.
* **no capture unless someone is watching.** The capture thread sleeps while a
stream has zero viewers. That is what lets a stream be *enabled* without a
`grim` running: the port is open, the route exists, and the CPU is idle until
a request arrives. It is also why the broker can safely treat a connected
viewer as activity and an enabled-but-unwatched stream as nothing at all.
A `Stream` never touches the broker or the HTTP layer; it is a capture loop with
a frame slot and a viewer count, and the server hands it viewers.
"""
from __future__ import annotations
import logging
import threading
import time
from dataclasses import dataclass
from typing import Any, Callable, Protocol
log = logging.getLogger("hhstream.live")
#: The plan's cap. 5 fps default is "one frame per ~200 ms plus a ~46 ms grim",
#: chosen from the measured JPEG cost, not a guess; 10 is the ceiling because
#: the box is also running the compositor and the main publisher.
DEFAULT_FPS = 5
FPS_CAP = 10
#: After this many *consecutive* capture failures a stream gives up and closes
#: its viewers. `grim` against a removed output ("unknown output") fails every
#: time, and retrying forever would be a busy loop on a dead session -- which is
#: as true after a thousand good frames as it is from a cold start, so the count
#: is of failures in a row, not of failures-before-the-first-frame. A successful
#: grab resets it; giving up does not reset, because a dead output stays dead.
FAILURE_LIMIT = 10
class Capture(Protocol):
def grab(self) -> bytes: # pragma: no cover - structural
...
@dataclass(frozen=True)
class StreamSpec:
"""What to publish, and under which phrase.
`session` is the broker's session id (``None`` in a standalone run); `label`
is what the viewer page shows. `output` is the compositor output the capture
is aimed at (``hh-<session-id>``), kept here so the page and the status
payload can name it without re-deriving it.
"""
phrase: str
output: str
fps: int = DEFAULT_FPS
session: str | None = None
label: str = ""
def public(self) -> dict[str, Any]:
return {
"phrase": self.phrase,
"output": self.output,
"fps": self.fps,
"session": self.session,
"label": self.label or self.session or self.output,
}
class Stream:
"""One output's capture loop, frame slot and viewer count."""
def __init__(self, spec: StreamSpec, capture: Capture,
clock: Callable[[], float] = time.monotonic) -> None:
if isinstance(spec.fps, bool) or not isinstance(spec.fps, int) or spec.fps <= 0:
raise ValueError(f"fps must be a positive int, got {spec.fps!r}")
if spec.fps > FPS_CAP:
raise ValueError(f"fps {spec.fps} exceeds the cap of {FPS_CAP}")
self.spec = spec
self.capture = capture
self._clock = clock
self._cond = threading.Condition()
self._frame: bytes | None = None
self._seq = 0
self._viewers = 0
self._captured = 0
self._failures = 0
self._consecutive_failures = 0
self._last_frame_at: float | None = None
self._stopped = False
self._thread: threading.Thread | None = None
# ---- lifecycle ---------------------------------------------------
def start(self) -> None:
# Named for the output, never the phrase: faulthandler, the default
# `threading.excepthook` and any profiler that samples the process all
# print thread names, and the phrase is this stream's only credential.
self._thread = threading.Thread(
target=self._run, name=f"hhstream-live-{self.spec.output}", daemon=True
)
self._thread.start()
def stop(self) -> None:
with self._cond:
self._stopped = True
self._cond.notify_all()
if self._thread is not None:
self._thread.join(timeout=5)
def _sleep(self, seconds: float) -> bool:
"""Wait `seconds`, or until stopped. True means 'stop now'."""
with self._cond:
return self._cond.wait_for(lambda: self._stopped, timeout=seconds)
def _run(self) -> None:
interval = 1.0 / self.spec.fps
while True:
# Idle at zero viewers: this is the "not on for all sessions at all
# times" rule made concrete -- an enabled stream costs nothing until
# a request arrives.
with self._cond:
while not self._stopped and self._viewers == 0:
self._cond.wait(0.5)
if self._stopped:
return
try:
frame = self.capture.grab()
except Exception as exc: # noqa: BLE001 - any backend failure is a skip
with self._cond:
self._failures += 1
self._consecutive_failures += 1
give_up = self._consecutive_failures >= FAILURE_LIMIT
if give_up:
# Sticky on purpose: a stream that reached the cap is
# not re-armed by `add_viewer`, because the case this
# guards -- an output that was removed -- does not come
# back. Re-enabling the session is how it comes back.
self._stopped = True
self._cond.notify_all()
if give_up:
# By output, not by phrase: a log line is the easiest place
# for a credential to escape to.
log.warning("stream on %s: giving up after %d consecutive "
"capture failures: %s",
self.spec.output, FAILURE_LIMIT, exc)
return
log.debug("stream on %s: capture failed: %s", self.spec.output, exc)
if self._sleep(0.5):
return
continue
with self._cond:
self._frame = frame
self._seq += 1
self._captured += 1
self._consecutive_failures = 0
self._last_frame_at = self._clock()
self._cond.notify_all()
if self._sleep(interval):
return
# ---- viewers -----------------------------------------------------
def add_viewer(self) -> int:
with self._cond:
self._viewers += 1
self._cond.notify_all()
return self._viewers
def remove_viewer(self) -> int:
with self._cond:
self._viewers = max(0, self._viewers - 1)
self._cond.notify_all()
return self._viewers
@property
def viewers(self) -> int:
with self._cond:
return self._viewers
def wait_frame(self, last_seq: int, timeout: float = 2.0) -> tuple[bytes | None, int]:
"""Block for a frame newer than `last_seq`.
Returns `(frame, seq)`; `(None, last_seq)` on timeout or after the
stream stopped with no new frame, which tells a viewer loop to re-check
whether it should keep going.
"""
with self._cond:
self._cond.wait_for(
lambda: self._seq > last_seq or self._stopped, timeout=timeout
)
if self._seq > last_seq:
return self._frame, self._seq
return None, last_seq
@property
def stopped(self) -> bool:
with self._cond:
return self._stopped
def snapshot(self) -> bytes | None:
with self._cond:
return self._frame
# ---- reporting ---------------------------------------------------
def status(self) -> dict[str, Any]:
with self._cond:
age = None if self._last_frame_at is None else max(0.0, self._clock() - self._last_frame_at)
return {
"session": self.spec.session,
"label": self.spec.label or self.spec.session or self.spec.output,
"output": self.spec.output,
"fps": self.spec.fps,
"viewers": self._viewers,
"captured": self._captured,
"failures": self._failures,
"last_frame_age_s": age,
"stopped": self._stopped,
}
class StreamRegistry:
"""The router's table: phrase -> `Stream`, compared case-insensitively.
A `Stream` is added already started; removing it stops the capture thread.
This is the whole of what "enabled" means on the publisher side -- there is
no disabled stream held in reserve, so an unknown phrase and a never-enabled
session are the same 404.
"""
def __init__(self) -> None:
self._streams: dict[str, Stream] = {}
def add(self, spec: StreamSpec, capture: Capture, *, start: bool = True) -> Stream:
from . import phrase as phrase_mod
key = phrase_mod.validate(spec.phrase)
if key in self._streams:
raise ValueError(f"phrase {key!r} is already enabled")
stream = Stream(spec, capture)
self._streams[key] = stream
if start:
stream.start()
return stream
def remove(self, phrase: str) -> Stream | None:
from . import phrase as phrase_mod
stream = self._streams.pop(phrase_mod.normalize(phrase), None)
if stream is not None:
stream.stop()
return stream
def get(self, phrase: str) -> Stream | None:
from . import phrase as phrase_mod
return self._streams.get(phrase_mod.normalize(phrase))
def phrases(self) -> list[str]:
return sorted(self._streams)
def all(self) -> list[Stream]:
return [self._streams[key] for key in sorted(self._streams)]
def stop_all(self) -> None:
for stream in self.all():
stream.stop()
__all__ = ["StreamSpec", "Stream", "StreamRegistry", "Capture", "DEFAULT_FPS",
"FPS_CAP", "FAILURE_LIMIT"]
+357
View File
@@ -0,0 +1,357 @@
"""The web app: one HTTP listener, a router, a page per enabled session.
One port (default 9988) serves every stream. The routes are keyed by *phrase*,
which is the only credential:
GET / the landing form (never a list of sessions)
GET /<phrase>/ the viewer page
GET /<phrase>/stream.mjpg multipart/x-mixed-replace, JPEG parts
GET /<phrase>/frame.jpg the most recent single frame
GET /<phrase>/status.json {session, output, fps, viewers, ...}
Anything else is `404` with an **empty** body, and a wrong phrase is never
echoed back. Unknown phrases are rate-limited per client so the phrase is not
enumerable at line speed.
Deliberately the standard library: `http.server` plus the framing rules this
module writes itself. A phone's `<img>` is the only client that must work, and
that needs `multipart/x-mixed-replace` with `Content-Type: image/jpeg` parts --
nothing a dependency would make easier.
"""
from __future__ import annotations
import logging
import socket
import threading
import time
from collections import deque
from http import HTTPStatus
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from urllib.parse import unquote, urlsplit
from . import pages
from .publisher import Stream, StreamRegistry
log = logging.getLogger("hhstream.live")
#: The multipart separator. Fixed and simple; it never appears in a JPEG.
BOUNDARY = "hhstreamboundary"
#: How long a viewer loop waits for a new frame before re-checking liveness.
FRAME_WAIT_S = 2.0
DEFAULT_PORT = 9988
DEFAULT_HOST = "127.0.0.1"
#: Wrong-phrase lookups allowed per client per window before 429.
RATE_LIMIT = 30
RATE_WINDOW_S = 60.0
#: How long a connection may stay silent before it is dropped. It covers the
#: request line and the headers only: a client that connects and says nothing
#: would otherwise hold a handler thread in `readline` for ever, and on the
#: MJPEG route it would hold a viewer slot -- which keeps `grim` running.
HEADER_TIMEOUT_S = 15.0
class RateLimiter:
"""A small per-client counter for failed phrase lookups.
A phrase carries ~77 bits, so guessing is already hopeless; this is here so
that a LAN attacker cannot even *try* at line speed, and so the attempt is
visible in one place. Only misses are counted.
The table holds only clients that missed inside the current window: the key
is a client address, which on a LAN is the attacker's to choose, so a table
that only ever grew would be a slow memory leak driven from the network.
"""
def __init__(self, limit: int = RATE_LIMIT, window: float = RATE_WINDOW_S,
clock=time.monotonic) -> None:
self.limit = limit
self.window = window
self._clock = clock
self._hits: dict[str, deque[float]] = {}
self._lock = threading.Lock()
def allow(self, key: str) -> bool:
now = self._clock()
cutoff = now - self.window
with self._lock:
# Prune every key, not just this one. Pruning only `key` would leave
# a client that missed once and never came back in the table for the
# life of the process; the sweep is over the clients seen in the
# last window, which is what the limit already bounds.
for seen, hits in list(self._hits.items()):
while hits and hits[0] < cutoff:
hits.popleft()
if not hits:
del self._hits[seen]
hits = self._hits.setdefault(key, deque())
if len(hits) >= self.limit:
return False
hits.append(now)
return True
@property
def tracked(self) -> int:
"""How many clients the table is holding. For the leak test and status."""
with self._lock:
return len(self._hits)
def reset(self) -> None:
with self._lock:
self._hits.clear()
class _Server(ThreadingHTTPServer):
daemon_threads = True
allow_reuse_address = True
def __init__(self, address, handler, registry: StreamRegistry,
limiter: RateLimiter) -> None:
self.registry = registry
self.limiter = limiter
super().__init__(address, handler)
class _Handler(BaseHTTPRequestHandler):
protocol_version = "HTTP/1.1"
server_version = "hhstream-live"
sys_version = ""
#: `StreamRequestHandler.setup` applies this to the socket. Without it a
#: client that opens a connection and sends nothing pins a handler thread.
timeout = HEADER_TIMEOUT_S
# ---- plumbing ----------------------------------------------------
@property
def registry(self) -> StreamRegistry:
return self.server.registry # type: ignore[attr-defined]
def setup(self) -> None:
super().setup()
# Keepalive is for the other half of the same problem: a viewer whose
# network black-holes never closes, so `sendall` blocks and the viewer
# count never drops -- which keeps `grim` capturing for nobody. The
# kernel's probes end that connection instead.
try:
self.connection.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
except OSError as exc: # a platform without it is not worth failing over
log.debug("could not enable TCP keepalive: %s", exc)
def log_message(self, format: str, *args) -> None: # noqa: A002 - base signature
"""Log that something happened, never *what was asked for*.
The base class formats `self.requestline` into almost every message it
emits, and the request line contains the phrase -- the stream's only
credential. So the format string and its arguments are dropped here and
the method is logged on its own; `log_request` adds the status.
"""
log.debug("%s %s", self.address_string(), self.command or "-")
def log_request(self, code="-", size="-") -> None:
if isinstance(code, HTTPStatus):
code = code.value
log.debug("%s %s -> %s", self.address_string(), self.command or "-", code)
def _send(self, status: HTTPStatus, body: bytes, content_type: str,
headers: dict[str, str] | None = None) -> None:
self.send_response(status)
self.send_header("Content-Type", content_type)
self.send_header("Content-Length", str(len(body)))
self.send_header("Cache-Control", "no-store")
self.send_header("X-Content-Type-Options", "nosniff")
for name, value in (headers or {}).items():
self.send_header(name, value)
self.end_headers()
if body:
self.wfile.write(body)
def _empty(self, status: HTTPStatus) -> None:
self._send(status, b"", "text/plain; charset=utf-8")
# ---- routes ------------------------------------------------------
def do_GET(self) -> None: # noqa: N802 - the name http.server dispatches to
path = unquote(urlsplit(self.path).path)
try:
self._route(path)
except (BrokenPipeError, ConnectionResetError):
pass # the viewer went away mid-write; that is not an error
except OSError as exc:
# Not the path: it carries the phrase.
log.debug("client error from %s: %s", self.address_string(), exc)
def _route(self, path: str) -> None:
if path in ("", "/"):
body = pages.landing_page().encode("utf-8")
self._send(HTTPStatus.OK, body, "text/html; charset=utf-8")
return
if path == "/healthz":
self._send(HTTPStatus.OK, b"ok\n", "text/plain; charset=utf-8")
return
if not path.startswith("/"):
self._empty(HTTPStatus.NOT_FOUND)
return
head, _, rest = path.lstrip("/").partition("/")
stream = self.registry.get(head)
if stream is None:
# A miss. `head` is user input and is never reflected anywhere.
if not self.server.limiter.allow(self.client_address[0]): # type: ignore[attr-defined]
self._empty(HTTPStatus.TOO_MANY_REQUESTS)
else:
self._empty(HTTPStatus.NOT_FOUND)
return
if rest in ("", "index.html"):
self._viewer(stream)
elif rest == "stream.mjpg":
self._mjpeg(stream)
elif rest == "frame.jpg":
self._snapshot(stream)
elif rest == "status.json":
self._status(stream)
else:
self._empty(HTTPStatus.NOT_FOUND)
# ---- handlers ----------------------------------------------------
def _viewer(self, stream: Stream) -> None:
spec = stream.spec
label = spec.label or spec.session or spec.output
body = pages.viewer_page(spec.phrase, label, spec.output, spec.fps).encode("utf-8")
self._send(HTTPStatus.OK, body, "text/html; charset=utf-8")
def _status(self, stream: Stream) -> None:
import json
body = json.dumps(stream.status()).encode("utf-8")
self._send(HTTPStatus.OK, body, "application/json; charset=utf-8")
def _snapshot(self, stream: Stream) -> None:
frame = stream.snapshot()
if frame is None:
# No frame yet: nobody has watched this stream, so nothing has been
# captured. 503 is the honest answer, not a blank image.
self._empty(HTTPStatus.SERVICE_UNAVAILABLE)
return
self._send(HTTPStatus.OK, frame, "image/jpeg")
def _mjpeg(self, stream: Stream) -> None:
self.send_response(HTTPStatus.OK)
self.send_header("Content-Type", f"multipart/x-mixed-replace; boundary={BOUNDARY}")
self.send_header("Cache-Control", "no-store")
self.send_header("X-Content-Type-Options", "nosniff")
# The connection *is* the stream: it has no `Content-Length` and is not
# chunked, so "the response ended" can only mean "the connection ended".
# HTTP/1.1 keeps a connection alive by default, and an intermediary that
# believed that would be entitled to buffer this response until an EOF
# that never comes. Saying so has to happen before `end_headers`, or it
# is not a header at all -- `send_header` also sets `close_connection`.
self.send_header("Connection", "close")
self.end_headers()
# The header timeout has done its job; from here the handler waits on
# frames, not on the client, and a quiet minute is normal.
self.connection.settimeout(None)
stream.add_viewer()
last_seq = 0
try:
while True:
frame, seq = stream.wait_frame(last_seq, timeout=FRAME_WAIT_S)
if frame is None:
if stream.stopped:
return
continue # nothing new yet; loop and re-check liveness
last_seq = seq
self.wfile.write(self._part(frame))
# unreachable; the loop exits by return/exception
finally:
stream.remove_viewer()
@staticmethod
def _part(frame: bytes) -> bytes:
header = (
f"--{BOUNDARY}\r\n"
"Content-Type: image/jpeg\r\n"
f"Content-Length: {len(frame)}\r\n"
"\r\n"
).encode("ascii")
return header + frame + b"\r\n"
class LiveServer:
"""Owns the listener and its lifetime. One instance, many streams."""
#: Binds that mean "every interface" rather than one address of its own.
WILDCARDS = ("", "0.0.0.0", "::")
def __init__(self, registry: StreamRegistry, *, host: str = DEFAULT_HOST,
port: int = DEFAULT_PORT, limiter: RateLimiter | None = None) -> None:
self.registry = registry
self.host = host
self.requested_port = port
self.limiter = limiter or RateLimiter()
self._httpd: _Server | None = None
self._thread: threading.Thread | None = None
self.ready = threading.Event()
@property
def port(self) -> int:
"""The bound port (resolves a requested port of 0 to the real one)."""
if self._httpd is None:
return self.requested_port
return self._httpd.server_address[1]
@property
def binds_everywhere(self) -> bool:
return self.host in self.WILDCARDS
@property
def address(self) -> str:
"""A host a browser can dial. A wildcard bind has none of its own, so
the loopback name stands in; `describe_bind` is what tells the truth
about the reach."""
return "127.0.0.1" if self.binds_everywhere else self.host
def describe_bind(self) -> str:
"""One line naming the real bind, and saying when it is every interface.
`url_for` has to print something dialable, which for `0.0.0.0` is a lie
of omission -- the stream is on the LAN, not on loopback. This is where
that is said out loud.
"""
where = f"{self.host or '0.0.0.0'}:{self.port}"
if self.binds_everywhere:
return f"bound to {where} -- all interfaces, reachable from the LAN"
return f"bound to {where}"
def url_for(self, phrase: str) -> str:
return f"http://{self.address}:{self.port}/{phrase}/"
def start(self) -> None:
"""Bind and serve. Raises `OSError` if the port is taken -- loudly.
Deliberately no hunting for a free port: a stream at a port nobody was
told is worse than a stream that refuses to start.
"""
self._httpd = _Server((self.host, self.requested_port), _Handler,
self.registry, self.limiter)
self._thread = threading.Thread(target=self._httpd.serve_forever,
name="hhstream-live-http", daemon=True)
self._thread.start()
self.ready.set()
log.info("listening, %s", self.describe_bind())
def stop(self) -> None:
if self._httpd is not None:
self._httpd.shutdown()
self._httpd.server_close()
self._httpd = None
if self._thread is not None:
self._thread.join(timeout=5)
self._thread = None
self.registry.stop_all()
__all__ = ["LiveServer", "RateLimiter", "DEFAULT_PORT", "DEFAULT_HOST", "BOUNDARY",
"HEADER_TIMEOUT_S"]
File diff suppressed because it is too large. Load diff