Files
hyprland-headless-stream-pl…/src/hhstreamd/registry.py
T
HermenixandClaude Opus 5 129430bffd wire the broker to the compositor, containers and pointer
The four slices existed and nothing called them. Now the broker builds a
spatial container per session, reads back what the compositor actually did,
and aims the cursor into one container under a broker-wide lease.

Record schema 3 (registry.py): `container` {output, workspace, mode,
created_by, state}, `pointer` {bookmark, last_released_at, released_by} and
the compositor's `signature`. A v2 record is migrated on load, not refused —
its application is still running. Every field that could not be verified is
null; `container: null` means "no container", never "probably something".

Containers (new container_host.py, manager.py, launcher.py): `start`
allocates a slot from `WorkspaceAllocator`, runs `container_plan()`'s argv,
launches the app with the container stamp in its environment, then reads the
workspace, output and window back out of `clients -j`. Teardown removes the
output, releases the slot and reports survivors rather than claiming success
— and runs for an unprompted exit and for `gc` too, so no container is ever
left unattended. Broker start-up reconciles: adopt `hh-*` outputs a live
session claims, remove the rest, `contains_output()` being the only filter.
No IPC means `container: degraded` and lifecycle verbs unchanged.

The plans' argv are dispatched over the compositor's request socket rather
than by executing `hyprctl`, which is what hyprctl does with the same words;
the reply is checked, so a refused command is not recorded as a container.

Pointer (new driver.py, manager.py): a client for the driver's private
socket, and `pointer.acquire|move|button|click|release|state` on top of
`PointerLease`. The broker owns the lease, the driver executes. The bookmark
is marked on every move and before release and stored in the record, so it
survives a restart. On expiry `pressed()` is read before `expire()` and
`release_all` goes to the driver, because the lease cannot lift a button
itself; a sweep runs unprompted so the TTL is a promise.

containers.py to v2: `quarantine_plan()` -> `quarantine_argv()`, with the
before-any-per-session-rule ordering stated where the caller could get it
wrong. protocol.py: 9 NO_LEASE, 10 COMPOSITOR, the verb table, and the
`request_id` cache moved out of the server so the CLI agrees with it.

Tests: real AF_UNIX everywhere — a stateful fake compositor and the shipped
pointer driver behind a recording sink. No mocked sockets, no compositor, no
VM. 551 -> 693.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017mUq7QvQ9EZb2Wi35xj4cs
2026-10-02 19:47:33 +00:00

518 lines
19 KiB
Python

"""The session record and its persistence (DESIGN.md §2.1, §2.3, §2.4).
Authoritative state is in daemon memory; the files under ``<state>/sessions/``
are a mirror, written ``tmp`` + :func:`os.replace` so a reader never sees a
half-written record. **A mirror write failure never fails a command** — it is
logged, and memory stays truthful.
"""
from __future__ import annotations
import base64
import json
import logging
import os
import secrets
import time
from dataclasses import asdict, dataclass, replace
from pathlib import Path
from typing import Any, Iterator
from .clock import Clock, iso
log = logging.getLogger("hhstreamd.registry")
#: 3 adds the container, the pointer bookmark and the compositor signature
#: (PLAN §7). The number is read, never written by anyone but this module: a
#: record carrying an older one is **migrated on load** (:func:`migrate`), not
#: refused, because the processes it describes are still running and refusing
#: the record would strand them.
SCHEMA = 3
#: Tombstones survive this long so a ``status`` after an unexplained failure
#: still answers the question. ``gc`` prunes them.
DEFAULT_TOMBSTONE_TTL_S = 3600.0
#: ``last_activity`` is hot, so the mirror is flushed at most this often for a
#: touch-only change — losing 15 s of an idle clock across a crash is
#: irrelevant against a three-hour timeout. State transitions always flush.
ACTIVITY_FLUSH_INTERVAL_S = 15.0
class ContainerState:
"""What the broker knows about a session's spatial container.
These are the only four answers. ``container: null`` is a fifth and
different thing — "this session has no container at all" — and nothing here
means "probably something": a field the compositor did not confirm stays
``None`` (PLAN §7).
"""
#: Created by us, and the compositor still shows the output.
OWNED = "owned"
#: No IPC, so nothing was created and nothing can be verified. The session
#: still starts, stops and is reaped by process identity (PLAN §4.4).
DEGRADED = "degraded"
#: Torn down on purpose: the output was removed and the slot released.
REMOVED = "removed"
#: The record and the compositor disagree — the output we recorded is not
#: there, or an output of ours has no live session. Never deleted on a
#: guess; reconciliation decides.
ORPHAN = "orphan"
class State:
STARTING = "starting"
READY = "ready"
EXITED = "exited"
KILLED = "killed"
REAPED = "reaped"
COMPOSITOR_LOST = "compositor_lost"
#: States a session can never leave. Reaching one writes a tombstone.
TERMINAL = frozenset({State.EXITED, State.KILLED, State.REAPED, State.COMPOSITOR_LOST})
#: The error name a verb should be answered with when it arrives at a session
#: in a terminal state (all map to exit 4).
TERMINAL_ERROR = {
State.EXITED: "session_exited",
State.KILLED: "session_killed",
State.REAPED: "session_reaped",
State.COMPOSITOR_LOST: "compositor_lost",
}
def boot_id() -> str:
"""The kernel's boot id, or ``""`` if it cannot be read.
Sessions cannot outlive a reboot — the compositor does not — so a record
whose ``boot_id`` differs from the running kernel's describes processes
that are certainly gone, whatever ``/proc`` says about the pid today. This
is cheaper and far safer than hoping ``CLOCK_BOOTTIME`` values from a
previous boot happen not to collide.
"""
try:
return Path("/proc/sys/kernel/random/boot_id").read_text().strip()
except OSError: # pragma: no cover - /proc always has it on Linux
return ""
@dataclass(slots=True)
class Session:
"""DESIGN.md §2.1, minus the fields M0 cannot honestly fill.
``window``, ``workspace`` and ``foreground`` exist in the record and are
always ``None`` in M0: the compositor is not reachable from here, and the
design is explicit that nothing may be guessed.
"""
id: str
app: str
argv: list[str]
pid: int
pgid: int
sid: int
pid_starttime: int
cmdline_sha256: str
home: str
runtime_dir: str
created_at: str
created_boottime: float
last_activity_at: str
last_activity_boottime: float
idle_timeout_s: float
state: str = State.STARTING
adopted: bool = False
ready_signal: str | None = None
workspace: int | None = None
window: Any = None
foreground: bool | None = None
#: ``{output, workspace, mode, created_by, state}`` or ``None`` for "no
#: container". A dict rather than a nested dataclass because
#: :meth:`to_dict` is ``asdict`` and :meth:`from_dict` is ``cls(**data)``:
#: a dataclass would serialise and come back as a dict anyway, and the
#: asymmetry is where a half-migrated record would hide.
container: dict[str, Any] | None = None
#: ``{bookmark, last_released_at, released_by}``. Absent (``None``) means
#: the pointer was never aimed here — not that its position is unknown.
pointer: dict[str, Any] | None = None
#: ``HYPRLAND_INSTANCE_SIGNATURE`` of the compositor this session's
#: container lives in, so a restarted broker re-attaches to the same
#: instance instead of picking whichever one is there now.
signature: str | None = None
command_count: int = 0
exit_code: int | None = None
exit_detected_by: str | None = None
last_error: str | None = None
log_path: str | None = None
ephemeral_home: bool = False
tombstoned_boottime: float | None = None
tombstone_ttl_s: float = DEFAULT_TOMBSTONE_TTL_S
boot_id: str = ""
schema: int = SCHEMA
# ---- derived -----------------------------------------------------
@property
def live(self) -> bool:
return self.state not in TERMINAL
def deadline(self) -> float | None:
"""The boottime instant at which this session becomes idle-expired.
``None`` for a session that can never expire: one in a terminal state,
or one whose idle timeout is zero (disabled).
"""
if not self.live or not self.idle_timeout_s:
return None
return self.last_activity_boottime + self.idle_timeout_s
def expires_in(self, now: float) -> float | None:
deadline = self.deadline()
return None if deadline is None else deadline - now
def idle_for(self, now: float) -> float | None:
return None if not self.live else now - self.last_activity_boottime
def tombstone_expires(self) -> float | None:
if self.tombstoned_boottime is None:
return None
return self.tombstoned_boottime + self.tombstone_ttl_s
def to_dict(self) -> dict[str, Any]:
return asdict(self)
@classmethod
def from_dict(cls, data: dict[str, Any]) -> "Session":
data = migrate(data)
known = {f for f in cls.__dataclass_fields__}
unknown = set(data) - known
if unknown:
# Forward compatibility: a newer daemon's extra fields are dropped
# rather than crashing the adopt path.
log.warning("dropping unknown record fields %s", sorted(unknown))
return cls(**{k: v for k, v in data.items() if k in known})
def public(self, now: float) -> dict[str, Any]:
"""The record as the protocol reports it, with the derived numbers."""
data = self.to_dict()
data["idle_s"] = self.idle_for(now)
data["expires_in_s"] = self.expires_in(now)
data["live"] = self.live
data["tombstone_expires_in_s"] = (
None if self.tombstone_expires() is None else self.tombstone_expires() - now
)
return data
def container_record(
*,
output: str | None,
workspace: int | None,
mode: tuple[int, int] | None,
created_by: str,
state: str,
) -> dict[str, Any]:
"""One ``container`` sub-record (PLAN §7).
Every argument is explicit and may be ``None`` because the degraded case is
the common one: there is a container *state* to report long before there is
an output to name, and the alternative — leaving the key out — would read
as "no container" when what we mean is "a container we could not verify".
"""
return {
"output": output,
"workspace": workspace,
"mode": None if mode is None else {"width": mode[0], "height": mode[1]},
"created_by": created_by,
"state": state,
}
def pointer_record(
*,
bookmark: dict[str, Any] | None,
last_released_at: str | None,
released_by: str | None,
) -> dict[str, Any]:
"""One ``pointer`` sub-record: where the cursor was left, and by what.
Written on every move and on release. A session that has one has been
aimed at; a session that has none never was.
"""
return {
"bookmark": bookmark,
"last_released_at": last_released_at,
"released_by": released_by,
}
def migrate(data: dict[str, Any]) -> dict[str, Any]:
"""Bring a record read from disk up to :data:`SCHEMA`.
A schema-2 record is **adopted, not rejected**: its application is very
likely still running, and the daemon that wrote it knew nothing of
containers, so the honest value for every field schema 3 added is ``None``
— "no container", "the pointer was never used here", "we do not know which
compositor instance this was". Each of those is a defined value, which is
the whole reason the migration can be this short.
A record from a *newer* schema is left alone and loudly logged: the fields
this daemon does not know are dropped by :meth:`Session.from_dict`, and
guessing at what a future version meant would be worse than running with
the subset we understand.
"""
schema = data.get("schema")
if schema == SCHEMA:
return data
if isinstance(schema, int) and schema > SCHEMA:
log.error(
"record %s is schema %s, newer than this daemon's %s; reading the fields "
"this version knows and leaving the rest",
data.get("id"),
schema,
SCHEMA,
)
return data
upgraded = dict(data)
# Only ever *adds* keys. A v2 record that somehow carries a container must
# keep it: the field is new to the schema, not new to the session.
for key in ("container", "pointer", "signature"):
upgraded.setdefault(key, None)
upgraded["schema"] = SCHEMA
log.info("migrated record %s from schema %s to %s", data.get("id"), schema, SCHEMA)
return upgraded
def mint_id(app: str, taken: set[str], *, rng: Any = secrets) -> str:
"""``<app-slug>-<6 base32 chars>`` (DESIGN.md §2.2).
Handles are never reused: an id present in the live *or* tombstone set is
refused and another drawn. Nothing parses the slug; it is a readability
affordance.
"""
slug = "".join(c if c.isalnum() else "-" for c in app.lower()).strip("-") or "app"
slug = slug[:24]
for _ in range(1000):
raw = rng.token_bytes(5)
suffix = base64.b32encode(raw).decode("ascii").rstrip("=").lower()[:6]
candidate = f"{slug}-{suffix}"
if candidate not in taken:
return candidate
raise RuntimeError("could not mint an unused session id") # pragma: no cover
def atomic_write_json(path: Path, data: dict[str, Any]) -> None:
"""Write ``data`` to ``path`` atomically: temp file in the same directory,
fsync, then :func:`os.replace`.
Same directory matters — ``os.replace`` is only atomic within a filesystem.
"""
path.parent.mkdir(parents=True, exist_ok=True)
tmp = path.parent / f".{path.name}.{os.getpid()}.{secrets.token_hex(4)}.tmp"
try:
with open(tmp, "w", encoding="utf-8") as fh:
json.dump(data, fh, ensure_ascii=True, sort_keys=True)
fh.write("\n")
fh.flush()
os.fsync(fh.fileno())
os.replace(tmp, path)
except BaseException:
try:
tmp.unlink()
except OSError:
pass
raise
class Registry:
"""The in-memory session table and its on-disk mirror."""
def __init__(self, state_dir: Path | str, clock: Clock | None = None) -> None:
self.state_dir = Path(state_dir)
self.sessions_dir = self.state_dir / "sessions"
self.quarantine_dir = self.sessions_dir / "bad"
self.clock = clock or Clock()
self._sessions: dict[str, Session] = {}
self._last_flush: dict[str, float] = {}
self._boot_id = boot_id()
# ---- directories -------------------------------------------------
def ensure_dirs(self) -> None:
"""Create the state directories if the supervisor has not.
In production these come from the unit's ``StateDirectory=``; M0 ships
as an ordinary process, so the daemon creates what is missing rather
than depending on how it is supervised.
"""
self.sessions_dir.mkdir(parents=True, exist_ok=True, mode=0o750)
# ---- table -------------------------------------------------------
def __contains__(self, session_id: object) -> bool:
return session_id in self._sessions
def __len__(self) -> int:
return len(self._sessions)
def __iter__(self) -> Iterator[Session]:
return iter(list(self._sessions.values()))
def get(self, session_id: str) -> Session | None:
return self._sessions.get(session_id)
def all(self) -> list[Session]:
return sorted(self._sessions.values(), key=lambda s: s.created_boottime)
def live(self) -> list[Session]:
return [s for s in self.all() if s.live]
def tombstones(self) -> list[Session]:
return [s for s in self.all() if not s.live]
def taken_ids(self) -> set[str]:
"""Live *and* tombstoned ids, so handles are never reused."""
return set(self._sessions)
def new_id(self, app: str) -> str:
return mint_id(app, self.taken_ids())
def add(self, session: Session) -> Session:
if session.id in self._sessions:
raise KeyError(f"session id already in use: {session.id}")
if not session.boot_id:
session.boot_id = self._boot_id
self._sessions[session.id] = session
self.persist(session, force=True)
return session
def record_path(self, session_id: str) -> Path:
return self.sessions_dir / f"{session_id}.json"
def persist(self, session: Session, *, force: bool = False) -> None:
"""Mirror one record. Never raises: a mirror failure is not a command failure."""
now = self.clock.boottime()
if not force and now - self._last_flush.get(session.id, 0.0) < ACTIVITY_FLUSH_INTERVAL_S:
return
try:
self.ensure_dirs()
atomic_write_json(self.record_path(session.id), session.to_dict())
self._last_flush[session.id] = now
except OSError as exc:
log.error("mirror write failed for %s: %s (memory is authoritative)", session.id, exc)
def flush_all(self) -> None:
for session in self.all():
self.persist(session, force=True)
def drop(self, session_id: str) -> None:
"""Forget a record entirely and remove its mirror (used by ``gc``)."""
self._sessions.pop(session_id, None)
self._last_flush.pop(session_id, None)
try:
self.record_path(session_id).unlink()
except FileNotFoundError:
pass
except OSError as exc:
log.error("could not remove mirror for %s: %s", session_id, exc)
# ---- transitions -------------------------------------------------
def touch(self, session: Session, *, force_flush: bool = False) -> None:
"""Stamp activity. Callers stamp *after* a command is accepted."""
session.last_activity_boottime = self.clock.boottime()
session.last_activity_at = iso(self.clock.wall())
session.command_count += 1
self.persist(session, force=force_flush)
def terminate(
self,
session: Session,
state: str,
*,
exit_code: int | None = None,
exit_detected_by: str | None = None,
last_error: str | None = None,
) -> None:
"""Move a session to a terminal state and write its tombstone."""
session.state = state
if exit_code is not None:
session.exit_code = exit_code
if exit_detected_by is not None:
session.exit_detected_by = exit_detected_by
if last_error is not None:
session.last_error = last_error
session.tombstoned_boottime = self.clock.boottime()
self.persist(session, force=True)
# ---- startup adoption -------------------------------------------
def load_records(self) -> tuple[list[Session], list[tuple[Path, str]]]:
"""Read the mirror. Returns ``(records, quarantined)``.
A record that will not parse is moved to ``sessions/bad/`` and logged
loudly; its processes are left running and reported by ``gc`` rather
than killed on the basis of unreadable state (DESIGN.md §7 #13).
"""
records: list[Session] = []
quarantined: list[tuple[Path, str]] = []
if not self.sessions_dir.is_dir():
return records, quarantined
for path in sorted(self.sessions_dir.glob("*.json")):
try:
data = json.loads(path.read_text(encoding="utf-8"))
if not isinstance(data, dict):
raise ValueError("record is not an object")
session = Session.from_dict(data)
except (OSError, ValueError, TypeError) as exc:
reason = f"{type(exc).__name__}: {exc}"
log.error("quarantining unreadable record %s: %s", path, reason)
quarantined.append((path, reason))
self._quarantine(path)
continue
records.append(session)
return records, quarantined
def _quarantine(self, path: Path) -> None:
try:
self.quarantine_dir.mkdir(parents=True, exist_ok=True, mode=0o750)
os.replace(path, self.quarantine_dir / f"{path.name}.{int(time.time())}")
except OSError as exc: # pragma: no cover - best effort
log.error("could not quarantine %s: %s", path, exc)
def insert_loaded(self, session: Session) -> Session:
"""Put a record read from disk back into the table without re-persisting."""
self._sessions[session.id] = session
return session
def stale_boot(self, session: Session) -> bool:
"""True if the record was written before the current boot.
Its processes are gone whatever ``/proc`` says, because pids are
recycled from zero at boot.
"""
return bool(session.boot_id) and bool(self._boot_id) and session.boot_id != self._boot_id
def clone(session: Session, **changes: Any) -> Session:
return replace(session, **changes)