From ca5e309e511a1221ad25ba623de97accf59fb4c6 Mon Sep 17 00:00:00 2001 From: Hermenix Date: Sun, 11 Oct 2026 06:32:23 +0000 Subject: [PATCH] feat(mcp): the hhstream CLI as Model Context Protocol tools `hhstream_mcp` serves the whole `hhstream` surface -- the four frame verbs, the ten session verbs, the eight pointer verbs and the four keyboard verbs -- as 26 MCP tools over stdio, so an agent drives this desktop through typed tools instead of writing command lines. It is a shim and nothing more. Each tool builds the argv list the CLI's own parser takes, hands it to the CLI's own handler **in this process**, and maps what came back. The idle clock, the pointer lease, the allowlist, the exit-code table and every refusal message stay where they already were. Four decisions worth stating, each argued at length where it lives: * **One tool per verb**, not one per family with an `action` enum. The arguments do not rhyme, the two rules an agent gets wrong have to be on the tool it gets wrong, and `readOnlyHint` is a real per-verb distinction here -- `session_container` with an id runs `container.reconcile`, so it is not marked read-only. * **In process, not as a subprocess.** `hhstream.cli`'s parser already takes an argv list and its handlers already return a code from the frozen table. Two of its paths cannot be called here and both end in an `exec` that would replace this server: `main`'s `sg` re-exec, and `cmd_run`'s `execvpe`. The first is therefore unavailable and said out loud (`access_hint`); the second is why `run` goes through `posix_spawn` into its own session. * **The transport leaves file descriptor 1.** The CLI prints, and `record` spawns an `ffmpeg` whose output it does not capture. `claim_transport` takes a private duplicate of stdin and stdout at start-up and points fd 1 at the log, so a stray write is a log line and never a corrupted JSON-RPC stream. * **No state, and the lease is an argument.** The CLI caches a lease id between invocations because its surface is flagless; this passes `--lease` on every call and never reads that cache. A cached lease is a lease an agent cannot see. Refusals come back as tool errors -- never as a success -- carrying the remedy sentence, the CLI's own `error`/`reason`/`hint`, everything it said, and the frozen exit code in `structuredContent`. Exits 8 and 9 are held apart by a test, because an agent that sees 8 must wait and one that sees 9 must acquire. The root pyproject excludes `hhstream_mcp*` from package discovery: without that, `packages.hhstream` would ship a module whose first import line fails. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_011ZeH65zd6pFTgBSu52W9PE --- pyproject.toml | 9 + src/hhstream_mcp/__init__.py | 43 ++ src/hhstream_mcp/__main__.py | 8 + src/hhstream_mcp/invoke.py | 496 ++++++++++++ src/hhstream_mcp/media.py | 115 +++ src/hhstream_mcp/pyproject.toml | 46 ++ src/hhstream_mcp/server.py | 285 +++++++ src/hhstream_mcp/tools.py | 1263 +++++++++++++++++++++++++++++++ 8 files changed, 2265 insertions(+) create mode 100644 src/hhstream_mcp/__init__.py create mode 100644 src/hhstream_mcp/__main__.py create mode 100644 src/hhstream_mcp/invoke.py create mode 100644 src/hhstream_mcp/media.py create mode 100644 src/hhstream_mcp/pyproject.toml create mode 100644 src/hhstream_mcp/server.py create mode 100644 src/hhstream_mcp/tools.py diff --git a/pyproject.toml b/pyproject.toml index ba764e1..18c4aec 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -34,6 +34,15 @@ hhstream-live = "hhstream_live.__main__:main" [tool.setuptools.packages.find] where = ["src"] +# `hhstream_mcp` lives under src/ beside the others -- so the dev shell's +# PYTHONPATH reaches it and the tests import it the same way they import +# everything else -- but it is a *different distribution* +# (src/hhstream_mcp/pyproject.toml, built by nix/mcp.nix) because it depends on +# the MCP SDK. Without this exclusion `find` would sweep it into this wheel, +# and `packages.hhstream` would ship a module whose first import line fails: +# the dependency policy above would be intact and the package would still be +# broken. +exclude = ["hhstream_mcp*"] # The phrase generator reads this at runtime, so it has to be *installed*, not # just present in the checkout: a wheel (and the Nix build) that dropped it diff --git a/src/hhstream_mcp/__init__.py b/src/hhstream_mcp/__init__.py new file mode 100644 index 0000000..29d51eb --- /dev/null +++ b/src/hhstream_mcp/__init__.py @@ -0,0 +1,43 @@ +"""``hhstream-mcp`` -- the ``hhstream`` CLI, as Model Context Protocol tools. + +A **shim**, and deliberately the thinnest one that can be honest: every tool +builds the argv list the CLI's own parser takes, hands it to the CLI's own +handler *in this process*, and maps what came back. No behavioural rule lives +here. The idle clock, the pointer lease, the container policy, the allowlist, +the exit-code table and the refusal messages are all the CLI's and the broker's, +and this package's one job is to not get between an agent and them. + +In process, and not as a subprocess: ``hhstream.cli`` is importable Python, its +``main`` already takes an argv list and returns an exit code from the frozen +table, and spawning a Python interpreter per tool call to re-parse arguments +this process just built would be paying for an isolation nobody asked for. The +seam that matters is the *argv list* -- the thing this layer actually decides -- +and that is identical either way. :mod:`hhstream_mcp.invoke` says which two of +the CLI's paths cannot be called in process, and what is done about them. + +Why a separate distribution from ``hhstream`` rather than one more entry point +in ``pyproject.toml``: the frame socket, the CLI and the broker are stdlib-only +by policy (``pyproject.toml``, ``docs/CONTRACT-sessions.md`` §6 rule 3), and the +MCP SDK is a third-party dependency. Growing ``packages.hhstream`` an ``mcp`` +dependency to serve a shim would put that dependency on the compositor's own +units. So the code lives beside the others under ``src/`` -- the dev shell's +``PYTHONPATH`` reaches it for free -- and ``nix/mcp.nix`` builds it as its own +derivation, with its own pyproject, that *depends on* the ``hhstream`` package. + +The interface is frozen in ``docs/CONTRACT-mcp.md``: the tool names, their +arguments, and which exit code arrives as which MCP error. The modules split +along the one line that matters for testing -- :mod:`hhstream_mcp.invoke`, +:mod:`hhstream_mcp.media` and :mod:`hhstream_mcp.tools` are standard library +only, so the argv each tool builds, the parsing of what the CLI printed and the +error mapping are all testable without the SDK and without a live socket. Only +:mod:`hhstream_mcp.server` imports ``mcp``. +""" + +from __future__ import annotations + +__version__ = "0.1.0" + +#: What the server calls itself in the ``initialize`` handshake. The same name +#: an MCP client configuration block is expected to use for it, so a log line +#: and a config entry can be matched up by eye. +SERVER_NAME = "hhstream" diff --git a/src/hhstream_mcp/__main__.py b/src/hhstream_mcp/__main__.py new file mode 100644 index 0000000..6b3caff --- /dev/null +++ b/src/hhstream_mcp/__main__.py @@ -0,0 +1,8 @@ +"""``python -m hhstream_mcp`` — the same entry point as the ``hhstream-mcp`` script.""" + +from __future__ import annotations + +from .server import main + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/src/hhstream_mcp/invoke.py b/src/hhstream_mcp/invoke.py new file mode 100644 index 0000000..c1583f1 --- /dev/null +++ b/src/hhstream_mcp/invoke.py @@ -0,0 +1,496 @@ +"""Run the CLI's own functions, in this process. Standard library only. + +No subprocess. A tool builds the same argv list the CLI's parser takes, this +module hands it to ``hhstream.cli``'s parser, runs the handler argparse +selected, and reports what that handler printed and returned. The behavioural +rules stay exactly where they were -- the idle clock, the lease, the exit-code +table, the allowlist and every refusal message are the CLI's and the broker's -- +and what is saved is a process spawn per tool call and a ``PATH`` to get wrong. + +**Why not ``hhstream.cli.main``.** Two of its paths end in an ``exec``, and in +a CLI that is correct while here it would replace the MCP server mid-session and +the client would see its connection die: + +1. ``main`` calls ``ensure_access`` before the four frame verbs, which re-execs + the whole process through the setuid ``sg`` wrapper when the frame socket is + out of reach. +2. ``cmd_run`` ends in ``os.execvpe`` -- that *is* the verb. ``run`` therefore + forks first; see :meth:`Runner.launch`. + +So the parser is the CLI's (``cli.build_parser``), the handler is the one +argparse selected, and the exit code is that handler's own return value. What +this module adds is the dispatch line between them, the capture, and the remedy +sentence per exit code. + +**The one capability lost with the ``sg`` re-exec**, stated plainly because it +is a real difference from running the CLI: a process that cannot read the frame +socket does not recover here. ``shot`` and ``record`` fail with the CLI's own +message plus a hint, where the CLI would have rejoined the group and succeeded. +The remedy is to give the server the group in the first place -- see the README +section on registering it. + +**Serialised.** The CLI prints to process-global ``sys.stdout``, so two +concurrent calls would interleave their output into each other's reply. One +lock around the whole call is the honest fix, and costs nothing a caller would +notice: the broker already serialises every mutation, and the CLI was one +process per verb anyway. +""" + +from __future__ import annotations + +import contextlib +import dataclasses +import io +import json +import os +import shutil +import sys +import tempfile +import threading +import time +import traceback +from typing import Any, Callable, Iterator + +from hhstream import cli +from hhstreamd.protocol import ExitCode + +#: Exit 1 is not in the frozen table: it is what the *frame* verbs return when +#: the publisher sent no frame or a tool they need is missing, and what this +#: module reports for an exception no handler caught. +GENERIC_FAILURE = 1 + +#: The three frame verbs' handlers, by the name the CLI's parser gives them. +#: `run` is deliberately absent -- it is the one that execs -- and the session +#: namespace needs no entry, because argparse sets `func` for it. +FRAME_VERBS: dict[str, Callable[[Any], int]] = { + "status": cli.cmd_status, + "shot": cli.cmd_shot, + "record": cli.cmd_record, +} + +#: One CLI call at a time. See the module docstring. +_ONE_AT_A_TIME = threading.Lock() + + +@dataclasses.dataclass(frozen=True) +class Outcome: + """One completed CLI call, exactly as it ended.""" + + argv: list[str] + returncode: int + stdout: str + stderr: str + + @property + def ok(self) -> bool: + return self.returncode == 0 + + def payload(self) -> dict[str, Any] | None: + """The protocol object a ``--json`` verb printed, wherever it printed it. + + stdout on success and **stderr on failure** -- that is the CLI's own + split (``hhstream_session._finish``), not a guess: an error always goes + to stderr whatever the output mode, because that is what an agent + branches on. Both streams are searched in both cases anyway, because a + verb that grows a warning on the happy path should not take the parsing + with it. + """ + + for text in (self.stdout, self.stderr): + found = first_json_object(text) + if found is not None: + return found + return None + + +def first_json_object(text: str) -> dict[str, Any] | None: + """The first JSON *object* in ``text``, or ``None``. + + A scan rather than ``json.loads`` on the whole stream: ``session start`` + prints the bare id above its object, and a verb that one day prints a line + above one should not stop being parseable. + """ + + decoder = json.JSONDecoder() + index = text.find("{") + while index != -1: + try: + value, _ = decoder.raw_decode(text, index) + except ValueError: + index = text.find("{", index + 1) + continue + if isinstance(value, dict): + return value + index = text.find("{", index + 1) + return None + + +# ── what each exit code means to an agent ────────────────────────────────── + +#: The sentence a calling agent needs after a refusal: not what went wrong -- +#: the CLI's own ``reason`` says that better than any table could -- but what to +#: do about it. Keyed by the frozen exit codes of +#: ``docs/CONTRACT-sessions.md`` §4, so a new code is a missing key here and +#: not a silently generic message. +REMEDY: dict[int, str] = { + ExitCode.USAGE: ( + "The call was rejected before anything happened: an argument is " + "missing, malformed, or names something this build does not support. " + "Fix the arguments; retrying the same call will fail the same way." + ), + ExitCode.NO_SUCH_SESSION: ( + "No session (or process) by that id. Call `session_list` to see which " + "ids exist; a reaped session is gone and has to be started again." + ), + ExitCode.BAD_STATE: ( + "The session exists but is not in a state this verb can act on -- it " + "is still starting, or it has already exited, been killed or been " + "reaped. Call `session_status` to see which." + ), + ExitCode.DAEMON_UNREACHABLE: ( + "The session broker (`hhstreamd`) could not be reached on its control " + "socket. Nothing was done. This is an operator problem, not an " + "argument problem: the daemon is down, or $HHSTREAM_SOCKET points " + "somewhere else. Frame tools (`status`, `shot`, `record`, `run`) do " + "not need the broker and may still work." + ), + ExitCode.POLICY: ( + "Refused by policy. The application is not on the broker's allowlist, " + "or an argument is not one the allowlist permits, or a budget (session " + "cap, workspace band) is full. No retry will change it; this is a " + "configuration decision." + ), + ExitCode.TIMEOUT: ( + "The broker accepted the request and did not finish in time. Whether " + "it took effect is unknown -- read the state back (`session_status`, " + "`session_list`, `pointer_state`) before retrying, and retry with the " + "same `request_id` so a mutation cannot happen twice." + ), + ExitCode.BUSY: ( + "Something else holds what this call needs -- for the pointer and " + "keyboard verbs, another session holds the one pointer lease, and the " + "refusal names the holder. Nothing was injected. Call `pointer_state` " + "to see who holds it, then wait for them to release it or for its TTL " + "to expire. Do NOT try to take it from them; there is no such verb." + ), + ExitCode.NO_LEASE: ( + "This verb needs the pointer lease and this session does not hold one " + "(never acquired, already released, or expired -- an idle lease is " + "force-released by the broker). Call `pointer_acquire` for this " + "session and pass the `lease_id` it returns as `lease` on every " + "pointer and keyboard call. A lease id that was refused is dead; " + "acquire a new one rather than retrying with it." + ), + ExitCode.COMPOSITOR: ( + "The compositor or the input driver failed. **The session itself is " + "not lost** -- it keeps running in degraded mode -- so do not treat " + "this as a dead session. The compositor may be down, the container may " + "be gone, or the driver refused the request (an unmappable character, " + "a keysym this layout does not have)." + ), + GENERIC_FAILURE: ( + "The command failed without one of the broker's classified codes. For " + "the frame tools this is usually the publisher sending no frame (is the " + "compositor up? call `status`) or `grim`/`ffmpeg` missing from the " + "server's PATH. The text below is the CLI's own." + ), +} + + +def remedy_for(returncode: int) -> str: + try: + name = ExitCode(returncode).name + except ValueError: + name = "UNCLASSIFIED" + advice = REMEDY.get( + returncode, + "The CLI exited with a code this server does not know. Treat it as a " + "failure and read the output below; do not assume anything happened.", + ) + return f"exit {returncode} ({name}): {advice}" + + +# ── capture ──────────────────────────────────────────────────────────────── + + +@contextlib.contextmanager +def captured() -> Iterator[tuple[io.StringIO, io.StringIO]]: + """Redirect the CLI's ``print``s into a pair of buffers. + + Python-level only, and that is enough **because the protocol is not on file + descriptor 1 any more**: :func:`hhstream_mcp.server.claim_transport` takes a + private duplicate of stdout at start-up and points fd 1 at the log. So a + stray write from a grandchild (``ffmpeg``, which the CLI does not capture) + lands in the server's log rather than in the client's stream, and this + context manager only has to collect what the handler itself printed. + """ + + out, err = io.StringIO(), io.StringIO() + with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err): + yield out, err + + +def dispatch(argv: list[str]) -> int: + """Parse ``argv`` with the CLI's own parser and run the handler it selects. + + The CLI's ``main`` minus its two ``exec`` paths; see the module docstring. + """ + + parser = cli.build_parser() + args = parser.parse_args(argv) + if args.command is None: + parser.print_help(sys.stderr) + return int(ExitCode.USAGE) + handler = FRAME_VERBS.get(args.command) + if handler is not None: + return int(handler(args)) + # The session namespace: argparse already chose the handler when it parsed. + return int(getattr(args, "func")(args)) + + +def access_hint(argv: list[str]) -> str: + """The sentence the ``sg`` re-exec would otherwise have made unnecessary. + + Added to a failure only when the diagnosis is actually true, and only for + the verbs it could be true of. The predicate is the CLI's own, asked about + the CLI's own resolved socket path; the verb list is the one ``cli.main`` + guards, because the frame socket has nothing to do with why a session verb + failed and an explanation that does not apply is worse than none. + """ + + try: + # Inside the capture, because an argv argparse refuses makes it print + # usage -- and this runs *after* the call, so that print would reach + # the log rather than the reply, saying the same thing twice in the + # wrong place. + with captured(): + args = cli.build_parser().parse_args(argv) + if args.command not in FRAME_VERBS and args.command != "run": + return "" + socket_path = getattr(args, "frame_socket", None) + if socket_path and not cli.have_access(socket_path): + return ( + f"\n\nThis process cannot read {socket_path}. The hhstream CLI " + "would have re-joined the compositor's group through the " + "setuid 'sg' wrapper at this point; this server cannot, " + "because that is an exec which would replace it and drop the " + "client's connection. Run it as a member of that group " + "instead -- `sg hhstream -c hhstream-mcp` in the client's " + "spawn command, or add the account to the group." + ) + except SystemExit: + # A parse that fails here fails identically in the call itself, and + # that failure is the one worth reporting. + return "" + return "" + + +# ── the runner ───────────────────────────────────────────────────────────── + + +@dataclasses.dataclass(frozen=True) +class Spawned: + """A process that was started and deliberately not waited for.""" + + argv: list[str] + pid: int + exited: bool + stdout: str + stderr: str + + +class Runner: + """Calls the CLI. Holds the entry point and nothing else. + + No lease, no session id, no last reply. The server is a shim + (``docs/CONTRACT-mcp.md`` §3), and the moment it remembers a lease an agent + can no longer see which lease its call used. + + ``entry`` exists so the tests can put a recorder where the CLI goes: what + this layer decides is the argv list, and a test that cannot see the argv + list is testing nothing. There is no second production value for it. + """ + + def __init__(self, entry: Callable[[list[str]], int] | None = None) -> None: + self.entry = entry or dispatch + + def run(self, argv: list[str]) -> Outcome: + """One call, start to finish. Never raises for a CLI failure.""" + + with _ONE_AT_A_TIME: + with captured() as (out, err): + try: + code = int(self.entry(list(argv))) + except SystemExit as exc: + # argparse's own refusals, and the handlers' `sys.exit` for + # an unreachable compositor or a grim that is not there. + code = _exit_status(exc) + except BaseException: # noqa: BLE001 -- a CLI process would + # have died here with a traceback on stderr and status 1; + # reproducing that is more honest than letting it reach the + # protocol as a transport error. + traceback.print_exc(file=sys.stderr) + code = GENERIC_FAILURE + stdout, stderr = out.getvalue(), err.getvalue() + if code != 0: + stderr += access_hint(list(argv)) + return Outcome(argv=list(argv), returncode=code, stdout=stdout, stderr=stderr) + + def launch(self, command: list[str], *, settle: float) -> Spawned: + """``run``: put an application on the screen and do not wait for it. + + ``cli.cmd_run`` is two lines -- the environment a Wayland client needs, + then ``os.execvpe`` -- and the second of them is why it cannot be + called here: it would turn this server into the application. So the + environment comes from the CLI's own + :func:`hhstream.cli.env_for_session` and the exec is done by + ``posix_spawn``: + + * ``posix_spawn`` and not ``fork`` because this runs on a worker thread. + A ``fork`` from a threaded process can inherit a lock another thread + was holding, and Python warns about exactly that; ``posix_spawn`` + leaves no Python running between the clone and the exec. + * ``setsid`` puts the application in its own session, so a client that + kills this server's process group does not take the desktop with it. + That, plus not waiting, is what "launch and forget" means. + * The executable is resolved *here*, so "no such command" is a refusal + an agent can read rather than a child exiting 127 into a log. + * A reaper thread waits on the pid, so a long-lived application that + eventually exits does not sit in this process's table as a zombie. + """ + + # The frame socket the CLI would have resolved -- its flag default, or + # $HHSTREAM_FRAME_SOCKET -- read out of its own parser rather than + # re-derived, because `run` takes the compositor's runtime directory + # from it and a second opinion about that path is a second bug. + frame_socket = cli.build_parser().parse_args(["status"]).frame_socket + runtime_dir = os.path.dirname(frame_socket) + try: + environment = dict(os.environ) + environment.update(cli.env_for_session(runtime_dir)) + except SystemExit as exc: + # `env_for_session` refuses with `sys.exit(message)` when the + # compositor has bound no Wayland socket. The message is the CLI's + # own and is handed on verbatim. + return Spawned( + argv=command, pid=0, exited=True, stdout="", stderr=str(exc.code) + ) + resolved = shutil.which(command[0], path=environment.get("PATH")) + if resolved is None: + return Spawned( + argv=command, + pid=0, + exited=True, + stdout="", + stderr=f"hhstream-mcp: no executable {command[0]!r} on PATH", + ) + return _spawn_detached(resolved, command, environment, settle) + + +def _exit_status(exc: SystemExit) -> int: + """``SystemExit`` as a status. A message is status 1, as in a real process.""" + + if exc.code is None: + return 0 + if isinstance(exc.code, int): + return exc.code + return GENERIC_FAILURE + + +def _spawn_detached( + resolved: str, command: list[str], environment: dict[str, str], settle: float +) -> Spawned: + """Start the application in its own session and report its pid.""" + + with tempfile.TemporaryDirectory(prefix="hhstream-mcp-run-") as scratch: + capture = os.path.join(scratch, "output") + # Its stdin must not be the protocol stream, and its first complaint is + # worth handing back to the agent rather than mixing into the server's + # log -- so both of its output descriptors go to one file, which this + # directory takes away afterwards. An application that keeps writing + # writes to a deleted inode and leaves nothing behind. + file_actions = [ + (os.POSIX_SPAWN_OPEN, 0, os.devnull, os.O_RDONLY, 0o666), + ( + os.POSIX_SPAWN_OPEN, + 1, + capture, + os.O_WRONLY | os.O_CREAT | os.O_TRUNC, + 0o600, + ), + (os.POSIX_SPAWN_DUP2, 1, 2), + ] + pid = os.posix_spawn( + resolved, command, environment, file_actions=file_actions, setsid=True + ) + exited = _watch(pid, settle) + if not exited: + _reap_later(pid) + try: + with open(capture, encoding="utf-8", errors="replace") as fh: + said = fh.read() + except OSError: # pragma: no cover -- the spawn created it + said = "" + return Spawned( + argv=command, + pid=pid, + exited=exited, + stdout="" if exited else said, + stderr=said if exited else "", + ) + + +#: How often to look at a launched process while the settle window runs. +_WATCH_INTERVAL_S = 0.05 + + +def _watch(pid: int, settle: float) -> bool: + """``True`` if the process is gone before ``settle`` seconds are up. + + ``waitpid`` with ``WNOHANG``, so an application that fails at once is both + detected and reaped here. + """ + + deadline = time.monotonic() + max(0.0, settle) + while True: + try: + reaped, _ = os.waitpid(pid, os.WNOHANG) + except ChildProcessError: # pragma: no cover -- already reaped + return True + if reaped == pid: + return True + if time.monotonic() >= deadline: + return False + time.sleep(_WATCH_INTERVAL_S) + + +def _reap_later(pid: int) -> None: + """Wait on a surviving application, on a thread that does nothing else. + + Without this the application sits in this process's table as a zombie from + the moment it exits until the server does. A thread blocked in ``waitpid`` + is the cheapest way to not have that, and a daemon one so it never delays + shutdown. + """ + + def wait() -> None: + try: + os.waitpid(pid, 0) + except (ChildProcessError, OSError): # pragma: no cover + pass + + threading.Thread(target=wait, name=f"hhstream-mcp-reap-{pid}", daemon=True).start() + + +def temp_name(prefix: str, suffix: str) -> str: + """A unique filename for an artefact, in the system temporary directory. + + Used for ``record``'s default destination, which is the one artefact a tool + hands back by *path*: it has to outlive the call, so it cannot live in a + directory this process cleans up. + """ + + directory = os.path.join(tempfile.gettempdir(), "hhstream-mcp") + os.makedirs(directory, exist_ok=True) + stamp = time.strftime("%Y%m%d-%H%M%S") + return os.path.join(directory, f"{prefix}-{stamp}-{os.getpid()}{suffix}") diff --git a/src/hhstream_mcp/media.py b/src/hhstream_mcp/media.py new file mode 100644 index 0000000..dc2bd14 --- /dev/null +++ b/src/hhstream_mcp/media.py @@ -0,0 +1,115 @@ +"""Read back what ``shot`` and ``record`` produced. Standard library only. + +Two small readers, and both are deliberately readers rather than deciders. +``shot`` writes a file and prints its path; ``record`` writes an mp4 and prints +the path with the rate it measured. The geometry and the rate are facts those +artefacts already carry, and an agent that gets a picture without its +dimensions has to guess at the coordinate space it is about to aim the pointer +in -- so they are parsed out and put in the text part of the reply. + +Nothing here re-derives anything the CLI decided. The frame rate is the CLI's +own measured number, read off its output; the dimensions come from the image's +header, not from the configured output size, because the configured size is +what the compositor was *asked* for. +""" + +from __future__ import annotations + +import dataclasses +import re +import struct + +PNG_MAGIC = b"\x89PNG\r\n\x1a\n" +JPEG_MAGIC = b"\xff\xd8" + +#: The SOFn markers that carry a JPEG's dimensions. Everything except DHT +#: (c4), DAC (cc) and the RSTn range (d0-d7), which share the 0xffc_ block. +_JPEG_SOF = {0xC0, 0xC1, 0xC2, 0xC3, 0xC5, 0xC6, 0xC7, 0xC9, 0xCA, 0xCB, 0xCD, 0xCE, 0xCF} + + +@dataclasses.dataclass(frozen=True) +class Image: + media_type: str + width: int + height: int + size: int + + +def sniff(data: bytes) -> Image | None: + """The media type and pixel size of one captured frame, or ``None``. + + By magic bytes and not by the extension this server asked for. The frame + socket publishes whatever format the module configured -- ``image/png`` by + default, ``image/jpeg`` on the deployment machine -- and ``hhstream shot`` + writes the payload it was given to the ``--out`` path verbatim, so a file + called ``.png`` holds a JPEG whenever the publisher is serving one. An + agent told the wrong media type gets an image its client cannot decode. + """ + + if data.startswith(PNG_MAGIC): + return _png(data) + if data.startswith(JPEG_MAGIC): + return _jpeg(data) + return None + + +def _png(data: bytes) -> Image | None: + # IHDR is required to be the first chunk: 8 bytes of signature, a 4-byte + # length, the type, then width and height. + if len(data) < 24 or data[12:16] != b"IHDR": + return None + width, height = struct.unpack(">II", data[16:24]) + return Image("image/png", width, height, len(data)) + + +def _jpeg(data: bytes) -> Image | None: + index = 2 + end = len(data) + while index + 3 < end: + if data[index] != 0xFF: + index += 1 + continue + marker = data[index + 1] + if marker in (0xD8, 0x01) or 0xD0 <= marker <= 0xD7: + index += 2 + continue + if marker == 0xD9 or marker == 0xDA: # end of image, or start of scan + return None + (length,) = struct.unpack(">H", data[index + 2 : index + 4]) + if marker in _JPEG_SOF: + if index + 9 > end: + return None + height, width = struct.unpack(">HH", data[index + 5 : index + 9]) + return Image("image/jpeg", width, height, len(data)) + index += 2 + length + return None + + +#: What ``hhstream record`` prints on success, which is the only place the +#: *measured* rate exists: a software-rendered compositor tops out well under +#: the advertised fps, so the encoded rate is the one it counted, and an agent +#: reading the mp4 later cannot recover how long the capture actually took. +RECORD_LINE = re.compile( + r"^(?P.+?) \((?P\d+) frames in (?P[\d.]+)s -> (?P\d+) fps\)\s*$", + re.MULTILINE, +) + + +@dataclasses.dataclass(frozen=True) +class Recording: + path: str + frames: int + seconds: float + fps: int + + +def recording(stdout: str) -> Recording | None: + match = RECORD_LINE.search(stdout) + if match is None: + return None + return Recording( + path=match["path"], + frames=int(match["frames"]), + seconds=float(match["seconds"]), + fps=int(match["fps"]), + ) diff --git a/src/hhstream_mcp/pyproject.toml b/src/hhstream_mcp/pyproject.toml new file mode 100644 index 0000000..6642c88 --- /dev/null +++ b/src/hhstream_mcp/pyproject.toml @@ -0,0 +1,46 @@ +# The second distribution in this repository, and the only one with a +# third-party runtime dependency. +# +# It sits *inside* the package directory rather than at the repository root +# because the root `pyproject.toml` is `hhstream`'s and a project has one. The +# alternatives were worse: a second source tree would duplicate nothing but a +# directory level, and adding `hhstream-mcp` to the root project would put the +# `mcp` dependency on the compositor's own units -- which is the one thing +# `docs/CONTRACT-sessions.md` §6 rule 3 and the root pyproject's own comment +# forbid. The root project excludes `hhstream_mcp*` from its package discovery +# for the same reason; see the note there. +# +# `package-dir` maps the package name onto this directory, so the layout stays +# `src/hhstream_mcp/` -- which is what puts it on the dev shell's +# `PYTHONPATH=$PWD/src` beside every other package here, with no second path to +# set and no second place for a test to import from. + +[build-system] +requires = ["setuptools>=68"] +build-backend = "setuptools.build_meta" + +[project] +name = "hhstream-mcp" +version = "0.1.0" +description = "Drive a headless Hyprland session over the Model Context Protocol" +requires-python = ">=3.11" +# `mcp` is the official Python SDK and the whole reason this is a separate +# distribution. `anyio` is declared rather than taken for granted through the +# SDK: `server.py` calls `anyio.run` and `anyio.to_thread` directly, and a +# dependency used by name belongs in the metadata. +# +# `hhstream` is NOT listed, and it is not optional either: this package imports +# `hhstream.cli`, `hhstreamd.protocol` and `hhstream_session` and does not work +# without them. It is absent because `hhstream` is not a wheel anyone installs +# from an index -- it is this repository's other derivation, and `nix/mcp.nix` +# is where the two are joined. A dependency naming a version nobody can +# resolve would be worse than the comment you are reading. +dependencies = ["mcp>=1.9", "anyio>=4"] + +[project.scripts] +# Stdio only. One process per client, spawned by the client; no port, no daemon. +hhstream-mcp = "hhstream_mcp.server:main" + +[tool.setuptools] +packages = ["hhstream_mcp"] +package-dir = { "hhstream_mcp" = "." } diff --git a/src/hhstream_mcp/server.py b/src/hhstream_mcp/server.py new file mode 100644 index 0000000..93ec36f --- /dev/null +++ b/src/hhstream_mcp/server.py @@ -0,0 +1,285 @@ +"""The MCP wiring. The only module here that imports the SDK. + +Transport is **stdio** and nothing else: one process per client, spawned by the +client, no port and no daemon. There is already a daemon in this system +(`hhstreamd`) and it owns all the state; a second long-lived process in front of +it would be a second place for a lease to be remembered. + +The low-level :class:`mcp.server.Server` rather than FastMCP, for two reasons. +The schemas here are written by hand -- ``pointer_move``'s exclusive ``to``/``by`` +pair, ``additionalProperties: false`` on every tool -- and deriving them from +Python type hints would mean writing the signatures that happen to produce them. +And ``shot`` returns image content beside its text, which is a content list, not +a return value. + +**The transport does not stay on file descriptor 1**, and that is load-bearing +rather than tidy: the tools run the CLI's handlers *in this process* +(:mod:`hhstream_mcp.invoke`), those handlers ``print``, and ``record`` spawns an +``ffmpeg`` whose output the CLI does not capture. Any one of those writing on +fd 1 would put a line of prose in the middle of a JSON-RPC stream, which a +client reports as a parse failure with no hint of where it came from. So +:func:`claim_transport` takes a private duplicate of stdin and stdout before +anything else runs and then points fd 1 at the log and fd 0 at ``/dev/null``. +After that the protocol is on descriptors nothing else in the process knows +about, and a stray write is a log line. +""" + +from __future__ import annotations + +import argparse +import io +import logging +import os +import sys +from typing import Any + +import anyio +import mcp.types as types +from mcp.server.lowlevel import NotificationOptions, Server +from mcp.server.stdio import stdio_server + +from . import SERVER_NAME, __version__, tools +from .invoke import Runner + +log = logging.getLogger("hhstream.mcp") + +#: Shown by a client that surfaces server instructions, and the one place the +#: whole surface can be introduced at once. The two rules are repeated on the +#: individual tools, because a per-tool description is what a client shows at +#: the point of use. +INSTRUCTIONS = """\ +Drive this machine's headless Hyprland desktop: look at it, put applications on +it, and aim its one pointer and keyboard at them. + +Each tool is one verb of the `hhstream` CLI and does nothing the CLI would not. +Where to start: `status` says whether the desktop is up; `session_start` puts an +allowlisted application on it and returns a session id; `shot` shows you the +screen; `pointer_acquire` gives you the lease that every pointer and keyboard +tool needs. + +Two rules this surface will not let you avoid, because they are the two an agent +gets wrong: + +1. `session_touch` is the ONLY thing that resets a session's idle clock. + Listing, reading status, reading logs, collecting garbage, clicking and + typing all deliberately do NOT. A session being driven is still a session + nobody has said they want kept, and the broker reaps it on schedule. + +2. Every keyboard tool needs the POINTER lease, and holds only while that lease + holds. There is no keyboard lease: one compositor seat means one keyboard + focus. A held non-modifier key floods the window until you lift it or the + lease expires, because the application does the repeating. + +Failures come back as tool errors carrying the CLI's own exit code, its +machine-readable error name, its reason and what to do next. Exit 8 means +another session holds the pointer lease (`pointer_state` names the holder); exit +9 means you need to acquire one. +""" + + +def annotations_for(tool: tools.Tool) -> types.ToolAnnotations: + """Hints, and only honest ones. + + ``readOnlyHint`` is set from the verb's own behaviour, which is why + ``session_container`` does not get it: with a session id that verb runs + ``container.reconcile`` and writes the record back. ``openWorldHint`` is + false throughout -- the world here is one desktop on one machine. + """ + + return types.ToolAnnotations( + title=tool.title, + readOnlyHint=tool.read_only, + # Meaningful only when `readOnlyHint` is false, and then stated rather + # than left out: the spec's default for it is *true*, so omitting it + # would mark `session_touch` and `record` as possibly destructive. + destructiveHint=None if tool.read_only else tool.destructive, + openWorldHint=False, + ) + + +def describe(tool: tools.Tool) -> types.Tool: + return types.Tool( + name=tool.name, + title=tool.title, + description=tool.description, + inputSchema=tool.schema, + annotations=annotations_for(tool), + ) + + +def listing() -> list[types.Tool]: + return [describe(tool) for tool in tools.TOOLS] + + +def content_for(result: tools.Result) -> list[types.ContentBlock]: + blocks: list[types.ContentBlock] = [types.TextContent(type="text", text=result.text)] + if result.image is not None and result.media_type: + import base64 + + blocks.append( + types.ImageContent( + type="image", + data=base64.b64encode(result.image).decode("ascii"), + mimeType=result.media_type, + ) + ) + return blocks + + +def call(runner: Runner, name: str, arguments: dict[str, Any]) -> types.CallToolResult: + """Run one tool. A refusal comes back as a tool error, never as a success. + + ``isError`` rather than a JSON-RPC error, throughout. That is not only a + preference: the SDK's own ``call_tool`` handler catches whatever this raises + and turns it into an ``isError`` result anyway, and for the CLI's refusals + it is the right channel regardless -- "another session holds the lease" is + something the calling agent has to read and act on, and a JSON-RPC error + reaches the client's log rather than the agent. So the error text is + written for the agent, including for the two cases that are not the CLI's + verdict at all: a tool name that does not exist, and this layer's own bug. + """ + + tool = tools.BY_NAME.get(name) + if tool is None: + return types.CallToolResult( + content=[ + types.TextContent( + type="text", + text=( + f"no such tool: {name!r}. This server exposes: " + f"{', '.join(sorted(tools.BY_NAME))}" + ), + ) + ], + structuredContent={"ok": False, "error": "no_such_tool"}, + isError=True, + ) + try: + result = tool.invoke(runner, arguments) + except (OSError, ValueError, KeyError) as exc: + # `Runner.run` already turns a handler's own exception into exit 1 with + # its traceback, so what reaches here is this layer's own fault -- a + # schema and an argv builder that disagree about a field name. Reported + # to the agent rather than only logged, because an agent that is told + # nothing retries forever. + log.exception("tool %s failed", name) + result = tools.Result( + text=f"the {name} tool failed before the CLI was called: {exc!r}", + data={"ok": False, "error": "tool_failed"}, + is_error=True, + ) + return types.CallToolResult( + content=content_for(result), + structuredContent=result.data, + isError=result.is_error, + ) + + +def build(runner: Runner) -> Server: + server: Server = Server( + SERVER_NAME, version=__version__, instructions=INSTRUCTIONS + ) + + @server.list_tools() + async def _list() -> list[types.Tool]: + return listing() + + # `validate_input` left on: the schemas are the first refusal, and they + # refuse before an argv list exists. + @server.call_tool() + async def _call(name: str, arguments: dict[str, Any]) -> types.CallToolResult: + # The CLI call is blocking and can take seconds (`record`, a `start` + # with a readiness budget), so it goes to a worker thread: the event + # loop stays free to answer a ping and to read the next request. The + # calls themselves are serialised inside `Runner.run`, because the CLI + # prints to a stdout this process shares. + return await anyio.to_thread.run_sync(lambda: call(runner, name, arguments)) + + return server + + +def claim_transport() -> tuple[Any, Any]: + """Take stdin and stdout for the protocol, then make fd 0 and 1 harmless. + + Returns the pair of streams to hand to ``stdio_server``. After this call + the process's own ``sys.stdout`` writes to the log beside ``sys.stderr``, + and nothing that reads stdin can eat a request: both are the right answer + for a server whose tools run a CLI in-process. + """ + + stdin = anyio.wrap_file( + io.TextIOWrapper(os.fdopen(os.dup(0), "rb"), encoding="utf-8", errors="replace") + ) + stdout = anyio.wrap_file( + io.TextIOWrapper(os.fdopen(os.dup(1), "wb"), encoding="utf-8") + ) + sys.stdout.flush() + # The literal descriptor 2, not `sys.stderr.fileno()`: what is wanted is + # "whatever the log is at the file-descriptor level", so that a grandchild + # which never heard of `sys` writes there too. + os.dup2(2, 1) + null = os.open(os.devnull, os.O_RDONLY) + try: + os.dup2(null, 0) + finally: + os.close(null) + return stdin, stdout + + +async def serve(server: Server, transport: tuple[Any, Any] | None = None) -> None: + stdin, stdout = transport if transport is not None else claim_transport() + async with stdio_server(stdin, stdout) as (read_stream, write_stream): + await server.run( + read_stream, + write_stream, + server.create_initialization_options(NotificationOptions()), + ) + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + prog="hhstream-mcp", + description=( + "Serve the hhstream CLI as Model Context Protocol tools over stdio. " + "Spawned by an MCP client; it speaks JSON-RPC on stdin/stdout and " + "logs to stderr." + ), + epilog=( + "There is no --socket flag here, deliberately: the sockets are the " + "CLI's own environment -- $HHSTREAM_FRAME_SOCKET for the frame " + "socket and $HHSTREAM_SOCKET for the broker's control socket -- and " + "a second place to configure them is a second place for them to " + "disagree. Set them in the MCP client's `env` block." + ), + ) + parser.add_argument( + "--list-tools", + action="store_true", + help="print the tool names and exit, without serving. For checking an install.", + ) + return parser + + +def main(argv: list[str] | None = None) -> int: + args = build_parser().parse_args(argv) + # stderr, never stdout: stdout is the transport, and one stray print on it + # is a protocol error the client reports as a parse failure. + logging.basicConfig( + level=os.environ.get("HHSTREAM_MCP_LOG", "WARNING").upper(), + stream=sys.stderr, + format="%(levelname)s %(name)s: %(message)s", + ) + if args.list_tools: + for tool in tools.TOOLS: + print(f"{tool.name}\t{tool.title}") + return 0 + log.info("serving %d tools over stdio", len(tools.TOOLS)) + # The transport is claimed before the event loop starts, so no coroutine can + # have written to fd 1 by the time it moves. + transport = claim_transport() + anyio.run(serve, build(Runner()), transport) + return 0 + + +if __name__ == "__main__": # pragma: no cover + raise SystemExit(main()) diff --git a/src/hhstream_mcp/tools.py b/src/hhstream_mcp/tools.py new file mode 100644 index 0000000..5756fda --- /dev/null +++ b/src/hhstream_mcp/tools.py @@ -0,0 +1,1263 @@ +"""The tool table: one tool per CLI verb. Standard library only. + +**Granularity, and why.** One tool per verb rather than one tool per family +with an ``action`` enum. Three reasons, in order of weight: + +1. The arguments do not rhyme. ``shot`` takes an output name, ``keyboard_type`` + takes a string, ``pointer_move`` takes a coordinate pair that is exclusive + with a delta pair. Folded into one schema those become a union of optional + fields that is valid for every combination and correct for almost none, and + the refusal moves from the schema (before anything runs) to the broker + (after a round trip). +2. The two rules an agent gets wrong have to be *on the tool it gets wrong*. + A client shows a tool's own description at the point of use; a family-level + note about the idle clock is not what is read when ``session_list`` is about + to be called for the ninth time. +3. ``readOnlyHint`` is per tool and is a real distinction here: ``session_list`` + reads, ``session_kill`` destroys, and ``session_container`` with an id + reconciles -- which is a write, and is marked as one. + +The cost is twenty-six tools in one listing. That is paid back by the names +being the CLI's own: an agent that has read ``hhstream session --help``, or this +repository's README, already knows what ``session_touch`` is, and a transcript +of tool calls can be re-run by hand as a shell session. + +**Nothing here decides anything.** Every function below builds an argv list and +hands back what the CLI said; :mod:`hhstream_mcp.invoke` is what runs it. The +one place this layer adds words of its own is the remedy sentence per exit code +(also in that module) and the notes below, which are the CLI's own epilogues +restated in terms of tools rather than command lines. +""" + +from __future__ import annotations + +import dataclasses +import os +import pathlib +import tempfile +from typing import Any, Callable + +# The button and axis vocabularies, from the CLI that owns them. Imported and +# not retyped: `hhstream_session` mirrors the driver's lists with a comment +# saying the driver stays the authority, and a third copy here would be a third +# thing to update when the driver learns a button. +from hhstream_session import AXES, BUTTONS + +from . import media +from .invoke import GENERIC_FAILURE, Outcome, Runner, remedy_for, temp_name + +# ── the two rules, in an agent's words ───────────────────────────────────── +# +# These are `hhstream_session.ACTIVITY_NOTE` and `KEYBOARD_LEASE_NOTE` said in +# terms of tools instead of command lines. Not imported, because the point of +# both notes is to tell the reader what to *call next*, and "run 'hhstream +# session touch '" is not something an MCP client can do. The CLI's +# constants remain the source of the rule; `tests/unit/test_mcp_tools.py` holds +# these against them so a rule that changes there cannot silently stay here. + +ACTIVITY_NOTE = ( + "ACTIVITY: `session_touch` is the ONLY thing that resets a session's idle " + "clock. `session_list`, `session_status`, `session_logs` and `session_gc` " + "do NOT count, and neither does any pointer or keyboard tool -- if polling " + "kept a session alive, an abandoned session would never be reaped. A " + "session being watched or driven is still a session nobody has said they " + "want kept. If you leave an application running, call `session_touch` " + "before its idle timeout expires or the broker will reap it." +) + +LEASE_NOTE = ( + "LEASE: this tool needs the POINTER lease and fails with exit 9 (NO_LEASE) " + "until you have called `pointer_acquire` for this session; pass the " + "`lease_id` it returned as `lease`. There is no keyboard lease of its own: " + "the compositor has one seat and one keyboard focus, so a second lease " + "would let another session's pointer warp retarget your keys mid-string. " + "The lease holds only while it holds -- the broker force-releases an idle " + "one on its TTL (30 s by default), and every pointer and keyboard call " + "refreshes that deadline. `pointer_release`, and a timeout, lift held keys " + "as well as held buttons." +) + +HOLD_NOTE = ( + "HOLDING A NON-MODIFIER FLOODS THE WINDOW. The compositor sends one key " + "event and publishes a repeat rate; the APPLICATION does the repeating, so " + "a held `a` inserts characters until the matching `keyboard_up` or until " + "the lease TTL expires -- measured on this build at 25 a second after " + "600 ms. This server cannot rate-limit it, because it does not generate " + "the repeats. The only safe thing to hold is a modifier, which is what " + "chords are for: `keyboard_down Control_L`, `keyboard_type c`, " + "`keyboard_up Control_L`." +) + +COORDINATES_NOTE = ( + "Coordinates are OUTPUT-NORMALISED 0..1 inside the session's own " + "container: (0,0) is its top-left corner and (1,1) its bottom-right. They " + "are not pixels, and not pixels of the image `shot` returned." +) + + +# ── what a tool hands back ───────────────────────────────────────────────── + + +@dataclasses.dataclass(frozen=True) +class Result: + """A tool's outcome, in terms that do not mention MCP. + + ``server.py`` turns this into content blocks. Keeping the two apart is what + lets the whole table be tested without the SDK installed. + """ + + text: str + data: dict[str, Any] | None = None + image: bytes | None = None + media_type: str | None = None + is_error: bool = False + + +Invoke = Callable[[Runner, dict[str, Any]], Result] + + +@dataclasses.dataclass(frozen=True) +class Tool: + name: str + title: str + description: str + schema: dict[str, Any] + invoke: Invoke + read_only: bool = False + destructive: bool = False + + +# ── argv construction ────────────────────────────────────────────────────── + + +def flag(name: str, value: Any) -> str: + """``--name=value`` as one token. + + The ``=`` form everywhere, not only where it is needed. ``--by -0.1,0.5`` + and ``--amount -3`` are values that begin with a dash, which argparse reads + as option strings unless they are a bare negative number; ``--by=-0.1,0.5`` + is unambiguous for every value, including the ones an agent sends that + nobody predicted. One rule is also one fewer thing to get wrong per flag. + """ + + return f"--{name}={value}" + + +def argv(verb: list[str], options: list[str], positionals: list[str]) -> list[str]: + """``verb options -- positionals``. + + The literal ``--`` goes in whenever there is a positional, for the same + reason the ``=`` form does: ``keyboard_type`` with a string that starts with + a dash is a reasonable thing to ask for, and without the separator argparse + rejects it before the broker ever sees it. + """ + + built = [*verb, *options] + if positionals: + built.append("--") + built.extend(positionals) + return built + + +def point(pair: Any) -> str: + """``[x, y]`` as the CLI's ``X,Y``. + + ``repr`` of a float rather than a format with a width: ``{:g}`` rounds to + six significant digits, and a caller that computed a coordinate deserves to + have the broker refuse or accept the number it actually sent. + """ + + x, y = pair + return f"{float(x)!r},{float(y)!r}" + + +def number(value: Any) -> str: + return repr(float(value)) + + +def _optional(args: dict[str, Any], name: str, as_flag: str | None = None) -> list[str]: + """The flag for ``name`` if the caller gave one, else nothing. + + Omission matters: a flag left off is the CLI's (or the broker's) own + default, and this server must not have an opinion about what that is. + """ + + if name not in args or args[name] is None: + return [] + return [flag(as_flag or name.replace("_", "-"), args[name])] + + +def _switch(args: dict[str, Any], name: str, as_flag: str | None = None) -> list[str]: + return [f"--{as_flag or name.replace('_', '-')}"] if args.get(name) else [] + + +# ── result mapping ───────────────────────────────────────────────────────── + +Summary = Callable[[Outcome, dict[str, Any] | None], str] + + +def report(outcome: Outcome, *, summary: Summary | None = None) -> Result: + """One ``Outcome`` as a ``Result``. A refusal is never a success. + + The successful text is the CLI's own stdout, which for every ``--json`` + verb *is* the protocol object: passing it through unchanged is the whole + point of a shim, and an agent that wants a field reads the field. A few + tools prepend one line (the lease id, the new session id) because that line + is what the next call needs. + """ + + payload = outcome.payload() + if outcome.ok: + text = summary(outcome, payload) if summary else outcome.stdout.strip() + return Result(text=text or "(the command succeeded and printed nothing)", data=payload) + return Result(text=failure_text(outcome, payload), data=failure_data(outcome, payload), is_error=True) + + +def failure_text(outcome: Outcome, payload: dict[str, Any] | None) -> str: + """A refusal, in the order an agent needs it: what to do, then what was said.""" + + lines = [remedy_for(outcome.returncode), ""] + if payload: + for key in ("error", "reason", "hint"): + value = payload.get(key) + if value: + lines.append(f"{key}: {value}") + said = (outcome.stderr or outcome.stdout).strip() + if said: + lines += ["", "hhstream said:", said] + return "\n".join(lines) + + +def failure_data(outcome: Outcome, payload: dict[str, Any] | None) -> dict[str, Any]: + data: dict[str, Any] = dict(payload or {}) + data["ok"] = False + data["exit_code"] = outcome.returncode + data["remedy"] = remedy_for(outcome.returncode) + return data + + +def json_verb( + verb: list[str], + *, + options: Callable[[dict[str, Any]], list[str]] = lambda args: [], + positionals: Callable[[dict[str, Any]], list[str]] = lambda args: [], + summary: Summary | None = None, +) -> Invoke: + """The common case: a ``session`` verb run with ``--json``. + + ``--json`` on every one of them deliberately. The human rendering is a + table for a terminal; the protocol object is what the broker actually said, + including the machine-readable ``error`` name on a refusal, and scraping a + table for it would be inventing a second wire format. + """ + + def run(runner: Runner, args: dict[str, Any]) -> Result: + built = argv(verb, ["--json", *options(args)], positionals(args)) + return report(runner.run(built), summary=summary) + + return run + + +# ── schema fragments ─────────────────────────────────────────────────────── + +SESSION_ID = { + "type": "string", + "description": "the session id, as `session_start` printed it or `session_list` reports it", +} +LEASE = { + "type": "string", + "description": ( + "the `lease_id` that `pointer_acquire` returned for this session. " + "Required: this server holds no lease state, deliberately, so that " + "which lease a call used is visible in the call." + ), +} +DURATION = { + "type": "string", + "description": "a duration: `30s`, `5m`, `3h`, `2d`, or a bare number of seconds", +} +REQUEST_ID = { + "type": "string", + "description": ( + "deduplication token. Retrying a mutation with the same token returns " + "the original reply instead of doing it twice -- which is what makes a " + "timed-out call safe to retry." + ), +} +BUTTON = { + "type": "string", + "enum": list(BUTTONS), + "description": "mouse button name (default: left)", +} + + +def obj(properties: dict[str, Any], required: list[str] | None = None, **extra: Any) -> dict[str, Any]: + """An object schema that refuses what it does not know. + + ``additionalProperties: false`` so a flag this server does not pass through + is refused by the schema rather than silently dropped on the way to an argv + list -- a dropped argument is a tool that reports success for something it + did not do. + """ + + schema: dict[str, Any] = { + "type": "object", + "properties": properties, + "additionalProperties": False, + } + if required: + schema["required"] = required + schema.update(extra) + return schema + + +# ── frames ───────────────────────────────────────────────────────────────── + + +def _status(runner: Runner, args: dict[str, Any]) -> Result: + # No `--json`: `hhstream status` has none. It is a human report of the + # units, the socket's reachability and one probe frame, and this passes it + # through as text rather than inventing a shape for it. + return report(runner.run(["status"])) + + +def _shot(runner: Runner, args: dict[str, Any]) -> Result: + """One frame, as MCP image content. + + Captured to a temporary file and read back, because that is the CLI's only + output shape -- `shot` writes a file and prints its path -- and because a + pipe would mean asking the CLI for something it does not offer. The + directory goes away with the call; the bytes are in the reply. + """ + + with tempfile.TemporaryDirectory(prefix="hhstream-mcp-shot-") as scratch: + destination = os.path.join(scratch, "frame.png") + options = [flag("out", destination), *_optional(args, "output")] + outcome = runner.run(argv(["shot"], options, [])) + if not outcome.ok: + return report(outcome) + try: + frame = pathlib.Path(destination).read_bytes() + except OSError as exc: + return Result( + text=( + f"hhstream shot reported success but wrote no readable file: {exc}\n\n" + f"hhstream said:\n{outcome.stdout.strip()}" + ), + data={"ok": False, "exit_code": 0, "error": "no_artifact"}, + is_error=True, + ) + + image = media.sniff(frame) + where = args.get("output") + scope = f"output {where!r}" if where else "the published main output" + if image is None: + # Handed back anyway, with the truth about it: the bytes are what the + # compositor produced, and a client that can decode them should get the + # chance. Guessing a media type would be worse than saying so. + return Result( + text=( + f"{len(frame)} bytes from {scope}, in a format this server does " + "not recognise (not PNG, not JPEG), so no geometry and no media " + "type can be reported. The capture backend may be misconfigured." + ), + data={"ok": True, "bytes": len(frame), "output": where}, + is_error=False, + ) + return Result( + text=( + f"{image.width}x{image.height} {image.media_type}, {image.size} bytes, " + f"from {scope}. {COORDINATES_NOTE}" + ), + data={ + "ok": True, + "width": image.width, + "height": image.height, + "media_type": image.media_type, + "bytes": image.size, + "output": where, + }, + image=frame, + media_type=image.media_type, + ) + + +def _record(runner: Runner, args: dict[str, Any]) -> Result: + """Film the screen and hand back the *path*. + + Not the bytes. An mp4 of a 720p desktop is megabytes, and a tool reply is + a message in a model's context -- so the artefact stays on disk and the + reply names it, which is also what makes it watchable by a human. + """ + + destination = args.get("out") or temp_name("recording", ".mp4") + options = [ + flag("out", destination), + *_optional(args, "seconds"), + *_optional(args, "output"), + *_optional(args, "fps"), + *_optional(args, "work_dir", "work-dir"), + *_switch(args, "keep"), + ] + outcome = runner.run(argv(["record"], options, [])) + if not outcome.ok: + return report(outcome) + measured = media.recording(outcome.stdout) + if measured is None: + return Result( + text=( + "hhstream record succeeded but its output could not be parsed " + f"for the measured rate:\n{outcome.stdout.strip()}" + ), + data={"ok": True, "path": destination}, + ) + return Result( + text=( + f"{measured.path}\n{measured.frames} frames in {measured.seconds:.2f}s, " + f"encoded at the measured {measured.fps} fps (the publisher's achieved " + "rate, not the configured one). The file is on the machine running " + "this server; nothing was copied into this reply." + ), + data={ + "ok": True, + "path": measured.path, + "frames": measured.frames, + "seconds": measured.seconds, + "fps": measured.fps, + }, + ) + + +#: How long `run` watches the process it spawned before reporting its pid. +DEFAULT_SETTLE_S = 1.5 + + +def _run(runner: Runner, args: dict[str, Any]) -> Result: + """Launch an unmanaged application. Forked and left alone. + + ``cli.cmd_run`` cannot be called in this process -- it ends in + ``os.execvpe``, which is the verb -- so :meth:`Runner.launch` does the one + thing it does, around a double fork. See that method for why the pid + handed back is the application's own and not a wrapper's. + """ + + command = [str(item) for item in args["argv"]] + settle = float(args.get("settle_s", DEFAULT_SETTLE_S)) + spawned = runner.launch(command, settle=settle) + if spawned.exited: + said = (spawned.stderr or spawned.stdout).strip() + return Result( + text=( + f"{remedy_for(GENERIC_FAILURE)}\n\n" + f"the application did not survive {settle:g}s, so nothing is on " + f"the screen.\n\nit said:\n{said or '(nothing)'}" + ), + data={ + "ok": False, + "error": "run_exited", + "pid": spawned.pid or None, + "remedy": remedy_for(GENERIC_FAILURE), + }, + is_error=True, + ) + return Result( + text=( + f"launched pid {spawned.pid}: {' '.join(command)}\n" + f"Still running after {settle:g}s. This is UNMANAGED: nothing tracks " + "it, it has no idle clock and no container, and `session_kill` " + "cannot stop it -- use `session_start` instead when it must be " + "findable, kept alive and stoppable by name, or `session_adopt` to " + "bring this pid under management." + ), + data={"ok": True, "pid": spawned.pid, "argv": command, "managed": False}, + ) + + +# ── the table ────────────────────────────────────────────────────────────── + + +def _acquire_summary(outcome: Outcome, payload: dict[str, Any] | None) -> str: + lease = (payload or {}).get("lease_id") or "(none reported)" + return ( + f"lease_id: {lease}\n" + "Pass this as `lease` on every pointer and keyboard call for this " + "session, and call `pointer_release` when the gesture is finished. The " + "keyboard is on this same lease; acquire also took keyboard focus.\n\n" + f"{outcome.stdout.strip()}" + ) + + +def _start_summary(outcome: Outcome, payload: dict[str, Any] | None) -> str: + session_id = (payload or {}).get("id") or "(not reported)" + return ( + f"session id: {session_id}\n" + f"Keep it alive with `session_touch` before its idle timeout expires.\n\n" + f"{outcome.stdout.strip()}" + ) + + +def build() -> tuple[Tool, ...]: + """The whole surface. A function, so the table is built once per process.""" + + tools: list[Tool] = [ + Tool( + name="status", + title="Session status", + description=( + "Is the headless Hyprland session up, and can this server reach " + "its frame socket? Reports each unit's state, the socket path " + "and whether it is readable, the Wayland socket the compositor " + "bound, and one probe frame's size and geometry. Start here " + "when any other tool fails in a way that smells like the " + "desktop being down. Human-readable text; this verb has no " + "JSON form." + ), + schema=obj({}), + invoke=_status, + read_only=True, + ), + Tool( + name="shot", + title="Capture one frame", + description=( + "Capture one frame and return it as an image. Reads the " + "published main output off the frame socket; with `output` " + "named it captures that one output in isolation through `grim` " + "instead, which is how a single session's container is looked " + "at (`session_container` lists the names). The geometry and " + "media type are in the text part. " + f"{COORDINATES_NOTE}" + ), + schema=obj( + { + "output": { + "type": "string", + "description": ( + "capture this output alone, e.g. a session's " + "container `hh-`. Omit for the " + "published main output." + ), + } + } + ), + invoke=_shot, + read_only=True, + ), + Tool( + name="record", + title="Film the screen", + description=( + "Film the screen for N seconds into an mp4 and return its " + "PATH, not its bytes. The encoded frame rate is the one the " + "capture actually measured, not the configured one -- a " + "software-rendered compositor tops out well below it, and the " + "reply reports both the frame count and the elapsed time. With " + "`output` named it films that output alone by running `grim` " + "once per frame, which is markedly slower. The file is written " + "on the machine running this server." + ), + schema=obj( + { + "seconds": { + "type": "number", + "exclusiveMinimum": 0, + "description": "how long to film (the CLI's default is 10)", + }, + "out": { + "type": "string", + "description": ( + "where to write the mp4. Omitted, it goes to a " + "generated path under the system temporary " + "directory and the reply names it." + ), + }, + "output": { + "type": "string", + "description": "film this output alone, e.g. `hh-`", + }, + "fps": { + "type": "number", + "exclusiveMinimum": 0, + "description": ( + "target frames per second, for `output` only; " + "ignored without it, where the publisher sets the rate" + ), + }, + "work_dir": { + "type": "string", + "description": "where the intermediate frames go (default: a temp dir)", + }, + "keep": { + "type": "boolean", + "description": "keep the intermediate frames after encoding", + }, + } + ), + invoke=_record, + # Not read-only: it writes an mp4. Additive and not destructive -- + # it creates a file, it does not change the desktop -- but a hint + # that said "modifies nothing" about a tool that leaves megabytes + # on disk would be a hint worth less than none. + ), + Tool( + name="run", + title="Launch an unmanaged application", + description=( + "Launch a GUI application into the session and forget it. " + "UNMANAGED on purpose: nothing tracks the process, it gets no " + "container and no idle clock, and no tool here can stop it by " + "name. Prefer `session_start`, which registers the " + "application, gives it its own output and workspace, and can " + "be killed and reaped -- `run` is the primitive underneath " + "that, and is the right choice only for something you will " + "stop yourself. The pid returned is the application's own: the " + "CLI `exec`s into it." + ), + schema=obj( + { + "argv": { + "type": "array", + "items": {"type": "string"}, + "minItems": 1, + "description": ( + "the command and its arguments, as a list. There " + "is never a shell: no globbing, no pipes, no quoting." + ), + }, + "settle_s": { + "type": "number", + "minimum": 0, + "description": ( + "how long to watch the spawned process for an " + "immediate failure before reporting its pid " + f"(default: {DEFAULT_SETTLE_S:g}). A launch that " + "fails -- no Wayland socket, no such executable -- " + "fails inside this window and is reported as an " + "error; one that survives it is reported as running." + ), + }, + }, + ["argv"], + ), + invoke=_run, + ), + ] + + tools += _session_tools() + tools += _pointer_tools() + tools += _keyboard_tools() + return tuple(tools) + + +def _session_tools() -> list[Tool]: + return [ + Tool( + name="session_start", + title="Start a tracked session", + description=( + "Launch an allowlisted application as a tracked session: it " + "gets a session id, its own output and workspace, a pid the " + "broker watches, and an idle clock. `app` is a NAME from the " + "broker's allowlist, never a path -- an unlisted name is " + "refused by policy (exit 6), and so is an argument the " + "allowlist does not permit.\n\n" + f"{ACTIVITY_NOTE}" + ), + schema=obj( + { + "app": { + "type": "string", + "description": "a name from the broker's allowlist, never a path", + }, + "argv": { + "type": "array", + "items": {"type": "string"}, + "description": ( + "arguments for the application, as a list. Passed " + "as an argv list; there is never a shell." + ), + }, + "idle_timeout": dict( + DURATION, + description=( + "reap the session after this long with no " + "`session_touch` (the broker's default applies if " + "omitted)" + ), + ), + "wait_ready": dict( + DURATION, description="how long to wait for the application to signal readiness" + ), + "home": { + "type": "string", + "description": "run the application with this HOME instead of the inherited one", + }, + "ephemeral_home": { + "type": "boolean", + "description": ( + "give the session a private HOME and delete it when " + "the session ends. Mutually exclusive with `home`." + ), + }, + "request_id": REQUEST_ID, + }, + ["app"], + # Mutually exclusive, spelled as "never both" rather than as a + # `oneOf`: a `oneOf` of two negations matches twice when + # NEITHER is given, which would refuse the ordinary call that + # names no home at all. + **{"not": {"required": ["home", "ephemeral_home"]}}, + ), + invoke=json_verb( + ["session", "start"], + options=lambda a: [ + *_optional(a, "idle_timeout", "idle-timeout"), + *_optional(a, "wait_ready", "wait-ready"), + *_optional(a, "home"), + *_switch(a, "ephemeral_home", "ephemeral-home"), + *_optional(a, "request_id", "request-id"), + ], + positionals=lambda a: [a["app"], *[str(x) for x in a.get("argv") or []]], + summary=_start_summary, + ), + ), + Tool( + name="session_list", + title="List sessions", + description=( + "Every session the broker is tracking, with its state, pid and " + "idle deadline. `all` includes the tombstones of ended ones.\n\n" + f"{ACTIVITY_NOTE}" + ), + schema=obj( + {"all": {"type": "boolean", "description": "include tombstones of ended sessions"}} + ), + invoke=json_verb(["session", "list"], options=lambda a: _switch(a, "all")), + read_only=True, + ), + Tool( + name="session_status", + title="One session, in detail", + description=( + "One session: its state, pid, container, idle deadline, and the " + "most recent entries from the broker's audit trail. The audit " + "events are the place to look when a session died for a reason " + "its own logs do not show.\n\n" + f"{ACTIVITY_NOTE}" + ), + schema=obj( + { + "id": SESSION_ID, + "events": { + "type": "integer", + "minimum": 0, + "description": "how many recent audit events to include (the CLI's default is 20)", + }, + }, + ["id"], + ), + invoke=json_verb( + ["session", "status"], + options=lambda a: _optional(a, "events"), + positionals=lambda a: [a["id"]], + ), + read_only=True, + ), + Tool( + name="session_touch", + title="Reset the idle clock", + description=( + "Reset this session's idle clock. Injects nothing and changes " + "nothing else.\n\n" + f"{ACTIVITY_NOTE}" + ), + schema=obj({"id": SESSION_ID}, ["id"]), + invoke=json_verb(["session", "touch"], positionals=lambda a: [a["id"]]), + ), + Tool( + name="session_kill", + title="Kill a session", + description=( + "Terminate a session and everything it spawned, and tear its " + "container down. The reply says what was actually signalled, " + "and WARNS if any process or any window survived -- read it, " + "because a kill that did not work looks the same as one that " + "did until you do." + ), + schema=obj( + { + "id": SESSION_ID, + "grace": dict( + DURATION, + description="how long to wait after the first signal before SIGKILL", + ), + "signal": { + "type": "string", + "description": "the first signal to send (the CLI's default is TERM)", + }, + "request_id": REQUEST_ID, + }, + ["id"], + ), + invoke=json_verb( + ["session", "kill"], + options=lambda a: [ + *_optional(a, "grace"), + *_optional(a, "signal"), + *_optional(a, "request_id", "request-id"), + ], + positionals=lambda a: [a["id"]], + ), + destructive=True, + ), + Tool( + name="session_gc", + title="Prune and reconcile the registry", + description=( + "Prune the tombstones of ended sessions and reconcile the " + "registry against `/proc`, so a session whose process died " + "without the broker noticing is recorded as gone.\n\n" + f"{ACTIVITY_NOTE}" + ), + schema=obj({"request_id": REQUEST_ID}), + invoke=json_verb( + ["session", "gc"], options=lambda a: _optional(a, "request_id", "request-id") + ), + ), + Tool( + name="session_adopt", + title="Adopt a running process", + description=( + "Bring an already-running process under management, so it has " + "a session id, an idle clock and a name to kill it by. Its exit " + "code will never be available -- it is not a child of the " + "broker -- so the broker watches `/proc` for it instead." + ), + schema=obj( + { + "pid": {"type": "integer", "minimum": 1, "description": "the pid to adopt"}, + "app": { + "type": "string", + "description": "name to record (default: the executable's basename)", + }, + "idle_timeout": dict(DURATION, description="reap it after this long with no touch"), + "request_id": REQUEST_ID, + }, + ["pid"], + ), + invoke=json_verb( + ["session", "adopt"], + options=lambda a: [ + flag("pid", int(a["pid"])), + *_optional(a, "app"), + *_optional(a, "idle_timeout", "idle-timeout"), + *_optional(a, "request_id", "request-id"), + ], + ), + ), + Tool( + name="session_logs", + title="A session's captured output", + description=( + "The tail of the application's own stdout and stderr, as the " + "broker captured it. The first place to look when an " + "application started and then did nothing.\n\n" + "There is no follow mode here, deliberately: the CLI's " + "`--follow` polls until interrupted, and a tool call that never " + "returns hangs the client that made it. Call this again " + "instead.\n\n" + f"{ACTIVITY_NOTE}" + ), + schema=obj( + { + "id": SESSION_ID, + "lines": { + "type": "integer", + "minimum": 1, + "description": "how many lines to tail (the CLI's default is 100)", + }, + }, + ["id"], + ), + invoke=json_verb( + ["session", "logs"], + options=lambda a: _optional(a, "lines"), + positionals=lambda a: [a["id"]], + ), + read_only=True, + ), + Tool( + name="session_container", + title="A session's output and workspace", + description=( + "The output, workspace and mode the broker gave a session -- " + "the output name is what `shot` and `record` take as `output`. " + "With no id this lists every container plus the `hh-*` outputs " + "no live session claims (orphans), and reads nothing but the " + "record.\n\n" + "WITH an id this re-reads from the compositor and brings the " + "record up to that answer, which is a write: asking where a " + "session's window is is almost always asking where it is NOW, " + "and the stored record can only say where it was put. Use no " + "id for a pure read.\n\n" + f"{ACTIVITY_NOTE}" + ), + schema=obj({"id": dict(SESSION_ID, description=SESSION_ID["description"] + "; omit for every container and the orphan list")}), + invoke=json_verb( + ["session", "container"], positionals=lambda a: [a["id"]] if a.get("id") else [] + ), + ), + Tool( + name="session_reconcile", + title="Re-read containers from the compositor", + description=( + "Ask the compositor where a session's window actually is and " + "bring the record up to that answer; the compositor is the " + "authority and the record is a cache of it. With no id every " + "live session is reconciled and the start-up scan is re-run: " + "the quarantine rule is re-installed, adopted containers " + "re-take their workspace slots, and `hh-*` outputs with no live " + "session are removed. The reply reports the drift it found.\n\n" + f"{ACTIVITY_NOTE}" + ), + schema=obj( + { + "id": dict(SESSION_ID, description=SESSION_ID["description"] + "; omit for all of them"), + "request_id": REQUEST_ID, + } + ), + invoke=json_verb( + ["session", "reconcile"], + options=lambda a: _optional(a, "request_id", "request-id"), + positionals=lambda a: [a["id"]] if a.get("id") else [], + ), + ), + ] + + +def _pointer_tools() -> list[Tool]: + gesture = f"{LEASE_NOTE}\n\n{COORDINATES_NOTE}\n\n{ACTIVITY_NOTE}" + return [ + Tool( + name="pointer_acquire", + title="Take the pointer lease", + description=( + "Take the one cursor for this session, aim it into the " + "session's own container, and return the `lease_id` every " + "pointer and keyboard call needs. It takes KEYBOARD FOCUS as " + "well as the cursor -- that is what makes typing possible, " + "since a virtual keyboard is seat-global and has nothing to " + "aim -- and the reply's `focus` field names which mechanism " + "took it.\n\n" + "The cursor is singular, so it is leased: a second acquire " + "while another session holds it fails with exit 8 (BUSY) and " + "names the holder. There is no way to take it from them. A " + "lease left idle past its TTL is force-released by the broker, " + "which also lifts whatever it was holding down.\n\n" + "By default the cursor resumes where this session last left " + "it; `no_restore` starts at the centre of the container " + "instead.\n\n" + f"{ACTIVITY_NOTE}" + ), + schema=obj( + { + "id": SESSION_ID, + "ttl": dict( + DURATION, + description=( + "idle lifetime for this lease before the broker " + "force-releases it (the broker's default applies if " + "omitted, and its hard maximum caps it)" + ), + ), + "no_restore": { + "type": "boolean", + "description": "start at the centre of the container rather than where it was left", + }, + }, + ["id"], + ), + invoke=json_verb( + ["session", "pointer", "acquire"], + options=lambda a: [*_optional(a, "ttl"), *_switch(a, "no_restore", "no-restore")], + positionals=lambda a: [a["id"]], + summary=_acquire_summary, + ), + ), + Tool( + name="pointer_move", + title="Warp the cursor", + description=( + "Warp the cursor inside this session's container. Exactly one " + "of `to` (absolute) or `by` (relative to where it is now). " + "`to` is REFUSED outside 0..1, because a caller that computed " + "1.4 has a bug and clamping would hide it; `by` IS clamped, " + "because running into the edge of the screen is what a mouse " + "does.\n\n" + f"{gesture}" + ), + schema=obj( + { + "id": SESSION_ID, + "lease": LEASE, + "to": { + "type": "array", + "items": {"type": "number"}, + "minItems": 2, + "maxItems": 2, + "description": "absolute [x, y], output-normalised 0..1", + }, + "by": { + "type": "array", + "items": {"type": "number"}, + "minItems": 2, + "maxItems": 2, + "description": "relative [dx, dy] from the current position", + }, + }, + ["id", "lease"], + oneOf=[{"required": ["to"]}, {"required": ["by"]}], + ), + invoke=json_verb( + ["session", "pointer", "move"], + options=lambda a: [ + flag("lease", a["lease"]), + flag("to", point(a["to"])) if "to" in a else flag("by", point(a["by"])), + ], + positionals=lambda a: [a["id"]], + ), + ), + Tool( + name="pointer_click", + title="Click", + description=( + "Press and release a button where the cursor already is. Warp " + "first with `pointer_move` if it is somewhere else -- there is " + "no click-at-a-coordinate, and the lease is what makes " + "move-then-click one gesture no other session can interleave " + "with.\n\n" + f"{gesture}" + ), + schema=obj({"id": SESSION_ID, "lease": LEASE, "button": BUTTON}, ["id", "lease"]), + invoke=json_verb( + ["session", "pointer", "click"], + options=lambda a: [flag("lease", a["lease"]), *_optional(a, "button")], + positionals=lambda a: [a["id"]], + ), + ), + Tool( + name="pointer_scroll", + title="Scroll", + description=( + "Scroll where the cursor is. `amount` is signed wheel clicks " + "and POSITIVE IS DOWN (vertical) or RIGHT (horizontal) -- the " + "protocol's direction, not a choice of ours, which is why there " + "is no up/down vocabulary over it. Bounded at 100 clicks a " + "call, so a decimal-point slip is a refusal rather than a " + "session scrolled somewhere unrecoverable. The cursor does not " + "move.\n\n" + f"{gesture}" + ), + schema=obj( + { + "id": SESSION_ID, + "lease": LEASE, + "axis": { + "type": "string", + "enum": list(AXES), + "description": "vertical (default) or horizontal", + }, + "amount": { + "type": "number", + "description": "wheel clicks, signed; positive is down/right (the CLI's default is 3)", + }, + }, + ["id", "lease"], + ), + invoke=json_verb( + ["session", "pointer", "scroll"], + options=lambda a: [ + flag("lease", a["lease"]), + *_optional(a, "axis"), + *([flag("amount", number(a["amount"]))] if "amount" in a else []), + ], + positionals=lambda a: [a["id"]], + ), + ), + Tool( + name="pointer_down", + title="Hold a button down", + description=( + "Half a click: press a button and leave it pressed. With " + "`pointer_move` and `pointer_up` this is a drag, and the lease " + "is what makes the three calls atomic. The broker lifts " + "whatever is still down on release or on a lease timeout, so a " + "crashed agent cannot leave a button pressed.\n\n" + f"{gesture}" + ), + schema=obj({"id": SESSION_ID, "lease": LEASE, "button": BUTTON}, ["id", "lease"]), + invoke=json_verb( + ["session", "pointer", "down"], + options=lambda a: [flag("lease", a["lease"]), *_optional(a, "button")], + positionals=lambda a: [a["id"]], + ), + ), + Tool( + name="pointer_up", + title="Release a held button", + description=( + "The other half of `pointer_down`. The reply lists what is " + "still held.\n\n" + f"{gesture}" + ), + schema=obj({"id": SESSION_ID, "lease": LEASE, "button": BUTTON}, ["id", "lease"]), + invoke=json_verb( + ["session", "pointer", "up"], + options=lambda a: [flag("lease", a["lease"]), *_optional(a, "button")], + positionals=lambda a: [a["id"]], + ), + ), + Tool( + name="pointer_release", + title="Give the lease back", + description=( + "Give the cursor back, lifting every button AND every key still " + "held. Call it when a gesture is finished rather than letting " + "the TTL expire: until it is released no other session can " + "acquire. The lease id is dead afterwards.\n\n" + f"{ACTIVITY_NOTE}" + ), + schema=obj({"id": SESSION_ID, "lease": LEASE}, ["id", "lease"]), + invoke=json_verb( + ["session", "pointer", "release"], + options=lambda a: [flag("lease", a["lease"])], + positionals=lambda a: [a["id"]], + ), + ), + Tool( + name="pointer_state", + title="Who holds the pointer", + description=( + "Who holds the one pointer lease, since when, and which buttons " + "and KEYS are currently held down. This is the tool to call " + "after an exit 8 (BUSY): it names the holder. It takes no " + "session id -- there is one cursor.\n\n" + f"{ACTIVITY_NOTE}" + ), + schema=obj({}), + invoke=json_verb(["session", "pointer", "state"]), + read_only=True, + ), + ] + + +def _keyboard_tools() -> list[Tool]: + note = f"{LEASE_NOTE}\n\n{ACTIVITY_NOTE}" + keysym = { + "type": "string", + "description": ( + "an XKB keysym NAME, not a character code: `a`, `A`, `Return`, " + "`BackSpace`, `Left`, `Control_L`, `Shift_L`. Resolved against the " + "layout the driver uploaded (an operator setting, `us` by default), " + "so a name that layout does not carry is exit 2." + ), + } + return [ + Tool( + name="keyboard_type", + title="Type a string", + description=( + "Type one string into whatever the seat has focused -- which " + "`pointer_acquire` aimed at this session. Raw UTF-8, no " + "backslash escapes: a newline in the string is a newline, and " + "`\\n` is a backslash and an n.\n\n" + "A CHARACTER THE LAYOUT CANNOT PRODUCE FAILS THE WHOLE STRING " + "and types nothing -- there is no partial type and no prefix to " + "clean up. On the default `us` layout that includes accented " + "letters and every C0 control but newline and tab. The refusal " + "names the character's INDEX, never the character and never the " + "string, because a typed string is a password as often as not; " + "the audit log records a count and nothing else. Capped at 4096 " + "Unicode scalars.\n\n" + "A modifier held with `keyboard_down` is a baseline: its mask " + "is ORed into every character here and it is still held " + "afterwards, which is how a chord is spelled. A non-modifier " + "left held is a refusal.\n\n" + f"{note}" + ), + schema=obj( + { + "id": SESSION_ID, + "lease": LEASE, + "text": { + "type": "string", + "description": "the string to type, raw UTF-8, at most 4096 Unicode scalars", + }, + }, + ["id", "lease", "text"], + ), + invoke=json_verb( + ["session", "keyboard", "type"], + options=lambda a: [flag("lease", a["lease"])], + positionals=lambda a: [a["id"], a["text"]], + ), + ), + Tool( + name="keyboard_key", + title="Tap one key", + description=( + "Press and release one key: down then up, as two calls on the " + "wire. This is the tool for `Return`, `Tab`, `BackSpace` and " + "the arrows -- anything `keyboard_type` cannot express because " + "it has no character. A refused down sends no up.\n\n" + f"{note}" + ), + schema=obj({"id": SESSION_ID, "lease": LEASE, "name": keysym}, ["id", "lease", "name"]), + invoke=json_verb( + ["session", "keyboard", "key"], + options=lambda a: [flag("lease", a["lease"])], + positionals=lambda a: [a["id"], a["name"]], + ), + ), + Tool( + name="keyboard_down", + title="Hold a key down", + description=( + "Press a key and leave it pressed. The justified use is a " + "chord.\n\n" + f"{HOLD_NOTE}\n\n" + f"{note}" + ), + schema=obj({"id": SESSION_ID, "lease": LEASE, "name": keysym}, ["id", "lease", "name"]), + invoke=json_verb( + ["session", "keyboard", "down"], + options=lambda a: [flag("lease", a["lease"])], + positionals=lambda a: [a["id"], a["name"]], + ), + ), + Tool( + name="keyboard_up", + title="Release a held key", + description=( + "Release a key that was held with `keyboard_down`. The reply " + "lists what is still held. `pointer_release` and a lease " + "timeout both lift everything, so a crashed agent cannot leave " + "a key down -- but until one of those happens the application " + "is still being typed at.\n\n" + f"{note}" + ), + schema=obj({"id": SESSION_ID, "lease": LEASE, "name": keysym}, ["id", "lease", "name"]), + invoke=json_verb( + ["session", "keyboard", "up"], + options=lambda a: [flag("lease", a["lease"])], + positionals=lambda a: [a["id"], a["name"]], + ), + ), + ] + + +#: The surface, built once. A tuple so a caller cannot reorder the listing an +#: agent reads. +TOOLS: tuple[Tool, ...] = build() + +BY_NAME: dict[str, Tool] = {tool.name: tool for tool in TOOLS}