Merge pull request 'feat(mcp): an HTTP transport, and the bearer token it exists for' (#5) from feat/mcp-auth into main

Reviewed-on: http://gitea.app.internal:3000/Hermenix/hyprland-headless-stream-plugin/pulls/5
This commit is contained in:
ari committed 2026-10-13 19:50:39 +00:00
commit 965485d8e5
23 files changed
+4153 -78

No files matched your search

+28 -3
View File
@@ -51,10 +51,21 @@ import the SDK:
```bash
nix develop
hhstream-mcp --list-tools # the 21 acting verbs
hhstream-mcp --list-tools # the 23 acting verbs
hhstream-mcp --list-resources # the 10 read-only URIs
# the HTTP transport, which is the only part that needs a credential
export HHSTREAM_MCP_STATE_DIR=$PWD/.mcp-state
hhstream-mcp token mint --label dev # the token, once, on stdout
hhstream-mcp serve --http # 127.0.0.1:9989/mcp
```
stdio is unauthenticated on purpose and the HTTP transport is not: a stdio
client spawns the server and so already holds the broker's socket, while a port
is reachable by someone who passed no permission check. `docs/CONTRACT-mcp.md`
§1 has the argument; `token mint` prints the secret once and the table on disk
keeps only its digest.
## The checks
| Check | What it proves | Needs a VM |
@@ -72,6 +83,8 @@ hhstream-mcp --list-resources # the 10 read-only URIs
| `vm-stream-firewall` | the stream's **port**: the guest's own running firewall accepts the configured port and not the module's default, a peer outside the guest reaches it, and is refused on a port beside it that was never opened | yes |
| `vm-mcp` | the **MCP server over stdio**: `mcp.enable` puts the binary on PATH and starts no unit, the `initialize` handshake completes over a real pipe, all 23 tools and 10 resources are listed with their annotations and their two rules, both a tool call and a resource read reach the live broker and return its own protocol object, two reads of a session do **not** move its idle deadline while `session_touch` does, a frame arrives as a blob at exactly its output's configured size, the pointer lease is granted/used/returned against the real driver, `--toolsets` narrows the tools and leaves the resources alone, and a refusal arrives as a tool error (or, for a read, a protocol error) carrying the frozen exit code | yes |
| `vm-mcp-http` | the **HTTP transport, its token and its unit**: the unit refusing to start while the authorization table is empty *and* saying so in the journal, then coming up by itself within one retry interval of the first `token mint` with no `systemctl start` — the half that nothing in Python can test; the state directory `0700` and owned by the account that mints tokens, the table `0600` and holding no token; the port on loopback only and **not** in the firewall; an unauthenticated and a wrong-token request both `401` with an empty body and no echo; a good token completing the handshake, reading a resource and calling a tool against the live broker; two tokens served by **one process** with different tool budgets; and a revocation taking effect on the next request with no restart | yes |
All of them at once:
```bash
@@ -96,6 +109,7 @@ nix build -L .#checks.x86_64-linux.vm-keyboard
nix build -L .#checks.x86_64-linux.vm-keyboard-de
nix build -L .#checks.x86_64-linux.vm-stream-firewall
nix build -L .#checks.x86_64-linux.vm-mcp
nix build -L .#checks.x86_64-linux.vm-mcp-http
nix build -L .#checks.x86_64-linux.vm-stream
```
@@ -125,11 +139,15 @@ output is on that path beside it, and `mcp` is in the test environment, because
— not optionally: a file that skipped when the SDK was absent would be green in
exactly the build where a malformed tool schema mattered.
The MCP suite is four files, split along what each can be tested without:
The MCP suite is six files, split along what each can be tested without:
`test_mcp_invoke.py` (the in-process CLI call and the exit-code mapping),
`test_mcp_tools.py` (the argv each tool builds, and the toolsets),
`test_mcp_resources.py` (URI routing, query parameters, and that a read never
writes) and `test_mcp_server.py` (the SDK wiring, over its memory transport).
writes), `test_mcp_authorizations.py` (the token store and the `token` verbs --
standard library, no SDK), `test_mcp_server.py` (the SDK wiring, over its
memory transport, plus the command line now that it has subcommands) and
`test_mcp_http.py` (the bearer check and the per-token toolsets, over a real
ASGI stack).
The check fails if the suite is **empty**, not just if it is red. `pytest` exits
5 on "collected nothing", but it exits 0 when every test was skipped or
@@ -264,11 +282,18 @@ checks/
vm-stream-firewall.nix nixosTest, the stream's port in the guest's firewall
vm-mcp.nix nixosTest, hhstream-mcp driven over stdio by a raw
JSON-RPC client against the live broker
vm-mcp-http.nix nixosTest, the HTTP transport: the unit's refusal while
no token exists and its self-heal after the first mint,
the 0700 state dir, the refusals, and two tokens with
different tool budgets served by one process
vm-stream.nix nixosTest, `session stream enable` and a real HTTP
viewer reading MJPEG from inside the guest
lib/mcp_client.py that client: hhs-mcp-call, stdlib only and deliberately
NOT the MCP SDK's -- the SDK on both ends would agree
with itself about a framing neither side had to get right
lib/mcp_http_client.py the same choice over HTTP: hhs-mcp-http, urllib only,
with a distinct exit status for an HTTP refusal so a
401 cannot be mistaken for a server that died
lib/test-tools.nix packages the tests/ helpers for the guest
lib/session-tools.nix packages the two guest programs below
lib/session-machine.nix the guest the three session checks share
+58 -5
View File
@@ -122,8 +122,8 @@ services.hyprland-headless-stream = {
};
```
Then register it with a client. The transport is **stdio** — one process per
client, spawned by the client, no port and no daemon — so the whole
Then register it with a client. The default transport is **stdio** — one
process per client, spawned by the client, no port and no daemon — so the whole
configuration is a command and an environment:
```json
@@ -161,9 +161,62 @@ what is offered to whichever families are wanted — `frames`, `sessions`,
```
Resources are deliberately *not* narrowed by it: they are the host's context,
not the model's tool budget. Nothing else about the server is configurable — it
holds no state and caches no lease, and the pointer lease is an explicit
argument on every gesture tool for exactly that reason.
not the model's tool budget. The server holds no state and caches no lease, and
the pointer lease is an explicit argument on every gesture tool for exactly that
reason.
### Over a port instead, for a client that cannot spawn a process
`mcp.http.enable` serves the same surface over Streamable HTTP, as a unit,
behind a bearer token:
```nix
services.hyprland-headless-stream.mcp.http.enable = true; # 127.0.0.1:9989/mcp
```
```console
$ sudo -u hermes hhstream-mcp token mint --label laptop --expires-in 90d
hhsmcp_…
minted a7k2qp (laptop), expires 2026-01-09T12:00:00Z, toolsets all.
This is the only time the token is shown.
```
```json
{
"mcpServers": {
"hhstream": {
"url": "http://127.0.0.1:9989/mcp",
"headers": { "Authorization": "Bearer hhsmcp_…" }
}
}
}
```
**stdio stays unauthenticated and that is not an oversight.** A stdio client
spawns the server, so it has already passed a filesystem permission check and
holds the broker's socket either way — a token there would be a lock whose key
is in the same pocket. A port is the first time the caller is someone who did
neither, which is the entire reason the token exists. The two options are
independent; neither implies the other.
Three things worth knowing before enabling it:
- **The unit refuses to start until a token exists**, and retries every five
seconds, so it comes up on its own moments after the first `token mint`. A
listener that `401`s every request is almost always a forgotten mint, and
`systemctl status hhstream-mcp` says so in as many words.
- **There is no TLS.** The token crosses the wire in clear text. Loopback is the
default and an SSH tunnel is the simplest way to reach it from elsewhere; a
LAN bind wants a reverse proxy, and the module warns about it.
- **It is not MCP's OAuth profile.** A client that can only discover an
authorization server will not connect; one that lets you set a header will.
See `docs/TODO/mcp-http-auth-is-not-oauth.md`.
`token list` shows the ids, labels and last use; `token revoke <id>` takes
effect on the next request with no restart. A token can carry its own
`--toolsets`, which is the only way to give two clients different tool budgets
from one process — though it narrows tools and *not* resources, which is
`docs/TODO/mcp-tokens-do-not-scope-resources.md`.
| Document | What it is |
|---|---|
+10
View File
@@ -2,6 +2,7 @@
#
# unit -- pytest over tests/, against the nix/package.nix build
# vm-mcp -- the MCP server over stdio, against a live broker
# vm-mcp-http -- the MCP server's HTTP transport, its token, and its unit
# vm-stream -- the per-session live stream, with a real HTTP viewer
# vm-basic -- nixosTest with a fake capture backend: module + wire contract
# vm-hyprland -- nixosTest with the real compositor and grim
@@ -82,6 +83,15 @@ in
# the others have no reason to carry.
vm-mcp = pkgs.callPackage ./vm-mcp.nix (sessionArgs // { inherit hhstream-mcp; });
# The same machine again with `mcp.http.enable`. A separate check and not a
# subtest of `vm-mcp`, because it proves a different thing: that one is about
# the surface over a pipe, this one is about a unit systemd supervises and a
# credential on the way in. Sharing a guest would also mean one failure
# hiding the other, and the first subtest here deliberately waits for a unit
# to FAIL -- which is not something to do in the middle of a check whose
# other assertions need it up.
vm-mcp-http = pkgs.callPackage ./vm-mcp-http.nix (sessionArgs // { inherit hhstream-mcp; });
# The live stream: `stream enable`, a real HTTP viewer inside the guest, and
# the one thing no in-process test can say -- that a connected viewer moves
# the idle clock. Spends real time on purpose; see the file's header.
+224
View File
@@ -0,0 +1,224 @@
#!/usr/bin/env python3
"""``hhs-mcp-http`` -- drive an MCP server over Streamable HTTP from a shell.
What ``checks/vm-mcp-http.nix`` uses, and the sibling of
``checks/lib/mcp_client.py``: the same deliberate choice not to use the MCP
SDK's own client, for the same reason. The SDK on both ends would agree with
itself about a framing and a header neither side had to get right, and the
header is half of what this check is about.
Standard library only -- ``urllib.request`` and ``json``. One process per
invocation, which re-does the ``initialize`` handshake every time; that is a
feature here, because the handshake behind a bearer token is one of the things
under test.
hhs-mcp-http --token T --initialize
hhs-mcp-http --token T --list
hhs-mcp-http --token T --resources
hhs-mcp-http --token T --read hhstream://sessions
hhs-mcp-http --token T session_gc
hhs-mcp-http --token T session_touch '{"id": "s-1"}'
hhs-mcp-http --initialize (no token: expect exit 4)
Exit codes, chosen so no assertion can mistake one failure for another:
==== ===============================================================
0 the server answered; whatever it said is on stdout as JSON
2 the server did not answer -- unreachable, timed out, or not JSON
3 the server answered with a JSON-RPC *error* object
4 an HTTP status other than 200 or 202; the status and the headers
the check cares about are on stdout as JSON
==== ===============================================================
Exit 4 is the interesting one and the reason this program exists rather than a
``curl`` pipeline: a 401 has an empty body, so the thing to assert is a status
code and a ``WWW-Authenticate`` header, and both have to come back in a shape a
test script can read without parsing curl's header dump.
"""
from __future__ import annotations
import argparse
import json
import sys
import urllib.error
import urllib.request
PROTOCOL_VERSION = "2025-06-18"
#: Both, in this order, because the specification requires a client to accept
#: either an immediate JSON body or an SSE stream. The server under test is run
#: with `--json-response`, so the body comes back as JSON -- but a client that
#: advertised only `application/json` would be testing a narrower contract than
#: the one a real client has to satisfy.
ACCEPT = "application/json, text/event-stream"
class NoAnswer(Exception):
"""The server did not answer in a way this client can read."""
class HttpStatus(Exception):
"""The server answered with a status other than 200."""
def __init__(self, status: int, headers: dict[str, str], body: bytes) -> None:
super().__init__(f"HTTP {status}")
self.status = status
self.headers = headers
self.body = body
def report(self) -> dict[str, object]:
"""What a test script needs, and deliberately not the body's bytes.
A refusal's body is empty and must stay empty; reporting its *length*
lets a check assert that without this program being the thing that
decides what "empty" means.
"""
return {
"http_status": self.status,
"www_authenticate": self.headers.get("www-authenticate"),
"cache_control": self.headers.get("cache-control"),
"body_len": len(self.body),
"body_prefix": self.body[:200].decode("utf-8", "replace"),
}
class Client:
"""One connection's worth of state: the token and the session id."""
def __init__(self, url: str, token: str | None, timeout: float) -> None:
self.url = url
self.token = token
self.timeout = timeout
self.session_id: str | None = None
def send(self, body: dict[str, object]) -> dict[str, object] | None:
payload = json.dumps(body).encode("utf-8")
headers = {"Content-Type": "application/json", "Accept": ACCEPT}
if self.token:
headers["Authorization"] = f"Bearer {self.token}"
if self.session_id:
headers["Mcp-Session-Id"] = self.session_id
request = urllib.request.Request(
self.url, data=payload, headers=headers, method="POST"
)
try:
with urllib.request.urlopen(request, timeout=self.timeout) as response:
raw = response.read()
lowered = {k.lower(): v for k, v in response.headers.items()}
status = response.status
except urllib.error.HTTPError as exc:
# The refusals arrive here: urllib raises on 4xx and 5xx, and the
# headers are on the exception.
raise HttpStatus(
exc.code,
{k.lower(): v for k, v in (exc.headers or {}).items()},
exc.read() or b"",
) from None
except (urllib.error.URLError, OSError, TimeoutError) as exc:
raise NoAnswer(f"no answer from {self.url}: {exc!r}") from None
# 200 for a request, 202 for a notification -- the specification's own
# answer to a message with no id, and it carries no body. Treating it
# as a refusal is a mistake worth naming here, because it looks exactly
# like one: the `notifications/initialized` that every handshake sends
# is answered 202, so a client that only accepts 200 fails on its
# second message with something that reads like an auth problem.
if status not in (200, 202):
raise HttpStatus(status, lowered, raw)
# Remembered from the `initialize` response and sent on every request
# after it; a stateful server refuses the second request without it.
if "mcp-session-id" in lowered:
self.session_id = lowered["mcp-session-id"]
# A notification is answered with 202 and no body.
if not raw:
return None
try:
return json.loads(raw)
except json.JSONDecodeError:
raise NoAnswer(f"not JSON-RPC from {self.url}: {raw[:400]!r}") from None
def request(self, method: str, params: dict[str, object] | None, request_id: int):
body: dict[str, object] = {"jsonrpc": "2.0", "id": request_id, "method": method}
if params is not None:
body["params"] = params
answer = self.send(body)
if answer is None:
raise NoAnswer(f"empty answer to {method}")
if "error" in answer:
raise SystemExit(_print_and(3, answer["error"]))
return answer.get("result")
def notify(self, method: str) -> None:
self.send({"jsonrpc": "2.0", "method": method})
def handshake(self) -> dict[str, object]:
result = self.request(
"initialize",
{
"protocolVersion": PROTOCOL_VERSION,
"capabilities": {},
"clientInfo": {"name": "hhs-mcp-http", "version": "0"},
},
1,
)
self.notify("notifications/initialized")
return result
def _print_and(code: int, obj: object) -> int:
print(json.dumps(obj, indent=2, sort_keys=True))
return code
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
prog="hhs-mcp-http",
description="Drive an MCP server over Streamable HTTP. Raw JSON-RPC, "
"standard library only; not the MCP SDK's client.",
)
parser.add_argument("--url", default="http://127.0.0.1:9989/mcp")
parser.add_argument("--token", default=None, help="the bearer token; omitted, none is sent")
parser.add_argument("--timeout", type=float, default=60.0)
parser.add_argument("--initialize", action="store_true", help="the handshake's own result")
parser.add_argument("--list", action="store_true", help="tools/list")
parser.add_argument("--resources", action="store_true", help="resources/list")
parser.add_argument("--read", metavar="URI", help="one resources/read")
parser.add_argument("tool", nargs="?", help="a tool name for one tools/call")
parser.add_argument("arguments", nargs="?", default="{}", help="its arguments, as JSON")
return parser
def main(argv: list[str] | None = None) -> int:
parser = build_parser()
args = parser.parse_args(argv)
client = Client(args.url, args.token, args.timeout)
try:
handshake = client.handshake()
if args.initialize:
return _print_and(0, handshake)
if args.list:
return _print_and(0, client.request("tools/list", None, 2))
if args.resources:
return _print_and(0, client.request("resources/list", None, 2))
if args.read:
return _print_and(0, client.request("resources/read", {"uri": args.read}, 2))
if args.tool:
try:
arguments = json.loads(args.arguments)
except json.JSONDecodeError as exc:
parser.error(f"arguments are not JSON: {exc}")
return _print_and(
0, client.request("tools/call", {"name": args.tool, "arguments": arguments}, 2)
)
except HttpStatus as refused:
return _print_and(4, refused.report())
except NoAnswer as exc:
print(str(exc), file=sys.stderr)
return 2
parser.error("nothing to do: give --initialize, --list, --resources, --read or a tool name")
return 2 # pragma: no cover -- parser.error exits
if __name__ == "__main__":
raise SystemExit(main())
+13
View File
@@ -32,11 +32,20 @@ let
# `mcp.types.Tool` objects is where a malformed schema or a name the spec
# refuses would show up, and a file that skipped when the SDK was absent
# would be green in exactly the build where that mattered.
# `starlette` and `httpx` beside `mcp`, and for the same reason the comment
# above gives: `tests/unit/test_mcp_http.py` drives the HTTP transport
# through `starlette.testclient`, which needs `httpx`. Both arrive through
# the SDK today -- measured -- and relying on that is the trap. An SDK
# release that moved its HTTP transport behind an optional extra would make
# 43 tests vanish from this gate, and a gate that loses tests silently is
# worse than one that fails.
pytestEnv = python3.withPackages (ps: [
ps.pytest
ps.pywayland
ps.xkbcommon
ps.mcp
ps.starlette
ps.httpx
]);
# Only the suite itself, so an editor's droppings or a local .pytest_cache
@@ -84,6 +93,10 @@ runCommand "hhstream-unit"
# so here, as for `hhstream`, the import comes from what Nix installed, and a
# packaging mistake is distinguishable from a test failure.
python3 -c 'import hhstream_mcp.tools as t; print("mcp tool table:", len(t.TOOLS), "tools")'
# The HTTP transport's stack, proved present here rather than discovered as a
# skip in the report: `test_mcp_http.py` imports `starlette.testclient` at
# module scope and is not allowed to skip.
python3 -c 'import starlette.testclient, hhstream_mcp.http; print("http transport imports")'
set +e
pytest suite \
+352
View File
@@ -0,0 +1,352 @@
# checks.<system>.vm-mcp-http -- the HTTP transport, its token, and the unit, in
# a guest with a live broker and a real compositor.
#
# The sibling of `vm-mcp.nix`, which proves the *surface* over stdio. This one
# proves nothing about the surface and everything about the two things stdio
# does not have: a unit that systemd supervises, and a credential on the way
# in. So the tool and resource assertions here are one of each -- just enough
# to show the request reached the broker through an authenticated HTTP session
# -- and the weight is on the refusals, on the state directory's permissions,
# and on the lifecycle of the unit itself.
#
# THE ONE THAT MATTERS MOST is the first subtest. The server exits 2 while the
# authorization table is empty, which puts a freshly provisioned machine in a
# failed state: that is the deliberate trade (a `systemctl status` naming the
# remedy beats a listener that 401s every request), and it is only a *good*
# trade because the unit retries rather than giving up, so minting the first
# token brings it up with no `systemctl start`. That self-heal is a property of
# `StartLimitIntervalSec = 0` in the module and nothing in the Python can test
# it. If it ever regresses, every machine that enables this option comes up
# dead and stays dead, and this is the only thing that would notice.
#
# The client is `lib/mcp_http_client.py`: raw JSON-RPC over `urllib`, not the
# MCP SDK's client, for the reason `lib/mcp_client.py` is not either -- the SDK
# on both ends would agree with itself about a framing and a header neither
# side had to get right, and the header is half of what this file is about.
#
# Habits copied from the other session checks: every wait is bounded, each
# subtest asserts once, and a failure dumps diagnostics before it propagates.
{
testers,
hhstream,
hhstream-mcp,
hhsModule,
testTools,
hyprland,
foot,
jq,
grim,
procps,
coreutils,
dejavu_fonts,
width ? 1280,
height ? 720,
sessionUser ? "hermes",
port ? 9989,
}:
testers.runNixOSTest (
{ pkgs, ... }:
let
tools = pkgs.callPackage ./lib/session-tools.nix { inherit hhstream; };
guest = import ./lib/session-machine.nix {
inherit
hhsModule
hhstream
hyprland
testTools
foot
jq
procps
grim
coreutils
dejavu_fonts
;
sessionTools = tools;
};
prelude = import ./lib/session-prelude.nix {
inherit hhstream coreutils sessionUser;
};
client = pkgs.runCommandLocal "hhs-mcp-http" { } ''
install -Dm755 ${./lib/mcp_http_client.py} $out/bin/hhs-mcp-http
sed -i '1s|.*|#!${pkgs.python3}/bin/python3|' $out/bin/hhs-mcp-http
$out/bin/hhs-mcp-http --help > /dev/null
# It must refuse to invent a request rather than printing nothing: a
# client that exits 0 having done nothing would make every subtest below
# vacuous.
if $out/bin/hhs-mcp-http --url http://127.0.0.1:1/mcp --initialize; then
echo "hhs-mcp-http succeeded against a port nothing is listening on" >&2
exit 1
fi
'';
in
{
name = "hhstream-vm-mcp-http";
globalTimeout = 1800;
nodes.machine = {
imports = [
(guest {
inherit width height sessionUser;
pointer = true;
apps.foot = {
exec = "${foot}/bin/foot";
argsAllow = [ ];
};
})
];
services.hyprland-headless-stream.mcp = {
enable = true;
package = hhstream-mcp;
http = {
enable = true;
inherit port;
# Left at the default loopback, and asserted to be loopback: a check
# that bound the wildcard would stop being able to prove the default
# is the safe one.
#
# `openFirewall` is left off, also deliberately, and also asserted:
# the option exists so that opening the port is an explicit act, and
# a check that opened it would never notice if it became implied.
# `--json-response` so the client reads one JSON body instead of
# parsing SSE. The client's convenience, not the server's default.
extraArgs = [ "--json-response" ];
};
};
environment.systemPackages = [ client ];
};
testScript =
prelude
+ ''
import shlex
HTTP = "${client}/bin/hhs-mcp-http"
URL = "http://127.0.0.1:${toString port}/mcp"
UNIT = "hhstream-mcp.service"
STATE = "/var/lib/hhstream-mcp"
TABLE = f"{STATE}/authorizations.json"
MCP = "${hhstream-mcp}/bin/hhstream-mcp"
def http(argv, token=None, timeout=120):
"""One `hhs-mcp-http` call that must succeed. Returns its JSON."""
auth = f"--token {shlex.quote(token)} " if token else ""
out = machine.succeed(
f"{HTTP} --url {URL} {auth}{argv}", timeout=timeout
)
try:
return json.loads(out)
except json.JSONDecodeError:
raise AssertionError(f"not JSON from `hhs-mcp-http {argv}`: {out!r}")
def refused(argv, token=None, timeout=120):
"""One call that must come back as an HTTP refusal (exit 4).
Exit 4 and not merely "it failed": the client uses 2 for "the
server did not answer", so this cannot pass because the unit died.
"""
auth = f"--token {shlex.quote(token)} " if token else ""
status, out = machine.execute(
f"{HTTP} --url {URL} {auth}{argv}", timeout=timeout
)
assert status == 4, (
f"expected an HTTP refusal (4) for `{argv}`, got {status}: {out}"
)
return json.loads(out)
def mint(label, extra=""):
"""`hhstream-mcp token mint` as the broker's user, as an operator would.
As that user and not as root, because the state directory is 0700
owned by it -- which means this is also the assertion that an
operator can actually issue a token on a machine configured this
way.
"""
out = machine.succeed(
as_user(f"{MCP} token mint --label {shlex.quote(label)} {extra}"),
timeout=120,
)
token = out.strip().splitlines()[-1]
assert token.startswith("hhsmcp_"), f"not a token: {out!r}"
return token
start_all()
bring_up("the compositor and broker")
with subtest("the unit refuses to run while no token exists"):
# The deliberate failure. `wait_until_fails` rather than a single
# check, because systemd restarts it every five seconds and the
# instant sampled matters otherwise.
machine.wait_until_fails(f"systemctl is-active {UNIT}", timeout=180)
status = machine.execute(f"systemctl status {UNIT}")[1]
journal = machine.succeed(f"journalctl -u {UNIT} --no-pager | tail -40")
assert "no tokens" in journal, (
"the unit did not say why it refused to start; an operator has "
f"nothing to act on. status was:\n{status}\njournal:\n{journal}"
)
# And nothing is listening, which is the point of refusing.
machine.fail("ss -ltn | grep -q ':${toString port}'")
with subtest("the state directory is the operator's and nobody else's"):
machine.wait_for_file(STATE, timeout=120)
mode = machine.succeed(f"stat -c %a {STATE}").strip()
owner = machine.succeed(f"stat -c %U {STATE}").strip()
assert mode == "700", f"{STATE} is {mode}, not 700"
assert owner == USER, f"{STATE} is owned by {owner}, not {USER}"
# The group is `sessions.group` and is deliberately not asserted
# here: at 0700 it grants nothing, and pinning it would make this
# subtest fail for a machine that merely named its group something
# else. What matters is that the owner is the account that mints
# tokens, and that nobody else can read the table at all.
with subtest("minting the first token brings the unit up by itself"):
# The self-heal. No `systemctl start` anywhere in this subtest, on
# purpose: if this needs one, the option is a trap on every freshly
# provisioned machine.
token = mint("the check")
machine.wait_for_unit(UNIT, timeout=180)
machine.wait_for_open_port(${toString port}, addr="127.0.0.1", timeout=120)
with subtest("the table is private and holds no token"):
mode = machine.succeed(f"stat -c %a {TABLE}").strip()
assert mode == "600", f"{TABLE} is {mode}, not 600"
raw = machine.succeed(f"cat {TABLE}")
assert token not in raw, "the authorization table contains the token itself"
assert '"hash"' in raw, f"no digest in the table: {raw!r}"
with subtest("the port is loopback only"):
listening = machine.succeed("ss -ltnH | grep ':${toString port}' || true")
assert "127.0.0.1:${toString port}" in listening, (
f"not bound on loopback: {listening!r}"
)
assert "0.0.0.0:${toString port}" not in listening, (
f"bound on every interface, which the default must never do: {listening!r}"
)
with subtest("the firewall does not open the port"):
# `openFirewall` is off by default and must not be implied by
# `http.enable`; this asserts it against the guest's own running
# rule set, the way vm-stream-firewall does.
machine.fail("iptables -C nixos-fw -p tcp --dport ${toString port} -j nixos-fw-accept")
with subtest("an unauthenticated request is refused and says nothing"):
report = refused("--initialize")
assert report["http_status"] == 401, report
assert report["body_len"] == 0, f"a refusal with a body: {report}"
assert (report["www_authenticate"] or "").startswith("Bearer"), report
with subtest("a wrong token is refused the same way"):
report = refused("--initialize", token="hhsmcp_not-a-real-token")
assert report["http_status"] == 401, report
assert report["body_len"] == 0, report
# And the token is not echoed anywhere in what came back.
assert "not-a-real-token" not in json.dumps(report), report
with subtest("the handshake completes with a good token"):
result = http("--initialize", token=token)
assert result["serverInfo"]["name"] == "hhstream", result
assert result["protocolVersion"], result
with subtest("the whole surface is offered to an unscoped token"):
listed = http("--list", token=token)["tools"]
assert len(listed) == 23, f"{len(listed)} tools, expected 23"
fixed = http("--resources", token=token)["resources"]
assert len(fixed) == 6, f"{len(fixed)} resources, expected 6"
with subtest("a resource read reaches the broker"):
contents = http("--read hhstream://sessions", token=token)["contents"]
payload = json.loads(contents[0]["text"])
assert payload.get("ok") is True, payload
with subtest("a tool call reaches the broker"):
result = http("session_gc", token=token)
assert not result.get("isError"), result
with subtest("a scoped token is offered only its toolsets"):
scoped = mint("frames only", "--toolsets frames")
listed = http("--list", token=scoped)["tools"]
names = sorted(tool["name"] for tool in listed)
assert names == ["record", "run", "shot"], names
with subtest("a scoped token cannot call outside its toolsets"):
result = http("session_gc", token=scoped)
assert result.get("isError"), result
text = result["content"][0]["text"]
assert "session_gc" in text, text
with subtest("the unscoped token still sees everything"):
# The two tokens are served by ONE process, which is the whole
# argument for putting toolsets on the token rather than only on
# the command line: a launch-time flag cannot do this.
assert len(http("--list", token=token)["tools"]) == 23
with subtest("token list shows both and neither secret"):
out = machine.succeed(as_user(f"{MCP} token list"), timeout=120)
assert "the check" in out, out
assert "frames only" in out, out
assert token not in out, "`token list` printed a token"
assert scoped not in out, "`token list` printed a token"
with subtest("using a token stamps it"):
records = json.loads(
machine.succeed(as_user(f"{MCP} token list --json"), timeout=120)
)
stamped = [r for r in records if r["last_used"]]
assert len(stamped) == 2, records
with subtest("revoking a token refuses it immediately"):
# No restart of the unit: the table is read per request, so a
# revocation takes effect on the next one. An operator who has to
# restart a service to lock someone out does not have revocation.
records = json.loads(
machine.succeed(as_user(f"{MCP} token list --json"), timeout=120)
)
doomed = next(r["id"] for r in records if r["label"] == "frames only")
machine.succeed(as_user(f"{MCP} token revoke {doomed}"), timeout=120)
report = refused("--initialize", token=scoped)
assert report["http_status"] == 401, report
# And the other token is untouched.
assert http("--initialize", token=token)["serverInfo"]["name"] == "hhstream"
with subtest("the unit survives a restart and the token still works"):
machine.succeed(f"systemctl restart {UNIT}")
machine.wait_for_unit(UNIT, timeout=180)
machine.wait_for_open_port(${toString port}, addr="127.0.0.1", timeout=120)
assert http("--initialize", token=token)["serverInfo"]["name"] == "hhstream"
restarts = machine.succeed(
f"systemctl show -p NRestarts --value {UNIT}"
).strip()
# It restarted while the table was empty, so this is not zero --
# what matters is that it is not still climbing now that it is up.
machine.succeed("sleep 12")
again = machine.succeed(
f"systemctl show -p NRestarts --value {UNIT}"
).strip()
assert restarts == again, (
f"the unit is still restarting ({restarts} -> {again}): it is "
"crash-looping, not running"
)
with subtest("revoking every token does not take the unit down"):
# The running process keeps its port; it simply refuses everyone.
# Stopping on an empty table would mean a revocation could not be
# undone without a `systemctl start`, and the refusal at startup
# exists to catch a *forgotten* mint, not to police a deliberate one.
machine.succeed(as_user(f"{MCP} token revoke --all"), timeout=120)
machine.succeed(f"systemctl is-active {UNIT}")
report = refused("--initialize", token=token)
assert report["http_status"] == 401, report
'';
}
)
+16 -3
View File
@@ -44,10 +44,10 @@
│ in-process: argv in, exit code out
│
┌────────────┴──────────────┐
│ hhstream-mcp │ spawned per client, stdio only
│ hhstream-mcp │ stdio: spawned per client
│ 23 tools (the acts) │◀─── an MCP client's JSON-RPC
│ 10 resources (the state)│
└───────────────────────────┘
│ 10 resources (the state)│ http: one unit, one port,
└───────────────────────────┘ a bearer token per request
```
Six pieces, deliberately separate, and a seventh that is deliberately not a
@@ -104,6 +104,19 @@ And the seventh, which is a *shim* and not a layer:
which is why it is a separate derivation (`nix/mcp.nix`) and why
`packages.hhstream` — the compositor's own units — keeps the closure it had.
It is the only component with **two transports**, and the only one that
authenticates anything. Over stdio it does not, because there is nothing
there to authenticate: the client spawns the process, so it has already
passed a filesystem permission check and holds the `0660` control socket
either way — unix permissions are the gate, and a token checked by a process
the caller started guards a lock whose key is in the same pocket. Over HTTP
(`mcp.http.enable`) it is a unit on a port, the caller passed no such check,
and a bearer token is then the only gate there is. That asymmetry is the
whole of why the credential exists, and it is also why this shim has a state
directory of its own (`mcp.stateDir`) while owning no state about the
desktop: the authorization table is about *callers*, not about sessions, and
it has no business in the broker's `0700` directory beside the audit trail.
## Why containers are outputs
A session's container is a headless **output**, not just a workspace, and the
+227 -20
View File
@@ -1,4 +1,4 @@
# CONTRACT — the MCP server (mcp-v3)
# CONTRACT — the MCP server (mcp-v4)
This file is the interface of `hhstream-mcp`: the resource URIs, the tool names,
the arguments each takes, what comes back, and which of the CLI's exit codes
@@ -10,9 +10,19 @@ containers and the input lease).
these argument shapes is a consumer like any other, and renaming a tool breaks
it exactly as renaming a wire verb would.
Version: mcp-v3 (2026-10-11). v1 was the first freeze and lasted one day; v2
split state out into resources; v3 adds the live stream. v3 is **additive** —
every v2 name still means what it meant.
Version: mcp-v4 (2026-10-12). v1 was the first freeze and lasted one day; v2
split state out into resources and removed five tools; v3 adds the live stream;
v4 adds a second transport. v3 and v4 are **additive** — every v2 name still
means what it meant.
v4 changes: a second transport, and the credential it exists for
| § | Change | Why |
| --- | --- | --- |
| §1 | A **second transport**: Streamable HTTP, opt-in, behind a bearer token. stdio stays the default and stays unauthenticated. | A token means nothing on stdio — the client spawns the server, so it has already passed a filesystem permission check and holds the broker's `0660` socket either way. A port is the first time the caller is someone who did neither, and the transport and the credential are therefore one feature. |
| §2 | `serve --http` and its flags; `token mint\|list\|revoke\|prune` beside it. The bare `hhstream-mcp` and `hhstream-mcp --toolsets=…` forms are unchanged. | A client's `command` entry is a configuration someone else wrote — an HTTP endpoint is a configuration someone else *hosts*. |
| §3 | A token may carry its own `toolsets`, and the **intersection** with the server's `--toolsets` applies. | `--toolsets` is a launch-time flag, and one HTTP process serves many callers: a per-process tool budget stops being a per-caller one the moment the transport is shared. |
| §10 | What the HTTP transport's token **is not**: not OAuth 2.1, not RFC 9728, no metadata discovery. | A limit worth stating in the same register as the two the SDK imposes. |
v3 changes, all of them the per-session live stream
(`docs/CONTRACT-sessions.md` §4, v4):
@@ -75,16 +85,73 @@ documented in `hhstream_mcp.invoke`: `cli.main`'s `sg` re-exec, and `cmd_run`'s
## 1. Transport
**stdio, and only stdio.** One process per client, spawned by the client; no
port, no daemon, no socket of its own. There is already a daemon in this system
and it owns all the state.
**Two, and stdio is the default.**
The protocol is **not on file descriptor 1** once the server is running: the
tools run the CLI in process, the CLI prints, and `record` spawns an `ffmpeg`
whose output the CLI does not capture. The server takes a private duplicate of
stdin and stdout at start-up, then points fd 1 at its log and fd 0 at
`/dev/null`. A stray write is therefore a log line and never a corrupted
stream.
| transport | how | authentication |
| --- | --- | --- |
| **stdio** | `hhstream-mcp`, or `hhstream-mcp serve`. One process per client, spawned by the client; no port and no daemon. | **None, deliberately.** See below. |
| **Streamable HTTP** | `hhstream-mcp serve --http`, or the `mcp.http.enable` unit. One process, many clients, one port. | A bearer token on every request, or `401`. |
### Why stdio is unauthenticated, and why that is not an oversight
The stdio server is spawned *by* its client. To do that the client already had
to pass a filesystem permission check, and whoever can run `hhstream-mcp` can
run `hhstream` — which talks to the same `0660` control socket owned by
`sessions.user`, in the same group. **Unix permissions are the gate**, and
`mcp.enable`'s own option text has always said so: it "grants no access the
`hhstream` CLI does not already grant to whoever runs it". A token checked by a
process the caller started is a lock whose key is in the same pocket.
Which is also the whole argument for the HTTP transport existing. A port is
reachable by a caller who passed no filesystem check, and at that moment a
credential is the only gate there is. So the two arrived together: v1 and v2
said "stdio, and only stdio" because, for what the server then was, that was
the honest answer.
### stdio: the protocol is not on file descriptor 1
Once the server is running: the tools run the CLI in process, the CLI prints,
and `record` spawns an `ffmpeg` whose output the CLI does not capture. The
server takes a private duplicate of stdin and stdout at start-up, then points
fd 1 at its log and fd 0 at `/dev/null`. A stray write is therefore a log line
and never a corrupted stream.
This is also why `token mint` prints a secret on stdout and is safe: the token
verbs return *before* anything claims the transport, which
`tests/unit/test_mcp_authorizations.py` asserts with a `claim_transport` that
fails the test if it is reached.
### HTTP: what is on the wire
* **`Authorization: Bearer <token>` on every request.** The scheme is compared
case-insensitively (RFC 9110). Anything else — absent, malformed, wrong
scheme, unknown token, revoked token, expired token — is one answer: `401`,
an **empty body**, `WWW-Authenticate: Bearer realm="hhstream"`,
`Cache-Control: no-store`. The token is never echoed, in the body or in a
header, and the response does not distinguish *which* of those it was —
telling them apart would tell someone who guessed a real id that they had.
* **Failed attempts are rate-limited per client**, reusing
`hhstream_live.server`'s `RateLimiter`; past the limit the refusal is `429`.
* **The token is checked before the `Host`.** An unauthenticated caller learns
nothing about the rebinding allowlist, which is itself a hint about how the
server is deployed.
* **Any path but the mounted one is a `401` without a token** and a `404` with
one. An unauthenticated caller does not learn what is mounted, or that
anything is.
* **DNS-rebinding protection is on.** The `Host` allowlist is derived from the
bind address when none is given, because the SDK's check is allowlist-only
with no implicit localhost — protection on with an empty list `421`s *every*
request. A wildcard bind derives nothing and is refused at launch, which
makes binding every interface the one case where the hostnames must be named.
* **No TLS.** The token crosses the wire in clear text. Loopback is the
default; anything else wants a reverse proxy and says so in a module warning.
* **Tool calls are serialised across every caller**, not per session. The
`Runner` holds a process-wide lock because the CLI it calls prints to a
stdout this process shares — invisible while each client had a process of its
own, visible now that one process serves many. Two agents on two tokens queue
behind each other, and a slow call delays everyone's next one rather than
only its own caller's. Latency, never a hang: the event loop answers a `ping`
throughout. See `docs/TODO/mcp-in-process-has-no-call-deadline.md`.
## 2. Configuration
@@ -99,17 +166,87 @@ the CLI:
Set them in the MCP client's `env` block. One place to configure them is one
place for them to be wrong.
The server's own command line:
The HTTP transport reads one more, and `hhstream-mcp token` reads the same:
| flag | what it does |
| Variable | What it names | Default |
| --- | --- | --- |
| `HHSTREAM_MCP_STATE_DIR` | where the authorization table lives | `/var/lib/hhstream-mcp` |
**Not `sessions.stateDir`.** That directory is `0700` owned by the broker's
user and holds `events.jsonl`, the audit trail; the MCP server runs as whichever
user's client spawned it, and parking a token file there would mean granting
this shim write access to the audit trail's directory for no reason. The two
also have different lifetimes: revoking every token must not be able to touch a
session record.
The server's own command line. `hhstream-mcp` with no subcommand, and
`hhstream-mcp serve`, are the same thing — the bare form is not deprecated and
will not be, because an MCP client's `command` entry is a configuration
somebody else wrote:
| flag | where | what it does |
| --- | --- | --- |
| `--toolsets NAMES` | either | comma-separated families to offer as **tools**: `frames`, `sessions`, `pointer`, `keyboard`. Omitted, every tool is offered. An unknown name is argparse's exit 2, never a silently empty server. **Resources are not filtered by it** — they are the host's context, not the model's tool budget, and making `hhstream://sessions` conditional on the session *tools* would couple two decisions that are not the same decision. |
| `--list-tools` | either | print the table and exit. Honours `--toolsets`, so it is also how to see what a selection would offer. |
| `--list-resources` | either | print the resource URIs and templates and exit. |
| `--http` | `serve` | Streamable HTTP instead of stdio. |
| `--address HOST` | `serve` | default `127.0.0.1`. Loopback, because reaching the LAN is a second decision. |
| `--port N` | `serve` | default `9989`, one past the live stream's `9988`. |
| `--path P` | `serve` | default `/mcp`. |
| `--state-dir DIR` | `serve`, `token` | overrides `$HHSTREAM_MCP_STATE_DIR`. |
| `--allowed-host HOST` | `serve` | repeatable. Omitted, derived from `--address`; a trailing `:*` is a wildcard port. |
| `--allowed-origin ORIGIN` | `serve` | repeatable. Nothing needs this unless a browser is the client. |
| `--allow-no-tokens` | `serve` | start although the table is empty. Without it that is a usage error, because a listener that `401`s every request is almost always a forgotten `token mint`. |
| `--stateless` | `serve` | no session to pin, for a deployment behind a load balancer. |
| `--json-response` | `serve` | one JSON body instead of an SSE stream. |
### `hhstream-mcp token`
| verb | what it does |
| --- | --- |
| `--toolsets NAMES` | comma-separated families to offer as **tools**: `frames`, `sessions`, `pointer`, `keyboard`. Omitted, every tool is offered. An unknown name is argparse's exit 2, never a silently empty server. **Resources are not filtered by it** — they are the host's context, not the model's tool budget, and making `hhstream://sessions` conditional on the session *tools* would couple two decisions that are not the same decision. |
| `--list-tools` | print the table and exit. Honours `--toolsets`, so it is also how to see what a selection would offer. |
| `--list-resources` | print the resource URIs and templates and exit. |
| `token mint [--label L] [--expires-in D] [--toolsets NAMES]` | print a new token **once**, on stdout, with the prose for a human on stderr beside it. `D` is `30s`/`5m`/`3h`/`90d` or bare seconds. |
| `token list [--json]` | the table. The tokens are not in it and cannot be. |
| `token revoke <id> \| --all` | by **id**, so ending a grant never means pasting a live secret onto a command line that lands in a shell history. |
| `token prune` | drop expired grants. Housekeeping only — an expired token is refused whether or not it is still on disk. |
The table holds **SHA-256 digests, not tokens**, so `authorizations.json` is not
itself a credential; a token that was not written down can only be revoked and
replaced. No salt and no KDF: a token is `secrets.token_urlsafe(32)` behind an
`hhsmcp_` prefix — 256 bits, so there is no dictionary to attack and a work
factor would only slow the request path. The prefix is there so a leaked token
is identifiable on sight, which is the convention secret scanners are built
around.
Expiry is **wall clock**, and it is the one place in this project where that is
right. Every deadline in the broker is `CLOCK_BOOTTIME` (`hhstreamd.clock`)
because an idle window has to survive an NTP step and count time spent
suspended. "Valid for thirty days" is a calendar statement that has to outlive
a reboot, and BOOTTIME resets to zero there.
Every read-modify-write takes an exclusive `flock` on a sidecar lock file. Two
processes write this table — the server stamping `last_used`, and an operator
running `token mint` — and the atomic replace alone prevents a torn *read*, not
a lost *update*.
These verbs sit **outside** the frozen exit-code table in
`hhstreamd.protocol`, because nothing they do reaches the broker and no agent
calls them. Two of the codes are chosen by analogy with it anyway: `0` done,
`2` usage, `3` no token with that id (the analogue of `NO_SUCH_SESSION`), `4`
the store itself is unusable (the analogue of `BAD_STATE`).
`$HHSTREAM_MCP_LOG` sets the log level on stderr. Nothing else affects
behaviour; in particular the server holds no state and caches no lease.
On the HTTP transport a **token** may carry its own `toolsets`, and the
**intersection** with the server's own applies — a token cannot widen what the
unit was started with. That exists because `--toolsets` is a launch-time flag
and one HTTP process serves many callers: a per-process tool budget stops being
a per-caller one the moment the transport is shared, and the token is then the
only place the distinction can live. A tool the caller's token excludes is not
listed and cannot be reached by name; the refusal names both possible
narrowings, because the agent cannot tell them apart from where it stands and
neither is its to change.
**There is no grouping primitive in MCP**, and `--toolsets` is not pretending to
be one. The `Tool` object carries `name`, `title`, `description`,
`inputSchema`, `outputSchema`, `icons`, `annotations`, `_meta` and `execution` —
@@ -429,5 +566,75 @@ unaffected, which is what that code means.
`services.hyprland-headless-stream.mcp.enable` (default `false`) puts the
binary on the system PATH. That is the whole of the option: there is no unit, no
socket and no state directory, because the server is spawned per client over
stdio.
socket and no state directory, because the stdio server is spawned per client.
`services.hyprland-headless-stream.mcp.http.enable` (default `false`) is the
other half, and it **is** a unit — `hhstream-mcp.service`, running as
`sessions.user` with `sessions.group`, because the tools it serves are the
CLI's own handlers called in process and that CLI opens a `0660` control socket
owned by that user and a compositor runtime directory owned by that group. It
is `Wants`/`After` the broker and not `BindsTo`: the frame verbs answer with
the broker down and the rest answer exit 5, which is a true answer, whereas a
listener that vanished with the broker would turn that into a connection
refused and tell a client nothing.
The two options are independent and neither implies the other. `http.enable`
also puts the binary on PATH, because `token mint` is how the credential that
unit demands gets issued.
| option | default |
| --- | --- |
| `mcp.stateDir` | `/var/lib/hhstream-mcp`, created `0700` by a tmpfiles entry |
| `mcp.http.address` | `127.0.0.1` |
| `mcp.http.port` | `9989` |
| `mcp.http.path` | `/mcp` |
| `mcp.http.allowedHosts` / `allowedOrigins` | `[]` — hosts are derived from the address |
| `mcp.http.toolsets` | `null`, every tool |
| `mcp.http.openFirewall` | `false`, and not implied by `http.enable` |
| `mcp.http.extraArgs` | `[]` |
**The unit refuses to start while the authorization table is empty**, and
retries every five seconds rather than giving up
(`StartLimitIntervalSec = 0`). Both halves are deliberate and neither works
without the other: the refusal is there because a listener that `401`s every
request is almost always a forgotten mint, and "connection works, auth fails"
is far worse to debug than a `systemctl status` that names the remedy — but a
refusal that gave up would leave every freshly provisioned machine dead, and
minting the first token would then also need a `systemctl start` nothing told
anyone about. With the retry, the unit comes up on its own within one interval
of the first mint. `checks/vm-mcp-http.nix` asserts exactly that, because
nothing in the Python can.
Revoking *every* token does not stop the running process — it keeps its port
and refuses everyone. The startup refusal exists to catch a forgotten mint, not
to police a deliberate one.
## 10. What the HTTP transport's token is not
**Not MCP's authorization specification.** That is OAuth 2.1 plus RFC 9728
protected-resource metadata: a `401` is meant to carry a `WWW-Authenticate`
pointing at a metadata document, the client discovers an authorization server,
and a real flow happens. What this serves is a static token an operator minted
and pasted into a client's configuration.
Stated rather than discovered, because the consequences are not subtle:
* a client that can only *discover* an authorization server will not connect to
this at all;
* a client that lets you set a header works immediately;
* there is no refresh, no rotation and no revocation list beyond
`token revoke`, which is immediate because the table is read per request —
no restart, and an operator who had to restart a service to lock someone out
would not have revocation.
The gap is `docs/TODO/mcp-http-auth-is-not-oauth.md`, beside the two limits the
SDK imposes.
**A token does not scope resources.** §2 keeps resources out of `--toolsets` on
the grounds that they are the host's context rather than the model's budget, and
that argument carries over to the server's own flag. It does not carry over
cleanly to a *token*, which is an authorization and not a budget — so a token
scoped to `frames` can still read `hhstream://sessions`. That is a real gap, not
an oversight: `docs/TODO/mcp-tokens-do-not-scope-resources.md`, and
`tests/unit/test_mcp_http.py` asserts the current behaviour so that fixing it is
a visible change rather than a silent one.
+2
View File
@@ -17,6 +17,8 @@ Each file answers three questions and stops:
| [capture-binary-and-the-unit-path](capture-binary-and-the-unit-path.md) | medium | partly fixed: the module passes grim's store path and a check reads real frames; the app still never checks its capture binary |
| [mcp-in-process-has-no-call-deadline](mcp-in-process-has-no-call-deadline.md) | medium | two costs of running the CLI in process: no killable deadline, and no `sg` recovery |
| [mcp-resources-cannot-be-subscribed-to](mcp-resources-cannot-be-subscribed-to.md) | medium | the SDK hardcodes `subscribe=False`, so a log resource can only be re-read |
| [mcp-http-auth-is-not-oauth](mcp-http-auth-is-not-oauth.md) | medium | a static bearer token, not OAuth 2.1 + RFC 9728: the cost is which clients can connect, not how exposed the desktop is |
| [mcp-tokens-do-not-scope-resources](mcp-tokens-do-not-scope-resources.md) | medium | `token mint --toolsets frames` narrows the tools and not the ten resources; nothing until a second token is minted for someone else |
| [stream-does-not-reattach-after-a-broker-restart](stream-does-not-reattach-after-a-broker-restart.md) | low | the phrase survives and the sweep works; `attached` reports false until re-enabled |
| [a-stream-that-gave-up-still-reports-attached](a-stream-that-gave-up-still-reports-attached.md) | low | the other direction: a capture that permanently gave up still reads `LIVE: YES` |
+88
View File
@@ -0,0 +1,88 @@
# The HTTP transport's bearer token is not MCP authorization
## The issue
`hhstream-mcp serve --http` requires `Authorization: Bearer <token>` on every
request, where the token is a static secret an operator minted with
`hhstream-mcp token mint` and pasted into a client's configuration. That is not
what the Model Context Protocol says authorization is.
The specification's authorization profile is **OAuth 2.1 plus RFC 9728
protected-resource metadata**. A compliant server answers an unauthenticated
request with a `401` whose `WWW-Authenticate` carries a `resource_metadata`
parameter pointing at `/.well-known/oauth-protected-resource`; the client
fetches that, learns which authorization server to talk to, discovers *its*
metadata, and runs a real flow with scopes, refresh and revocation. The MCP
Python SDK ships the machinery for this — `mcp.server.auth`, its
`TokenVerifier` protocol, `AuthSettings` with an `issuer_url` and a
`resource_server_url`, and `RequireAuthMiddleware`.
What ships here is `hhstream_mcp.http.BearerAuth`, forty lines of ASGI
middleware that looks the token up in a file. The `401` carries
`WWW-Authenticate: Bearer realm="hhstream"` and nothing else, because there is
no authorization server to point at and advertising a metadata endpoint that
does not exist would be worse than advertising nothing.
The measured consequences, in order of how soon they are met:
1. **A client that only knows how to discover an authorization server cannot
connect.** It will read the `401`, find no `resource_metadata`, and have
nowhere to go. A client that lets a human set a header connects
immediately — which is most of them today, and is why this is usable at all.
2. **There is no rotation and no expiry negotiation.** `token mint
--expires-in` sets a fixed lifetime at mint time and nothing can extend it;
a client that wants to keep working past it needs a human to mint another
one. There is no refresh token because there is no token endpoint.
3. **There are no scopes in the protocol's sense.** A token carries
`toolsets`, which is this project's own notion and invisible to a client:
nothing in the handshake tells a caller what its token may do, so an agent
discovers its own limits by being refused. (`AccessToken.scopes` is where
these would go if the SDK's path were taken, and the SDK's middleware is
what would then enforce them.)
4. **Audience is unchecked.** RFC 8707 resource indicators exist so a token
minted for one server cannot be replayed at another. These tokens name no
audience, so a token is as good as the operator's care in handing it out.
## Ideas
1. **Keep it, and say so.** *(what ships, and the recommended route for now)*
The threat this transport actually faces is "someone on the LAN finds the
port", and a 256-bit token in a header answers it. OAuth would add an
authorization server to deploy, a metadata document to serve and a flow to
get right, in exchange for no additional protection against that threat on
a single-operator machine. The honest version of this option is the one
taken: a limit written down, not a capability implied.
2. **Implement the SDK's `TokenVerifier` over the same table.** The protocol is
one method (`async verify_token(token) -> AccessToken | None`) and the store
already satisfies it in all but name — `Store.verify` returns a `Grant` and
an `AccessToken` is a `Grant` with `client_id` and `scopes` spelled
differently. That alone buys spec-shaped *verification* and lets
`toolsets` become real `scopes`, enforced by the SDK's middleware rather
than by `offered_for`. It does **not** buy discovery: without an issuer
there is still no metadata to advertise. *(recommended as the next step, if
one is wanted — it is small, and it is the half that pays.)*
3. **Serve the metadata documents and be a resource server for real.** Mint
tokens as signed JWTs with an audience and an expiry, serve
`/.well-known/oauth-protected-resource`, and accept an external
authorization server's tokens as well as locally minted ones. This is the
complete answer and is a feature in its own right, worth doing when a client
that needs it actually appears.
4. **Drop the HTTP transport and tunnel stdio over SSH instead.** Rejected, but
worth recording as the thing this competes with: `ssh host hhstream-mcp` is
genuinely more secure — real keys, real TLS-grade transport, and no new
listener — and for one operator with one machine it is the better answer. The
HTTP transport exists for the client that cannot spawn a process over SSH,
which is most hosted ones.
## Why it matters
Medium, and the severity is about *interoperability* rather than about safety.
Nothing here is weaker than it claims to be: the token is 256 bits, the refusal
leaks nothing, revocation is immediate, and loopback is the default. What is
missing is the part that lets an arbitrary MCP client connect without a human
pasting a header — so the cost is paid in which clients work, not in how
exposed the desktop is.
It would move to high the moment this server is meant to be reached by a client
somebody else configured, because at that point "set this header" stops being
an instruction anyone can follow.
@@ -65,3 +65,13 @@ and not high.
(1) is latent. It becomes real the first time a compositor stops answering while
an agent is driving it: the agent's next tool call waits behind the stuck one
with no way to give up, where a CLI user would have pressed ctrl-C.
**The HTTP transport widens (1) without changing it.** The serialisation was
always there — `Runner` holds a process-wide lock because the CLI prints to a
shared stdout — but over stdio each client had a process of its own, so a call
only ever queued behind that same client's previous call. `mcp.http.enable`
puts every caller in one process, so a stuck call now delays *other people's*
agents too, and the one of them who can see something is wrong is not
necessarily the one who caused it. That makes idea 1 (a reachable
frame-socket timeout) worth more than it was, and idea 2 (a forked child per
call) less unreasonable than it was.
@@ -0,0 +1,75 @@
# A token narrows the tools it may call, and none of the state it may read
## The issue
`hhstream-mcp token mint --toolsets frames` makes a token that is offered three
tools — `shot`, `record`, `run` — and is refused `session_start`,
`session_kill`, every pointer verb and every keyboard verb. It is also offered
**all five resources and all three templates**, unchanged. So a token scoped to
"look at the screen" can read `hhstream://sessions` and learn every running
session's id, application, pid, log path and home directory; it can read
`hhstream://sessions/<id>/logs` and get an application's own output; and it can
read `hhstream://pointer` and see who holds the lease.
`tests/unit/test_mcp_http.py::test_resources_are_not_narrowed_by_a_token`
asserts this, deliberately, so the behaviour is pinned rather than incidental.
The reasoning that produced it is sound for the server's own flag and does not
survive being reused for a token. `docs/CONTRACT-mcp.md` §2 keeps resources out
of `--toolsets` because **a tool listing is a budget and a resource is not**:
tools compete for a model's attention, resources are the host's to put in
context, and making `hhstream://sessions` conditional on the session *tools*
being enabled would couple two decisions that are not the same decision. All
true.
But `--toolsets` is set by whoever starts the process, and a token is handed to
somebody else. A budget and an authorization are different things, and
`hhstream_mcp.http.offered_for` applies the budget's rule to the
authorization — which is the bug in one sentence.
How much this matters depends entirely on a fact the code cannot see: whether
the operator hands out more than one token. With one token it is nothing. With
two of different trust, it is a confused deputy — the narrow token's holder was
told it could only look at the screen, and it can enumerate the desktop.
## Ideas
1. **A separate `--resources` list on the token.** *(recommended)* Mint takes
`--resources` naming families (`status`, `sessions`, `pointer`,
`containers`, `screen`), defaulting to all of them, and
`hhstream_mcp.resources.read` refuses a URI outside the grant with the same
`ReadRefused` a bad URI already raises. Two fields because they are two
decisions — a token that may drive the pointer does not automatically need
to read session logs — and keeping them separate means the *server's*
`--toolsets` can go on meaning only what it means now.
2. **Derive the resource grant from the toolsets.** `frames` implies
`hhstream://status`, `screen` and `outputs/{name}`; `sessions` implies the
session resources; `pointer` implies `hhstream://pointer`. One field, no new
vocabulary, and it maps the way a reader would guess. Rejected as the
recommendation because the mapping is a guess: `hhstream://containers` is a
session resource that a pointer-only token genuinely needs, since the
pointer aims at a session's output, and a derived rule would have to special
case it.
3. **A single `--read-only-everything` / `--no-read` switch.** The cheap version:
a token either reads all state or none. Honest and tiny, and strictly better
than today. But "none" breaks the pointer tools for the reason above, so it
would be a switch nobody could use with a narrow token.
4. **Leave it and document it.** What ships. Defensible while this machine has
one operator and the tokens all belong to them, which is why this is medium
rather than high — but it is the option that gets less defensible the moment
a second token is minted for someone else.
## Why it matters
Medium. The surface it exposes is read-only and this is a single-operator
machine today, so nothing is currently reachable through it that the token
holder could not also learn by taking a screenshot and reading the window
titles — which `frames` lets it do.
It becomes real with the second token. `--toolsets` on a token is advertised
as the way to hand out a *restricted* credential, and the restriction is
narrower than it reads: an operator who mints a `frames` token for a
screen-watching agent has not stopped that agent from enumerating sessions, and
nothing in the surface or the mint output says so. Until this is fixed, that
sentence belongs in the answer to "can I give out a limited token" — which is
what this file is for.
+5
View File
@@ -41,6 +41,11 @@ python3.pkgs.buildPythonApplication {
dependencies = [
python3.pkgs.mcp
python3.pkgs.anyio
# The HTTP transport's own two, declared for the reason `anyio` is: the SDK
# propagates both today, and `hhstream_mcp.http` imports them by name. See
# the note in src/hhstream_mcp/pyproject.toml.
python3.pkgs.starlette
python3.pkgs.uvicorn
# An application is not importable by a dependent until this converts it,
# and `hhstream_mcp.invoke` imports `hhstream.cli` directly.
(python3.pkgs.toPythonModule hhstream)
+371 -11
View File
@@ -850,11 +850,22 @@ in {
#
# Its own top-level block rather than one under `sessions`, because it is
# not the broker's: four of its tools (`status`, `shot`, `record`, `run`)
# are frame-socket verbs that work with the broker switched off. It is also
# not a unit. The transport is stdio, one process per client, spawned by
# whichever client wants it -- so "installing" it means exactly putting the
# binary on PATH, and anything that started a daemon here would be
# inventing a second thing to supervise.
# are frame-socket verbs that work with the broker switched off.
#
# `mcp.enable` is still not a unit, and that is still the point: the stdio
# transport is one process per client, spawned by whichever client wants
# it, so "installing" it means exactly putting the binary on PATH.
#
# `mcp.http.enable` IS a unit, and it is the second thing to supervise that
# the comment here used to say was never worth inventing. What changed is
# not the cost of supervising it but what it buys: a port can be reached by
# a caller who never touched this filesystem, which is the first time an
# authorization token on this surface means anything at all. Over stdio the
# client spawns the server and already holds the broker's socket, so a
# token there guards a lock whose key is in the same pocket. The two
# options are therefore independent -- one is a binary on PATH for a local
# client, the other is a listener with a credential -- and neither implies
# the other.
mcp = {
enable = lib.mkOption {
type = lib.types.bool;
@@ -885,6 +896,157 @@ in {
are the same build.
'';
};
stateDir = lib.mkOption {
type = lib.types.str;
default = "/var/lib/hhstream-mcp";
description = ''
Where `hhstream-mcp` keeps its authorization table — the SHA-256
digests of the bearer tokens {option}`mcp.http.enable` accepts, and
nothing else.
Deliberately **not** {option}`sessions.stateDir`. That directory is
`0700` owned by {option}`sessions.user` and holds `events.jsonl`, the
broker's audit trail; parking a token file in it would mean granting
this shim write access to the audit trail's directory for no reason.
The two also have different lifetimes: revoking every token should
not be able to touch a session record, and vice versa.
The file holds digests rather than tokens, so it is not itself a
credential — but it is the list of who may drive this desktop, and it
is created `0700`.
'';
};
http = {
enable = lib.mkOption {
type = lib.types.bool;
default = false;
description = ''
Serve the MCP surface over Streamable HTTP on
{option}`mcp.http.port`, as a system unit, authenticated by a
bearer token.
This is the one shape of this server where a credential means
something: the stdio transport is spawned by its client, so that
caller has already passed a filesystem permission check and holds
the broker's `0660` control socket either way. A port is reachable
by someone who did neither.
Mint a token with `hhstream-mcp token mint`, run as
{option}`sessions.user`, and give it to the client as an
`Authorization: Bearer …` header.
**The unit refuses to start while the table is empty**, because a
listener that 401s every request is almost always a forgotten mint,
and "connection works, auth fails" is far worse to debug than a
`systemctl status` that names the remedy. It retries every five
seconds rather than giving up, so it comes up on its own within one
interval of the first `token mint` — no `systemctl start` needed.
Pass `--allow-no-tokens` through {option}`mcp.http.extraArgs` to
start anyway.
Not MCP's authorization specification, which is OAuth 2.1 plus
RFC 9728 metadata discovery. A client that can only *discover* an
authorization server will not connect; one that lets you set a
header will. See `docs/TODO/mcp-http-auth-is-not-oauth.md`.
'';
};
address = lib.mkOption {
type = lib.types.str;
default = "127.0.0.1";
description = ''
The address to bind.
Loopback by default. Binding somewhere the LAN can reach means a
bearer token is the only thing between the network and a desktop
something can type into, over plain HTTP with no TLS — put a
reverse proxy in front if that is the intent.
`0.0.0.0` is accepted but then {option}`mcp.http.allowedHosts` must
be set: DNS-rebinding protection is on, the allowlist has no
implicit localhost, and the `Host` header a client will send cannot
be derived from a wildcard bind.
'';
};
port = lib.mkOption {
type = lib.types.port;
default = 9989;
description = ''
The port to bind. One past {option}`sessions.stream.port`'s default,
so a machine running both has no collision.
'';
};
path = lib.mkOption {
type = lib.types.str;
default = "/mcp";
description = "The path the endpoint is mounted at.";
};
allowedHosts = lib.mkOption {
type = lib.types.listOf lib.types.str;
default = [];
example = ["desk.lan:9989" "desk.lan:*"];
description = ''
`Host` header values to accept, for DNS-rebinding protection.
Empty, the server derives them from {option}`mcp.http.address` —
which covers the loopback default and fails for a wildcard bind,
where nothing can be derived. A trailing `:*` is a wildcard port.
'';
};
allowedOrigins = lib.mkOption {
type = lib.types.listOf lib.types.str;
default = [];
description = ''
`Origin` header values to accept. Nothing needs this unless a
browser is the client: a request with no `Origin` is allowed, and
any present one not named here is refused. A browser page is the
attacker in the DNS-rebinding story, not a user of this server.
'';
};
toolsets = lib.mkOption {
type = lib.types.nullOr (lib.types.listOf lib.types.str);
default = null;
example = ["frames" "sessions"];
description = ''
Families to offer as tools, process-wide: `frames`, `sessions`,
`pointer`, `keyboard`. `null` offers every tool.
A ceiling, not a grant: a token may carry its own narrower list
(`token mint --toolsets`), and a tool has to survive both. Resources
are not filtered by either.
'';
};
openFirewall = lib.mkOption {
type = lib.types.bool;
default = false;
description = ''
Open {option}`mcp.http.port` in the firewall.
Off by default and not implied by {option}`mcp.http.enable`, the
same way {option}`sessions.stream.openFirewall` is not implied by
the stream: this port is a language model's path to starting
applications and typing on this desktop, and a bearer token over
plain HTTP is its only protection. Opening it is an explicit act.
'';
};
extraArgs = lib.mkOption {
type = lib.types.listOf lib.types.str;
default = [];
example = ["--stateless" "--allow-no-tokens"];
description = ''
Extra arguments for `hhstream-mcp serve --http`, appended last.
'';
};
};
};
};
@@ -898,6 +1060,52 @@ in {
assertion = lib.hasPrefix "/" cfg.socketPath;
message = "services.hyprland-headless-stream.socketPath must be an absolute path.";
}
# The HTTP server is a unit, so it needs an account -- and the one it
# needs is the broker's, because its tools are the CLI's handlers and
# that CLI opens the broker's control socket and the compositor's runtime
# directory. `sessions.enable` is what declares that identity.
#
# An assertion and not a warning, unlike `mcp.enable` against the same
# condition: there, every session tool answers exit 5 and the frame tools
# still work, so it is a thing worth saying. Here the unit would fail to
# start at all, with `User=` naming an account that does not exist, and a
# listener that never comes up is not a degraded mode.
{
assertion = !cfg.mcp.http.enable || cfg.sessions.enable;
message = ''
services.hyprland-headless-stream.mcp.http.enable needs sessions.enable.
The HTTP transport runs as sessions.user with sessions.group, because
the tools it serves are the `hhstream` CLI's own handlers and that CLI
opens the broker's 0660 control socket and the compositor's runtime
directory. With sessions.enable off, that account is not declared and
the unit fails to start.
For a frame-only surface, use mcp.enable (stdio, no unit) instead.
'';
}
# `--address 0.0.0.0` and no allowedHosts is a server that 421s every
# request: DNS-rebinding protection is on, the allowlist has no implicit
# localhost, and the Host header a client will send cannot be derived
# from a wildcard bind. Caught here rather than at start-up, because a
# rebuild that produces a unit which refuses everything is worse than a
# rebuild that does not evaluate.
{
assertion =
!cfg.mcp.http.enable
|| !(builtins.elem cfg.mcp.http.address ["0.0.0.0" "::" "*"])
|| cfg.mcp.http.allowedHosts != [];
message = ''
services.hyprland-headless-stream.mcp.http.address is
"${cfg.mcp.http.address}", which binds every interface, so
mcp.http.allowedHosts must name the Host values to accept.
DNS-rebinding protection is on and its allowlist has no implicit
localhost, so an empty list refuses every request with a 421. Name the
hostnames clients will use (a trailing ":*" is a wildcard port), or
bind one address.
'';
}
# Structural, not cosmetic: see the `runtimeDirName` comment above. The
# ceiling exists so the compositor's *event* socket still fits in
# `sun_path`, and the symptom of breaking it is silent -- the request
@@ -984,17 +1192,44 @@ in {
services.hyprland-headless-stream.mcp.package =
lib.mkDefault (pkgs.callPackage ./mcp.nix { hhstream = cfg.package; });
# Installing it IS the whole of `mcp.enable`: the server is spawned by a
# client over stdio, so there is no unit, no socket and no state directory
# to declare.
environment.systemPackages = lib.mkIf cfg.mcp.enable [ cfg.mcp.package ];
# Installing it IS the whole of `mcp.enable`: the stdio server is spawned
# by a client, so that option declares no unit and no socket.
#
# `mcp.http.enable` also puts it on PATH, and not only because the unit
# needs it: `hhstream-mcp token mint` is how an operator issues the
# credential that unit demands, and a listener nobody can mint a token for
# is a listener that 401s for ever.
environment.systemPackages =
lib.mkIf (cfg.mcp.enable || cfg.mcp.http.enable) [ cfg.mcp.package ];
# A warning and not an assertion: twenty of the twenty-three tools and
# seven of the ten resources need the broker, and an agent handed them against
# a machine where it is off gets exit 5 from every one of them -- which is
# a true answer, badly timed. The frame tools are genuinely useful on their
# own, so this is a thing worth saying rather than a thing worth refusing.
warnings = lib.optional (cfg.mcp.enable && !cfg.sessions.enable) ''
warnings =
# Plain HTTP, no TLS, and one bearer token. Said out loud because the
# option that gets you here reads like a bind address and not like a
# decision about what is on the wire.
lib.optional
(cfg.mcp.http.enable && !(builtins.elem cfg.mcp.http.address ["127.0.0.1" "::1" "localhost"]))
''
services.hyprland-headless-stream.mcp.http.address is
"${cfg.mcp.http.address}", which is reachable beyond this machine.
There is no TLS here: the bearer token travels in a header in clear text,
and it authorizes starting applications, clicking and typing on this
desktop. Put a reverse proxy that terminates TLS in front of it, or keep
the bind on loopback and reach it through an SSH tunnel.
''
++ lib.optional (cfg.mcp.http.enable && cfg.mcp.http.openFirewall) ''
services.hyprland-headless-stream.mcp.http.openFirewall is on: port
${toString cfg.mcp.http.port} is open to the network, and a bearer token
is the only thing between it and a language model driving this desktop.
Mint tokens with an expiry (`hhstream-mcp token mint --expires-in 30d`)
and revoke them when a client is retired.
''
++ lib.optional (cfg.mcp.enable && !cfg.sessions.enable) ''
services.hyprland-headless-stream.mcp.enable is on while sessions.enable
is off: the MCP server's session, pointer and keyboard tools, and every
resource but hhstream://status and hhstream://screen, will fail because
@@ -1036,6 +1271,22 @@ in {
mode = "0750";
};
})
# The MCP server's own state directory: the authorization table, and
# nothing else. A tmpfiles entry rather than `StateDirectory=` on the
# unit, for the reason the broker's is one -- `StateDirectory=` can only
# name a directory under /var/lib and would stop applying the moment
# `mcp.stateDir` is pointed anywhere else, whereas this is declarative
# for any path.
#
# `mcp.enable` alone does not create it. The stdio transport reads no
# tokens, so a directory for them would be a directory nothing opens.
(lib.mkIf cfg.mcp.http.enable {
"10-hhstream-mcp".${cfg.mcp.stateDir}.d = {
user = cfg.sessions.user;
group = cfg.sessions.group;
mode = "0700";
};
})
# Only needed when the socket was moved out of the runtime directory that
# `RuntimeDirectory=` already creates for us.
(lib.mkIf (socketDir != runtimeDir) {
@@ -1056,7 +1307,8 @@ in {
# plan; this is the only part of the stream that a machine declares before
# any of that exists.
networking.firewall.allowedTCPPorts =
lib.optional cfg.sessions.stream.openFirewall cfg.sessions.stream.port;
lib.optional cfg.sessions.stream.openFirewall cfg.sessions.stream.port
++ lib.optional (cfg.mcp.http.enable && cfg.mcp.http.openFirewall) cfg.mcp.http.port;
users.users.${cfg.user} = {
isSystemUser = true;
@@ -1399,5 +1651,113 @@ in {
);
};
};
# ── the MCP server, over HTTP ──────────────────────────────────────
#
# The fourth unit, and the first one that listens on a port rather than a
# Unix socket. That is the whole reason it exists: `mcp.enable` puts a
# stdio binary on PATH and declares nothing, because a client spawns it and
# has already passed a filesystem permission check to do so. A port is
# reachable by a caller who has not, which is the first time the bearer
# token this unit demands means anything.
#
# It runs as the broker's user, with the broker's group, for the same
# reason the pointer driver does: its tools are the `hhstream` CLI's own
# handlers called in process, and that CLI talks to a `0660` control socket
# owned by that user and opens a compositor runtime directory owned by that
# group. Running it as anything else would mean a listener that
# authenticates a caller and then cannot do a thing for them.
systemd.services.hhstream-mcp = lib.mkIf cfg.mcp.http.enable {
description = "MCP server for the headless Hyprland desktop (HTTP)";
wantedBy = ["multi-user.target"];
after = ["hyprland-headless.service" "hhstream-sessions.service"];
# `After` and `Wants`, not `BindsTo`: three of the twenty-one tools and
# two of the five resources are frame-socket verbs that answer with the
# broker down, and the rest answer exit 5, which is a true answer. A
# listener that vanished with the broker would turn that into a
# connection refused, which tells a client nothing.
wants = ["hhstream-sessions.service"];
environment = {
PYTHONUNBUFFERED = "1";
# The sockets are the CLI's own environment and there is no flag for
# them (`docs/CONTRACT-mcp.md` §2); this is the unit's half of that
# contract. The state directory is passed the same way rather than as
# `--state-dir`, so `hhstream-mcp token mint` run by an operator with
# this variable set and the unit agree by construction.
HHSTREAM_SOCKET = controlSocket;
HHSTREAM_FRAME_SOCKET = cfg.socketPath;
HHSTREAM_MCP_STATE_DIR = cfg.mcp.stateDir;
};
serviceConfig = {
Type = "simple";
User = cfg.sessions.user;
Group = cfg.sessions.group;
SupplementaryGroups = [cfg.sessions.group];
WorkingDirectory = cfg.mcp.stateDir;
# The CLI's own umask and not a tighter one, deliberately. This unit's
# tools write files an operator then reads: `shot --output` and
# `record` produce exactly the files the same verbs produce from a
# shell, and a unit that made them 0600 would be a surprising
# difference between the CLI and the tool that wraps it -- the sort
# that ends with a web server unable to read a capture and nothing
# saying why.
#
# The authorization table is not protected by this. It is `0600` by an
# explicit chmod, inside a `0700` directory declared by the tmpfiles
# entry above, which is what actually guards it.
UMask = "0022";
Restart = "on-failure";
RestartSec = 5;
# No start-rate limit, which is what makes the empty-table refusal
# usable rather than merely correct.
#
# The server exits 2 while no token exists, because a listener that
# 401s every request is almost always a forgotten `token mint` and
# "connection works, auth fails" is a far worse thing to debug than a
# unit whose `systemctl status` names the remedy. But that puts a
# freshly provisioned machine in a failed state until an operator acts,
# and systemd's default (5 starts in 10s, then give up) would leave it
# there for good -- so minting the first token would also need a
# `systemctl start` nothing told anyone about.
#
# With the limit off, the unit retries every 5 seconds and comes up on
# its own within one interval of the first mint. The cost is a journal
# line every 5 seconds while the table is empty, which names exactly
# what is wrong.
StartLimitIntervalSec = 0;
NoNewPrivileges = true;
# This unit launches nothing and adopts nothing: it is a listener in
# front of a CLI, and every process it starts (`grim`, `ffmpeg`) is
# part of answering one request. So the systemd default is right here
# and `KillMode=process` -- which the broker and the driver both need
# -- would be wrong: stopping this unit should take its children with
# it. Sessions are the broker's and live in the broker's cgroup.
ExecStart = lib.escapeShellArgs (
[
(lib.getExe' cfg.mcp.package "hhstream-mcp")
"serve"
"--http"
"--address"
cfg.mcp.http.address
"--port"
(toString cfg.mcp.http.port)
"--path"
cfg.mcp.http.path
]
++ lib.optionals (cfg.mcp.http.toolsets != null) [
"--toolsets"
(lib.concatStringsSep "," cfg.mcp.http.toolsets)
]
++ lib.concatMap (host: ["--allowed-host" host]) cfg.mcp.http.allowedHosts
++ lib.concatMap (origin: ["--allowed-origin" origin]) cfg.mcp.http.allowedOrigins
++ cfg.mcp.http.extraArgs
);
};
};
};
}
+13
View File
@@ -41,3 +41,16 @@ __version__ = "0.1.0"
#: an MCP client configuration block is expected to use for it, so a log line
#: and a config entry can be matched up by eye.
SERVER_NAME = "hhstream"
#: The HTTP transport's defaults. Here rather than in
#: :mod:`hhstream_mcp.http` because ``server.py`` needs them to build its
#: argument parser, and importing that module to read three numbers would make
#: the stdio transport pay for ``starlette`` and ``uvicorn`` at startup.
#:
#: Loopback, deliberately: binding somewhere a LAN can reach is a second
#: decision, and it wants a firewall rule taken with it.
DEFAULT_HTTP_ADDRESS = "127.0.0.1"
#: One past the live stream's 9988, so a machine running both has no collision
#: and an operator reading `ss -ltnp` sees the pair together.
DEFAULT_HTTP_PORT = 9989
DEFAULT_HTTP_PATH = "/mcp"
+457
View File
@@ -0,0 +1,457 @@
"""The token store: who may reach this server over HTTP.
Standard library only, and no MCP import, so the whole of it is testable
without the SDK -- the same split :mod:`hhstream_mcp.tools` keeps.
**This exists because of the HTTP transport and nothing else.** Over stdio the
client spawns the server, and whoever can spawn it already holds the broker's
control socket (``0660``, ``sessions.group``): a token checked by a process the
caller started is a lock whose key is in the same pocket. Bind a port and the
caller is no longer someone who passed a filesystem permission check on the way
in, and that is the moment a credential starts meaning something.
What is on disk
---------------
One JSON object in ``<state>/authorizations.json``::
{"version": 1, "grants": [{"id": …, "label": …, "hash": …, …}]}
**The file holds SHA-256 hashes, not tokens.** Nothing here ever needs the
secret back -- ``mint`` prints it once and forgets it -- so storing the digest
costs nothing and means a readable ``authorizations.json`` is not itself a
credential. No salt and no KDF, deliberately: a token is
:func:`secrets.token_urlsafe` over 32 bytes, so there is no dictionary to
attack and a work factor would only slow the legitimate request path. This
diverges from the live stream, which keeps a phrase in cleartext in the session
record (`docs/plans/2026-10-03-network-stream.md`) -- it has to, because
``serve --stream`` needs it back.
Expiry is **wall clock**, which is the one place in this project that is the
right choice. Every deadline in the broker is ``CLOCK_BOOTTIME`` (see
:mod:`hhstreamd.clock`) because an idle window must survive an NTP step and
count time spent suspended. A token minted for thirty days is a calendar
statement that has to outlive a reboot, and BOOTTIME resets to zero there.
Concurrency
-----------
Two processes write this file: the HTTP server, stamping ``last_used``, and an
operator running ``hhstream-mcp token mint`` in a shell. The atomic replace
borrowed from :func:`hhstreamd.registry.atomic_write_json` keeps a reader from
ever seeing half a file, but it does not stop a lost update -- the server could
rewrite the table it read a moment before the mint landed, and the new token
would vanish. So every read-modify-write takes an exclusive ``flock`` on a
sidecar lock file first. ``verify`` on the hot path takes no lock at all: it
only reads, and a stale read costs one request.
"""
from __future__ import annotations
import base64
import contextlib
import fcntl
import hashlib
import hmac
import json
import logging
import os
import secrets
from collections.abc import Iterator, Sequence
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
from hhstreamd.clock import Clock, iso
from hhstreamd.registry import atomic_write_json
log = logging.getLogger("hhstream.mcp.auth")
#: Where the tokens live when nothing says otherwise. **Not** the broker's
#: ``sessions.stateDir``: that directory is ``0700`` owned by the broker's user
#: and holds ``events.jsonl``, the audit trail, and the MCP server runs as
#: whichever user's client spawned it. Handing this shim write access to the
#: audit trail's directory to park a token file in it would be a wider grant
#: than the feature needs.
DEFAULT_STATE_DIR = "/var/lib/hhstream-mcp"
ENV_STATE_DIR = "HHSTREAM_MCP_STATE_DIR"
#: The file, and the lock beside it.
STORE_NAME = "authorizations.json"
LOCK_NAME = ".authorizations.lock"
#: Prefix on every minted token. Not a namespace and not parsed on the way in
#: -- it is there so a leaked token is identifiable on sight, in a log, a shell
#: history or a committed config, which is the convention secret scanners are
#: built around.
TOKEN_PREFIX = "hhsmcp_"
#: 32 bytes of ``secrets`` is 256 bits. Nothing about the length is a
#: trade-off: the token is pasted into a client configuration, never typed and
#: never read aloud, which is exactly why this is not
#: :mod:`hhstream_live.phrase` -- a phrase buys sayability with 77 bits, and
#: there is nobody here to say it to.
TOKEN_BYTES = 32
#: The on-disk format. Written on every save and refused on load if unknown,
#: so a file from a future version is a named error rather than a table that
#: silently loses the fields this code does not know about.
FORMAT_VERSION = 1
#: How often ``last_used`` is allowed to reach the disk. A write per request
#: would turn a read-only tool call into a file replace and a lock round-trip;
#: the stamp is for an operator asking "is this token still in use", and an
#: hour is well inside that question's resolution. Same reasoning, and the
#: same shape, as ``registry.ACTIVITY_FLUSH_INTERVAL_S``.
LAST_USED_FLUSH_S = 3600.0
#: Bound on the operator's label, so one pathological ``--label`` cannot make
#: the table unreadable.
MAX_LABEL_CHARS = 128
class StoreError(Exception):
"""The store could not be used. Carries a message fit for a CLI error."""
class NoSuchGrant(StoreError):
"""A token id that is not in the table."""
def hash_token(token: str) -> str:
"""The digest stored for ``token``. Hex, lowercase, 64 characters."""
return hashlib.sha256(token.encode("utf-8")).hexdigest()
def mint_token() -> str:
"""A fresh bearer token. Returned once and never stored."""
return TOKEN_PREFIX + secrets.token_urlsafe(TOKEN_BYTES)
def mint_id(taken: set[str]) -> str:
"""``<6 base32 chars>`` -- the handle revocation uses.
An id and not the token: ``token revoke`` takes this, so ending a grant
never means pasting a live secret onto a command line that lands in a shell
history. Shaped like :func:`hhstreamd.registry.mint_id`'s suffix for the
same reason -- base32 lowercase has no character a reader confuses.
"""
for _ in range(1000):
raw = secrets.token_bytes(5)
candidate = base64.b32encode(raw).decode("ascii").rstrip("=").lower()[:6]
if candidate not in taken:
return candidate
raise StoreError("could not mint an unused token id") # pragma: no cover
@dataclass
class Grant:
"""One authorization: what the table knows about a token it cannot show."""
id: str
hash: str
label: str = ""
created: str = ""
expires_at: float | None = None
last_used: str | None = None
#: The toolsets this token may call, or ``None`` for every tool. Stored
#: from the first version even though the HTTP server is what enforces it,
#: because ``--toolsets`` is a *launch-time* flag and the moment one
#: process serves several callers a per-process tool budget has stopped
#: being a per-caller one. The token is the only place that distinction
#: can live, and adding the field later would be a file-format migration
#: for no reason.
toolsets: tuple[str, ...] | None = None
def to_dict(self) -> dict[str, Any]:
return {
"id": self.id,
"hash": self.hash,
"label": self.label,
"created": self.created,
"expires_at": self.expires_at,
"last_used": self.last_used,
"toolsets": list(self.toolsets) if self.toolsets is not None else None,
}
@classmethod
def from_dict(cls, raw: dict[str, Any]) -> Grant:
toolsets = raw.get("toolsets")
return cls(
id=str(raw["id"]),
hash=str(raw["hash"]),
label=str(raw.get("label") or ""),
created=str(raw.get("created") or ""),
expires_at=(
float(raw["expires_at"]) if raw.get("expires_at") is not None else None
),
last_used=raw.get("last_used") or None,
toolsets=tuple(str(name) for name in toolsets) if toolsets else None,
)
def expired(self, now: float) -> bool:
return self.expires_at is not None and now >= self.expires_at
def expires_iso(self) -> str | None:
return iso(self.expires_at) if self.expires_at is not None else None
@dataclass
class Store:
"""The table on disk, read and written under a lock."""
path: Path
clock: Clock = field(default_factory=Clock)
@classmethod
def at(cls, state_dir: Path | str | None = None, clock: Clock | None = None) -> Store:
"""The store under ``state_dir``, ``$HHSTREAM_MCP_STATE_DIR``, or the default."""
root = Path(
state_dir
if state_dir is not None
else os.environ.get(ENV_STATE_DIR, DEFAULT_STATE_DIR)
)
return cls(path=root / STORE_NAME, clock=clock or Clock())
@property
def state_dir(self) -> Path:
return self.path.parent
# ---- the lock ----------------------------------------------------
@contextlib.contextmanager
def _locked(self) -> Iterator[None]:
"""Exclusive access for one read-modify-write.
The lock is a sidecar rather than the store itself, because the store
is *replaced* on every write -- a descriptor held on it would be a
descriptor on an unlinked inode, and the next writer would lock a
different file and think itself alone.
"""
self.ensure_dir()
lock_path = self.state_dir / LOCK_NAME
fd = os.open(lock_path, os.O_CREAT | os.O_RDWR, 0o600)
try:
fcntl.flock(fd, fcntl.LOCK_EX)
yield
finally:
os.close(fd)
def ensure_dir(self) -> None:
"""Create the state directory if the supervisor has not.
In the deployed shape ``StateDirectory=`` in the unit makes this (and
its mode and owner) declarative; the same reasoning, and the same
fallback, as :meth:`hhstreamd.registry.Registry.ensure_dirs`.
"""
try:
self.state_dir.mkdir(parents=True, exist_ok=True, mode=0o700)
except OSError as exc:
raise StoreError(f"cannot create {self.state_dir}: {exc}") from exc
# ---- reading -----------------------------------------------------
def load(self) -> list[Grant]:
"""Every grant, newest last. A missing file is an empty table."""
try:
raw = self.path.read_text(encoding="utf-8")
except FileNotFoundError:
return []
except OSError as exc:
raise StoreError(f"cannot read {self.path}: {exc}") from exc
try:
data = json.loads(raw)
except json.JSONDecodeError as exc:
raise StoreError(f"{self.path} is not valid JSON: {exc}") from exc
if not isinstance(data, dict):
raise StoreError(f"{self.path} is not an authorization table")
version = data.get("version")
if version != FORMAT_VERSION:
raise StoreError(
f"{self.path} is format version {version!r}, and this build "
f"understands {FORMAT_VERSION}. A newer hhstream-mcp wrote it."
)
try:
return [Grant.from_dict(entry) for entry in data.get("grants", [])]
except (KeyError, TypeError, ValueError) as exc:
raise StoreError(f"{self.path} has an unreadable grant: {exc}") from exc
def _write(self, grants: Sequence[Grant]) -> None:
"""Replace the table. Call under :meth:`_locked`."""
self.ensure_dir()
atomic_write_json(
self.path,
{"version": FORMAT_VERSION, "grants": [g.to_dict() for g in grants]},
)
# `atomic_write_json` creates the temp file with the default mode, so
# the mode is set on the thing that survives the replace. The file is
# not a credential (it holds digests), but it is the list of who may
# drive this desktop, and that is nobody else's business either.
try:
os.chmod(self.path, 0o600)
except OSError as exc: # pragma: no cover -- the write just succeeded
log.warning("cannot chmod %s: %s", self.path, exc)
# ---- writing -----------------------------------------------------
def mint(
self,
*,
label: str = "",
ttl: float | None = None,
toolsets: Sequence[str] | None = None,
) -> tuple[Grant, str]:
"""Add a grant and return it with its token. The token is not stored."""
token = mint_token()
with self._locked():
grants = self.load()
now = self.clock.wall()
grant = Grant(
id=mint_id({g.id for g in grants}),
hash=hash_token(token),
label=label[:MAX_LABEL_CHARS],
created=iso(now),
expires_at=(now + ttl) if ttl is not None else None,
toolsets=tuple(toolsets) if toolsets else None,
)
self._write([*grants, grant])
return grant, token
def revoke(self, grant_id: str) -> Grant:
"""Remove one grant by id. Raises :class:`NoSuchGrant` if it is not there."""
with self._locked():
grants = self.load()
kept = [g for g in grants if g.id != grant_id]
if len(kept) == len(grants):
raise NoSuchGrant(
f"no token with id {grant_id!r}. `hhstream-mcp token list` "
"shows the ids."
)
gone = next(g for g in grants if g.id == grant_id)
self._write(kept)
return gone
def revoke_all(self) -> list[Grant]:
"""Empty the table. Returns what was in it."""
with self._locked():
grants = self.load()
self._write([])
return grants
def prune(self) -> list[Grant]:
"""Drop expired grants. Returns what was dropped.
Not required for correctness -- :meth:`verify` refuses an expired grant
whether or not it is still on disk -- so this is housekeeping, and it
is a separate call rather than a side effect of a read for that reason.
"""
now = self.clock.wall()
with self._locked():
grants = self.load()
dead = [g for g in grants if g.expired(now)]
if dead:
self._write([g for g in grants if not g.expired(now)])
return dead
# ---- the hot path ------------------------------------------------
def verify(self, token: str, *, stamp: bool = True) -> Grant | None:
"""The grant this token belongs to, or ``None``.
``None`` covers every way a token can fail -- unknown, revoked, expired
-- on purpose. The caller's answer is one 401 either way, and a richer
return would only tempt it into saying which, to someone who should not
be told they guessed a real id.
The digest comparison is :func:`hmac.compare_digest` even though both
sides are already hashes: the cost is nothing and it keeps the function
free of a timing signal that a later refactor might otherwise make
meaningful.
"""
if not token:
return None
digest = hash_token(token)
now = self.clock.wall()
try:
grants = self.load()
except StoreError as exc:
# A broken table denies everything rather than allowing anything.
log.error("authorization table unusable, refusing: %s", exc)
return None
for grant in grants:
if not hmac.compare_digest(grant.hash, digest):
continue
if grant.expired(now):
return None
if stamp:
self._stamp(grant, now)
return grant
return None
def _stamp(self, grant: Grant, now: float) -> None:
"""Record that ``grant`` was used, at most once per flush interval.
Never raises: a stamp that could not be written is not a reason to
refuse a request that was authorized. Same trade as the audit log's
(:mod:`hhstreamd.audit`).
"""
if grant.last_used:
try:
age = now - _epoch(grant.last_used)
except ValueError:
age = LAST_USED_FLUSH_S + 1.0
if age < LAST_USED_FLUSH_S:
return
stamped = iso(now)
try:
with self._locked():
grants = self.load()
for entry in grants:
if entry.id == grant.id:
entry.last_used = stamped
break
else:
# Revoked between the verify and the stamp. Nothing to
# record, and re-adding it would resurrect a dead grant.
return
self._write(grants)
except (StoreError, OSError) as exc:
log.warning("cannot stamp last_used for %s: %s", grant.id, exc)
return
grant.last_used = stamped
def _epoch(stamp: str) -> float:
"""An ISO stamp this module wrote, back as an epoch."""
import datetime as _dt
return _dt.datetime.fromisoformat(stamp.replace("Z", "+00:00")).timestamp()
__all__ = [
"DEFAULT_STATE_DIR",
"ENV_STATE_DIR",
"FORMAT_VERSION",
"Grant",
"NoSuchGrant",
"STORE_NAME",
"Store",
"StoreError",
"TOKEN_PREFIX",
"hash_token",
"mint_token",
]
+516
View File
@@ -0,0 +1,516 @@
"""The HTTP transport: Streamable HTTP, behind a bearer token.
The second module here that imports the SDK, and the only one that imports
Starlette. :mod:`hhstream_mcp.server` stays what it was -- the stdio wiring --
and nothing in this file is reachable without ``serve --http``.
Why there is a second transport at all
--------------------------------------
Because a token has to travel somewhere. ``docs/CONTRACT-mcp.md`` v1 and v2
said "stdio, and only stdio", and the reasoning was sound for what the server
then was: the client spawns it, so the caller has already passed a filesystem
permission check, and `0660` on the broker's control socket is the real gate.
Authentication on that shape is theatre -- the caller holds the key to the lock
they are being asked to open.
A port changes who the caller is. The request now arrives from someone who
never touched the filesystem, and the token is the only thing standing between
them and a desktop they can type into. So the transport and the credential are
one feature and arrive together; neither is useful without the other.
**stdio remains the default and remains unauthenticated**, for the reason it
always was. This is not auth bolted onto the MCP server; it is a second
transport that needs it.
What this is not
----------------
**Not MCP's authorization specification.** That is OAuth 2.1 plus RFC 9728
protected-resource metadata: a `401` is supposed to carry a
`WWW-Authenticate` pointing at a metadata document, the client discovers an
authorization server and does a real flow. This serves a static token the
operator minted and pasted into a client config. Consequences, stated rather
than discovered: a client that only knows how to *discover* an authorization
server will not connect, and one that lets you set a header will. See
``docs/TODO/mcp-http-auth-is-not-oauth.md``.
The deliberate choices in the request path
------------------------------------------
* **Loopback by default.** ``--address`` defaults to ``127.0.0.1``. Reaching
the LAN is a second decision, taken twice (the flag and a firewall).
* **DNS-rebinding protection on**, with host and origin allowlists, through the
SDK's own :class:`TransportSecuritySettings`. A browser on the operator's
network is otherwise a usable path to a loopback-bound port.
* **A refusal says nothing.** `401` with an empty body; the token is never
echoed, never logged, and the response does not distinguish unknown from
revoked from expired. The same discipline `hhstream_live.server` already
applies to a wrong phrase, for the same reason.
* **Failed attempts are rate-limited per client**, reusing that app's
:class:`~hhstream_live.server.RateLimiter` rather than growing a second one.
256-bit tokens make guessing hopeless already; this is so an attacker cannot
even try at line speed and so the attempt is visible in one place.
One operational property worth knowing before enabling this
---------------------------------------------------------
**Tool calls are serialised across every caller, not per session.**
:class:`hhstream_mcp.invoke.Runner` holds a process-wide lock because the CLI
it calls prints to a stdout this process shares, and that was invisible while
each client had a process of its own. It is visible here: two agents on two
tokens queue behind each other, and a slow call (a long ``record``, a ``grim``
wedged on a compositor that has stopped answering) delays everyone's next call
rather than only its own caller's.
The event loop stays responsive throughout -- a ``ping`` is answered, a
resource read is still off the loop in its own thread -- so this is latency and
never a hang. It is the same limit
``docs/TODO/mcp-in-process-has-no-call-deadline.md`` already describes, with
one more consequence now that the transport is shared.
"""
from __future__ import annotations
import logging
from collections.abc import Callable, Sequence
from typing import Any
import anyio
from mcp.server.lowlevel import NotificationOptions
from mcp.server.streamable_http_manager import StreamableHTTPSessionManager
from mcp.server.transport_security import TransportSecuritySettings
from hhstream_live.server import RateLimiter
from . import (
DEFAULT_HTTP_ADDRESS,
DEFAULT_HTTP_PATH,
DEFAULT_HTTP_PORT,
SERVER_NAME,
tools,
)
from .authorizations import Grant, Store
from .invoke import Runner
log = logging.getLogger("hhstream.mcp.http")
#: Aliases, so this module reads in its own terms. The values live in
#: ``hhstream_mcp/__init__.py``: ``server.py`` builds the parser that advertises
#: them and must not import this module to do it.
DEFAULT_ADDRESS = DEFAULT_HTTP_ADDRESS
DEFAULT_PORT = DEFAULT_HTTP_PORT
DEFAULT_PATH = DEFAULT_HTTP_PATH
#: Where the verified grant is parked for the duration of one request. The
#: ASGI scope and not a ``contextvar``: in stateful mode the session manager
#: runs the protocol handler in a task owned by the *manager's* lifespan, not
#: by the request, so a contextvar set in middleware would not be visible
#: there. The scope survives, because the transport builds its own
#: ``Request(scope, receive)`` from the very dict this middleware annotated.
SCOPE_KEY = "hhstream_grant"
#: The scheme, spelled once. Compared case-insensitively on the way in: RFC
#: 9110 makes the scheme token case-insensitive and some clients send `bearer`.
SCHEME = "bearer"
#: Addresses that mean "this machine only", and for which a Host allowlist can
#: therefore be derived rather than demanded.
LOOPBACK = frozenset({"127.0.0.1", "::1", "localhost"})
#: Addresses that mean "every interface". Nothing can be derived from one:
#: the Host header will be whatever name the client used to reach us, and this
#: process has no way to know what that is.
WILDCARD = frozenset({"0.0.0.0", "::", "*"}) # noqa: S104 -- named, not bound
def default_allowed_hosts(address: str, port: int) -> list[str]:
"""The Host values to accept when the operator named none.
This has to exist because the SDK's DNS-rebinding check is *allowlist only*
with no implicit localhost: :class:`TransportSecuritySettings` with
protection on and an empty ``allowed_hosts`` rejects **every** request with
a 421. Turning the protection off to get a working default would be the
wrong trade -- it is the one thing standing between a loopback-bound port
and any web page the operator opens -- so the default is derived instead.
A wildcard bind returns the empty list, and the caller refuses to start.
That is deliberate: binding every interface is exactly the case where the
hostnames cannot be guessed and are worth being made to say out loud.
"""
if address in WILDCARD:
return []
hosts = {address} | (LOOPBACK if address in LOOPBACK else set())
out: list[str] = []
for host in sorted(hosts):
# `::1` has to be bracketed to appear in a Host header at all.
literal = f"[{host}]" if ":" in host else host
out.append(f"{literal}:{port}")
# The `:*` form is the SDK's own wildcard-port pattern. A client that
# reaches a reverse proxy on 443 and is forwarded here sends the port
# it dialled, not ours.
out.append(f"{literal}:*")
out.append(literal)
return out
def bearer_token(header: str) -> str | None:
"""The token out of an ``Authorization`` value, or ``None``.
Anything that is not exactly one ``Bearer <token>`` pair is ``None`` -- a
malformed header is a failed attempt, not a thing to repair on the
caller's behalf.
"""
parts = header.strip().split(None, 1)
if len(parts) != 2 or parts[0].lower() != SCHEME:
return None
token = parts[1].strip()
return token or None
class BearerAuth:
"""ASGI middleware: every HTTP request carries a live token, or gets a 401.
Hand-written rather than the SDK's ``RequireAuthMiddleware``, which is
built for the OAuth profile this is not: it wants ``AuthSettings`` with an
issuer and a resource-server URL, and it advertises metadata endpoints that
would then have to exist. Forty lines that say what they do beat a
configuration that claims a flow nobody implemented.
"""
def __init__(
self,
app: Any,
store: Store,
*,
limiter: RateLimiter | None = None,
) -> None:
self.app = app
self.store = store
self.limiter = limiter if limiter is not None else RateLimiter()
async def __call__(self, scope: dict[str, Any], receive: Any, send: Any) -> None:
# `lifespan` is the server talking to itself; only `http` carries a
# caller.
if scope["type"] == "lifespan":
await self.app(scope, receive, send)
return
if scope["type"] != "http":
# Nothing here mounts a websocket, so this is unreachable today --
# but it is refused in that protocol's own terms rather than with
# an `http.response.start` a websocket client cannot read. An
# unreachable branch that would send the wrong frame is a trap for
# whoever makes it reachable.
log.warning("refusing a %r scope: this server is http only", scope["type"])
await send({"type": "websocket.close", "code": 1008})
return
client = (scope.get("client") or ("unknown", 0))[0]
header = _header(scope, b"authorization")
token = bearer_token(header) if header else None
if token is None:
await self._refuse(send, client=client, reason="no bearer token")
return
# A file read, so off the event loop: small, but the discipline is the
# one the rest of this package keeps about blocking work.
grant = await anyio.to_thread.run_sync(lambda: self.store.verify(token))
if grant is None:
await self._refuse(send, client=client, reason="token refused")
return
scope[SCOPE_KEY] = grant
log.debug("request authorized by token %s", grant.id)
await self.app(scope, receive, send)
async def _refuse(
self,
send: Any,
*,
client: str = "unknown",
reason: str = "unauthorized",
) -> None:
"""401, or 429 once a client has missed too often.
The body is empty in both cases and the log line names the client and
the shape of the failure, never the token or any part of it.
"""
allowed = self.limiter.allow(client)
status = 401 if allowed else 429
log.warning("%s from %s: %s", status, client, reason if allowed else "rate limited")
headers: list[tuple[bytes, bytes]] = []
if status == 401:
# RFC 9110's challenge. A realm and nothing else: there is no
# metadata document to point at, because there is no authorization
# server -- see `docs/TODO/mcp-http-auth-is-not-oauth.md`.
headers.append((b"www-authenticate", f'Bearer realm="{SERVER_NAME}"'.encode()))
await _empty(send, status, headers)
def _header(scope: dict[str, Any], name: bytes) -> str:
for key, value in scope.get("headers") or ():
if key.lower() == name:
return value.decode("latin-1")
return ""
def current_grant(server: Any) -> Grant | None:
"""The grant behind the request being handled, if there is one.
``None`` whenever there is no request context at all -- which is the stdio
case and the unit tests' case -- so a caller can treat "no grant" as "no
restriction" without asking which transport it is on.
"""
try:
request = server.request_context.request
except (LookupError, AttributeError):
return None
scope = getattr(request, "scope", None) or {}
grant = scope.get(SCOPE_KEY)
return grant if isinstance(grant, Grant) else None
def offered_for(server: Any, base: tuple[tools.Tool, ...]) -> Callable[[], tuple[tools.Tool, ...]]:
"""A chooser that narrows ``base`` to the calling token's toolsets.
The **intersection**, not the token's list: the process was started with
its own ``--toolsets`` and a token cannot widen that. Two independent
narrowings, and a tool has to survive both.
Resources are deliberately not narrowed, which is worth stating because it
is a real limit and not an oversight. `docs/CONTRACT-mcp.md` §2 keeps
resources out of ``--toolsets`` on the grounds that they are the host's
context rather than the model's budget, and that argument carries over. But
a *token* is an authorization rather than a budget, so the consequence is
that a token scoped to ``frames`` can still read ``hhstream://sessions``.
Read-side scoping is a different feature; see
``docs/TODO/mcp-tokens-do-not-scope-resources.md``.
"""
names = {tool.name for tool in base}
def chooser() -> tuple[tools.Tool, ...]:
grant = current_grant(server)
if grant is None or grant.toolsets is None:
return base
try:
allowed = tools.selected(grant.toolsets)
except tools.UnknownToolset:
# A toolset name that was valid when the token was minted and is
# not a name this build knows. Refusing everything is the safe
# direction: the alternative is a token that silently widens.
log.error(
"token %s names a toolset this build does not know (%s); "
"offering nothing",
grant.id,
",".join(grant.toolsets),
)
return ()
return tuple(tool for tool in allowed if tool.name in names)
return chooser
class Endpoint:
"""The ASGI app under :class:`BearerAuth`: one path, and the lifespan.
Hand-rolled rather than ``Starlette(routes=[Mount(path, app=…)])``, and
the reason is a 307 that costs real clients their request. A Starlette
``Mount`` at ``/mcp`` serves ``/mcp/`` and answers ``/mcp`` with a redirect
(or a 404 with ``redirect_slashes`` off) -- measured, both ways. Every MCP
client configuration spells the URL without the trailing slash, and a
redirected **POST** has to be re-sent with its body, which ``urllib`` will
not do at all and some clients get wrong. So the path is matched here,
exactly, with and without the slash, and nothing redirects.
It also means the only routing in this server is three lines long and in
one place, which is the whole of what a single-endpoint server needs.
"""
def __init__(self, manager: StreamableHTTPSessionManager, path: str) -> None:
self.manager = manager
# Both spellings are the endpoint; neither is a redirect to the other.
self.paths = {path, path.rstrip("/") + "/"} if path != "/" else {"/"}
async def __call__(self, scope: dict[str, Any], receive: Any, send: Any) -> None:
if scope["type"] == "lifespan":
await self._lifespan(receive, send)
return
if scope.get("path") in self.paths:
await self.manager.handle_request(scope, receive, send)
return
await _empty(send, 404)
async def _lifespan(self, receive: Any, send: Any) -> None:
"""The startup/shutdown protocol, by hand.
``manager.run()`` has to stay open for the life of the process: it owns
the task group the per-session server tasks live in. Entered on
``startup`` and exited on ``shutdown``, both of which arrive in this
one coroutine -- which matters, because an anyio task group must be
exited by the task that entered it.
"""
running: Any = None
while True:
message = await receive()
kind = message["type"]
if kind == "lifespan.startup":
try:
running = self.manager.run()
await running.__aenter__()
except BaseException as exc: # noqa: BLE001 -- reported, not swallowed
log.exception("http transport failed to start")
await send({"type": "lifespan.startup.failed", "message": repr(exc)})
return
log.info("http transport ready on %s", ", ".join(sorted(self.paths)))
await send({"type": "lifespan.startup.complete"})
elif kind == "lifespan.shutdown":
if running is not None:
await running.__aexit__(None, None, None)
await send({"type": "lifespan.shutdown.complete"})
return
async def _empty(send: Any, status: int, headers: list[tuple[bytes, bytes]] | None = None) -> None:
"""A response with no body, which is every response this layer writes itself."""
await send(
{
"type": "http.response.start",
"status": status,
"headers": [
(b"content-length", b"0"),
(b"cache-control", b"no-store"),
(b"x-content-type-options", b"nosniff"),
*(headers or ()),
],
}
)
await send({"type": "http.response.body", "body": b""})
def build_app(
runner: Runner,
store: Store,
*,
toolsets: Sequence[str] | None = None,
path: str = DEFAULT_PATH,
allowed_hosts: Sequence[str] | None = None,
allowed_origins: Sequence[str] | None = None,
stateless: bool = False,
json_response: bool = False,
) -> Any:
"""The ASGI application: one endpoint, behind :class:`BearerAuth`.
``allowed_hosts`` must already be resolved -- :func:`default_allowed_hosts`
is the caller's to apply, because the empty list has to stay meaningful
here: it is what a wildcard bind produces, and it is a refusal rather than
a default.
``stateless`` defaults to false -- a client that initializes and then calls
gets one session, which is what the protocol's session header is for. It is
exposed because a deployment that puts several of these behind a load
balancer cannot keep a session pinned, and a stateless server is then the
only honest answer.
"""
from .server import build
security = TransportSecuritySettings(
enable_dns_rebinding_protection=True,
allowed_hosts=list(allowed_hosts or ()),
# Left empty on purpose when the operator names none. The SDK lets an
# *absent* Origin through and refuses any present one it does not know,
# which is exactly right here: no MCP client sends an Origin, and the
# only thing that does is a web page -- which is the attacker in the
# DNS-rebinding story, not a user of this server.
allowed_origins=list(allowed_origins or ()),
)
base = tools.selected(toolsets)
# The two references are circular: the server lists what the chooser
# returns, and the chooser reads the server's request context. One of them
# has to resolve late, and this is the one that safely can -- `chooser` is
# called per request, long after the next statement has bound it.
chooser: Callable[[], tuple[tools.Tool, ...]]
mcp_server = build(runner, toolsets, offered=lambda: chooser())
chooser = offered_for(mcp_server, base)
manager = StreamableHTTPSessionManager(
app=mcp_server,
stateless=stateless,
json_response=json_response,
security_settings=security,
)
# The token check wraps the *whole* app and not just the endpoint, so a
# request to any other path is a 401 rather than a 404. An unauthenticated
# caller learns nothing about what this server has mounted, including
# whether it is an hhstream server at all.
return BearerAuth(Endpoint(manager, path), store)
def serve(
runner: Runner,
store: Store,
*,
address: str = DEFAULT_ADDRESS,
port: int = DEFAULT_PORT,
path: str = DEFAULT_PATH,
toolsets: Sequence[str] | None = None,
allowed_hosts: Sequence[str] | None = None,
allowed_origins: Sequence[str] | None = None,
stateless: bool = False,
json_response: bool = False,
) -> None:
"""Run the HTTP transport until the process is signalled."""
import uvicorn
app = build_app(
runner,
store,
toolsets=toolsets,
path=path,
allowed_hosts=allowed_hosts,
allowed_origins=allowed_origins,
stateless=stateless,
json_response=json_response,
)
config = uvicorn.Config(
app,
host=address,
port=port,
# stderr, like everything else this package logs: the stdio transport's
# discipline about fd 1 is not relaxed just because this process is not
# using it, since the same binary does both.
log_config=None,
access_log=False,
# The default `h11` is fine and the only one in the closure; naming it
# keeps a uvicorn that grew a different default from changing ours.
http="h11",
)
uvicorn.Server(config).run()
def initialization_options(server: Any) -> Any:
"""The handshake options, shared with the stdio path."""
return server.create_initialization_options(NotificationOptions())
__all__ = [
"DEFAULT_ADDRESS",
"DEFAULT_PATH",
"DEFAULT_PORT",
"SCOPE_KEY",
"BearerAuth",
"Endpoint",
"bearer_token",
"build_app",
"current_grant",
"default_allowed_hosts",
"offered_for",
"serve",
]
+11 -1
View File
@@ -29,13 +29,23 @@ requires-python = ">=3.11"
# SDK: `server.py` calls `anyio.run` and `anyio.to_thread` directly, and a
# dependency used by name belongs in the metadata.
#
# `starlette` and `uvicorn` are declared for exactly that reason and no other.
# The SDK pulls both in today -- its streamable-HTTP transport needs them -- so
# leaving them out would work, which is the trap: `http.py` imports
# `uvicorn` and the SDK's `TransportSecuritySettings` sits on Starlette's
# `Request`, and an SDK release that moved either behind an optional extra
# would break `serve --http` with an ImportError naming someone else's package.
# They are only needed by the HTTP transport, which is why they are imported
# inside the functions that use them rather than at module scope: the stdio
# path must neither pay for them nor fail on them.
#
# `hhstream` is NOT listed, and it is not optional either: this package imports
# `hhstream.cli`, `hhstreamd.protocol` and `hhstream_session` and does not work
# without them. It is absent because `hhstream` is not a wheel anyone installs
# from an index -- it is this repository's other derivation, and `nix/mcp.nix`
# is where the two are joined. A dependency naming a version nobody can
# resolve would be worse than the comment you are reading.
dependencies = ["mcp>=1.9", "anyio>=4"]
dependencies = ["mcp>=1.9", "anyio>=4", "starlette>=0.40", "uvicorn>=0.30"]
[project.scripts]
# Stdio only. One process per client, spawned by the client; no port, no daemon.
+295 -34
View File
@@ -31,6 +31,7 @@ import io
import logging
import os
import sys
from collections.abc import Callable
from typing import Any
import anyio
@@ -41,7 +42,17 @@ from mcp.server.stdio import stdio_server
from mcp.shared.exceptions import McpError
from pydantic import AnyUrl
from . import SERVER_NAME, __version__, resources, tools
from . import (
DEFAULT_HTTP_ADDRESS,
DEFAULT_HTTP_PATH,
DEFAULT_HTTP_PORT,
SERVER_NAME,
__version__,
authorizations,
resources,
tokencli,
tools,
)
from .invoke import Runner
log = logging.getLogger("hhstream.mcp")
@@ -177,9 +188,16 @@ def call(
"resources/list for the whole read-only half of this server."
)
elif name in tools.BY_NAME:
# Two things can have narrowed the offer and the agent cannot tell
# them apart from here, so both are named: `--toolsets` at launch,
# and -- on the HTTP transport -- the toolsets carried by the token
# the request presented. Neither is the agent's to change, which is
# why the remedy is "ask whoever runs this" and not a retry.
hint = (
f" {name!r} exists but this server was started without its "
"toolset, so it is not offered."
f" {name!r} exists but is not offered to this caller: either "
"this server was started without its toolset (--toolsets), or "
"the authorization this request used does not carry it. "
"Neither is yours to change from here."
)
return types.CallToolResult(
content=[
@@ -267,16 +285,33 @@ def read(runner: Runner, uri: str) -> list[ReadResourceContents]:
return [ReadResourceContents(content=body, mime_type=reading.media_type)]
def build(runner: Runner, toolsets: Any = None) -> Server:
def build(
runner: Runner,
toolsets: Any = None,
*,
offered: Callable[[], tuple[tools.Tool, ...]] | None = None,
) -> Server:
"""The MCP server over the selected tools.
``offered`` is how the HTTP transport narrows the listing per *caller*: a
token can carry its own toolsets, and ``--toolsets`` is a launch-time flag
that stops being a per-caller budget the moment one process serves several
of them (:mod:`hhstream_mcp.http`). It is called on every ``tools/list``
and every ``tools/call``, so it must be cheap and must not block.
Omitted -- which is the stdio case -- the selection is fixed at build time
and the handlers close over it, exactly as before.
"""
server: Server = Server(
SERVER_NAME, version=__version__, instructions=INSTRUCTIONS
)
available = tools.selected(toolsets)
by_name = {tool.name: tool for tool in available}
fixed = tools.selected(toolsets)
chooser: Callable[[], tuple[tools.Tool, ...]] = offered or (lambda: fixed)
@server.list_tools()
async def _list() -> list[types.Tool]:
return [describe(tool) for tool in available]
return [describe(tool) for tool in chooser()]
@server.list_resources()
async def _list_resources() -> list[types.Resource]:
@@ -302,6 +337,7 @@ def build(runner: Runner, toolsets: Any = None) -> Server:
# loop stays free to answer a ping and to read the next request. The
# calls themselves are serialised inside `Runner.run`, because the CLI
# prints to a stdout this process shares.
by_name = {tool.name: tool for tool in chooser()}
return await anyio.to_thread.run_sync(lambda: call(runner, name, arguments, by_name))
return server
@@ -345,13 +381,146 @@ async def serve(server: Server, transport: tuple[Any, Any] | None = None) -> Non
)
def common_options() -> argparse.ArgumentParser:
"""The flags both ``hhstream-mcp`` and ``hhstream-mcp serve`` accept.
A parent parser, so ``--toolsets`` works on either side of the subcommand
and there is one help text for it. Every option here defaults to
``SUPPRESS``: argparse copies a subparser's namespace over the top-level
one, so a plain default on the subcommand copy would overwrite a value
given *before* the subcommand with ``None``. Suppressed, an option that was
not typed is simply absent, and callers read it with ``getattr``.
"""
common = argparse.ArgumentParser(add_help=False)
common.add_argument(
"--toolsets",
metavar="NAMES",
default=argparse.SUPPRESS,
help="comma-separated families to offer as tools: "
+ ", ".join(tools.TOOLSETS)
+ ". Omitted, every tool is offered. Resources are NOT filtered by "
"this -- they are the host's context, not the model's tool budget.",
)
common.add_argument(
"--list-tools",
action="store_true",
default=argparse.SUPPRESS,
help="print the tool names and exit, without serving. Honours "
"--toolsets, so it is also how to see what a selection would offer.",
)
common.add_argument(
"--list-resources",
action="store_true",
default=argparse.SUPPRESS,
help="print the resource URIs and templates and exit, without serving.",
)
return common
def add_serve_parser(subparsers: Any, common: argparse.ArgumentParser) -> None:
"""``serve`` -- the transports. stdio unless ``--http`` says otherwise."""
serve_cmd = subparsers.add_parser(
"serve",
parents=[common],
help="serve the tools (stdio by default, --http for a port)",
description=(
"Serve the surface. With no flags this is the stdio transport and "
"is identical to running `hhstream-mcp` with no subcommand at all, "
"which is what an MCP client's `command` entry should keep doing."
),
)
http = serve_cmd.add_argument_group(
"http transport",
description=(
"Streamable HTTP instead of stdio, behind a bearer token minted by "
"`hhstream-mcp token mint`. Every request carries one or gets a "
"401. stdio stays unauthenticated on purpose: the client spawns it, "
"so the caller has already passed a filesystem permission check "
"and holds the broker's socket either way."
),
)
http.add_argument(
"--http",
action="store_true",
help="serve Streamable HTTP on --address:--port instead of stdio",
)
http.add_argument(
"--address",
default=DEFAULT_HTTP_ADDRESS,
metavar="HOST",
help="the address to bind (default: %(default)s). Loopback by default, "
"deliberately: reaching the LAN is a second decision and wants a "
"firewall rule to go with it.",
)
http.add_argument(
"--port",
type=int,
default=DEFAULT_HTTP_PORT,
help="the port to bind (default: %(default)s)",
)
http.add_argument(
"--path",
default=DEFAULT_HTTP_PATH,
help="the path the endpoint is mounted at (default: %(default)s)",
)
http.add_argument(
"--state-dir",
metavar="DIR",
help="where the authorization table lives. Defaults to "
"$HHSTREAM_MCP_STATE_DIR, then /var/lib/hhstream-mcp.",
)
http.add_argument(
"--allowed-host",
action="append",
default=[],
metavar="HOST",
help="a Host header to accept. Repeatable. DNS-rebinding protection is "
"on, so a host this does not name is refused by the transport before "
"the token is even looked at.",
)
http.add_argument(
"--allowed-origin",
action="append",
default=[],
metavar="ORIGIN",
help="an Origin header to accept. Repeatable. Nothing needs this unless "
"a browser is the client; a token in a page is a token in a page.",
)
http.add_argument(
"--stateless",
action="store_true",
help="handle every request on its own, with no session to pin. For a "
"deployment behind a load balancer that cannot keep a client on one "
"process.",
)
http.add_argument(
"--allow-no-tokens",
action="store_true",
help="start even though the authorization table is empty, which means "
"refusing every request until a token is minted. For provisioning the "
"unit before its first token exists; without it, an empty table is a "
"usage error, because it is almost always a forgotten `token mint`.",
)
http.add_argument(
"--json-response",
action="store_true",
help="answer with a single JSON body instead of an SSE stream",
)
def build_parser() -> argparse.ArgumentParser:
common = common_options()
parser = argparse.ArgumentParser(
prog="hhstream-mcp",
parents=[common],
description=(
"Serve the hhstream CLI as Model Context Protocol tools over stdio. "
"Spawned by an MCP client; it speaks JSON-RPC on stdin/stdout and "
"logs to stderr."
"Serve the hhstream CLI as Model Context Protocol tools. The "
"default transport is stdio: spawned by an MCP client, speaking "
"JSON-RPC on stdin/stdout and logging to stderr. `serve --http` "
"binds a port instead and requires a bearer token; `token` mints "
"and revokes those."
),
epilog=(
"There is no --socket flag here, deliberately: the sockets are the "
@@ -361,25 +530,12 @@ def build_parser() -> argparse.ArgumentParser:
"disagree. Set them in the MCP client's `env` block."
),
)
parser.add_argument(
"--toolsets",
metavar="NAMES",
help="comma-separated families to offer as tools: "
+ ", ".join(tools.TOOLSETS)
+ ". Omitted, every tool is offered. Resources are NOT filtered by "
"this -- they are the host's context, not the model's tool budget.",
)
parser.add_argument(
"--list-tools",
action="store_true",
help="print the tool names and exit, without serving. Honours "
"--toolsets, so it is also how to see what a selection would offer.",
)
parser.add_argument(
"--list-resources",
action="store_true",
help="print the resource URIs and templates and exit, without serving.",
)
# Not required: `hhstream-mcp` and `hhstream-mcp --toolsets=...` have to go
# on serving stdio exactly as they did before this file grew subcommands.
# An MCP client's `command` entry is a configuration someone else wrote.
subparsers = parser.add_subparsers(dest="command", metavar="COMMAND")
add_serve_parser(subparsers, common)
tokencli.add_parser(subparsers)
return parser
@@ -400,31 +556,44 @@ def chosen_toolsets(raw: str | None) -> list[str] | None:
def main(argv: list[str] | None = None) -> int:
parser = build_parser()
args = parser.parse_args(argv)
# stderr, never stdout: stdout is the transport, and one stray print on it
# is a protocol error the client reports as a parse failure.
# stderr, never stdout: on the stdio transport stdout *is* the transport,
# and one stray print on it is a protocol error the client reports as a
# parse failure. The HTTP transport does not share that hazard, but the
# same binary serves both and one rule is easier to keep than two.
logging.basicConfig(
level=os.environ.get("HHSTREAM_MCP_LOG", "WARNING").upper(),
stream=sys.stderr,
format="%(levelname)s %(name)s: %(message)s",
)
if args.command == "token":
# Before anything claims the transport, which is the whole reason this
# returns here: `token mint` prints a secret on stdout.
token_parser = parser_for(parser, "token")
return tokencli.run(args, token_parser)
try:
toolsets = chosen_toolsets(args.toolsets)
toolsets = chosen_toolsets(getattr(args, "toolsets", None))
offered = tools.selected(toolsets)
except tools.UnknownToolset as exc:
# argparse's own exit code for a bad argument, because that is what
# this is: `choices=` cannot express a comma-separated subset.
parser.error(str(exc))
return int(2) # pragma: no cover -- parser.error exits
if args.list_tools:
if getattr(args, "list_tools", False):
for tool in offered:
print(f"{tool.name}\t{tool.title}")
return 0
if args.list_resources:
if getattr(args, "list_resources", False):
for entry in resources.FIXED:
print(f"{entry.uri}\t{entry.title}")
for entry in resources.TEMPLATES:
print(f"{entry.uri_template}\t{entry.title}")
return 0
if getattr(args, "http", False):
return serve_http(args, toolsets, parser)
log.info(
"serving %d tools and %d resources over stdio",
len(offered),
@@ -437,5 +606,97 @@ def main(argv: list[str] | None = None) -> int:
return 0
def parser_for(parser: argparse.ArgumentParser, name: str) -> argparse.ArgumentParser:
"""The subparser called ``name``, for a handler that wants to print its help.
argparse keeps no public index of these, so it is dug out of the one
``_SubParsersAction`` this parser has. A miss returns the top-level parser:
a slightly wrong help text is a better failure than an AttributeError in
place of a usage message.
"""
for action in parser._actions: # noqa: SLF001 -- argparse has no public accessor
if isinstance(action, argparse._SubParsersAction): # noqa: SLF001
found = action.choices.get(name)
if found is not None:
return found
return parser # pragma: no cover
def serve_http(
args: argparse.Namespace,
toolsets: list[str] | None,
parser: argparse.ArgumentParser,
) -> int:
"""Run the HTTP transport. Refuses to start with nothing that can connect.
The empty-table check is the one piece of policy in this function, and it
is here rather than in :mod:`hhstream_mcp.http` because it is a *launch*
decision: a server bound to a port that every request will 401 is almost
always a missing `token mint`, and discovering that from the far side of an
HTTP client is far more work than reading one line on stderr. `--allow-no-
tokens` is the escape hatch for the one case where it is deliberate --
provisioning the unit before its first token exists.
"""
# Imported here: `starlette` and `uvicorn` are only needed by this path,
# and the stdio transport must not pay for them or fail on them.
from . import http as http_transport
store = authorizations.Store.at(getattr(args, "state_dir", None))
try:
grants = store.load()
except authorizations.StoreError as exc:
print(f"error: {exc}", file=sys.stderr)
return tokencli.EXIT_STORE
if not grants and not args.allow_no_tokens:
parser.error(
f"no tokens in {store.path}: every request would be refused. Mint "
"one with `hhstream-mcp token mint`, or pass --allow-no-tokens to "
"start anyway."
)
return tokencli.EXIT_USAGE # pragma: no cover -- parser.error exits
# The Host allowlist, which cannot be left empty: the SDK's DNS-rebinding
# check is allowlist-only with no implicit localhost, so an empty list
# refuses every request with a 421 (see `http.default_allowed_hosts`).
hosts = list(args.allowed_host) or http_transport.default_allowed_hosts(
args.address, args.port
)
if not hosts:
parser.error(
f"--address {args.address} binds every interface, so the Host "
"header a client will send cannot be derived and DNS-rebinding "
"protection has nothing to check against. Name the hostnames with "
"--allowed-host (repeatable), or bind one address."
)
return tokencli.EXIT_USAGE # pragma: no cover -- parser.error exits
log.info(
"serving %d tools and %d resources over http on %s:%d%s, %d token(s), "
"hosts %s",
len(tools.selected(toolsets)),
len(resources.ENTRIES),
args.address,
args.port,
args.path,
len(grants),
",".join(hosts),
)
http_transport.serve(
Runner(),
store,
address=args.address,
port=args.port,
path=args.path,
toolsets=toolsets,
allowed_hosts=hosts,
allowed_origins=args.allowed_origin,
stateless=args.stateless,
json_response=args.json_response,
)
return 0
if __name__ == "__main__": # pragma: no cover
raise SystemExit(main())
+270
View File
@@ -0,0 +1,270 @@
"""``hhstream-mcp token`` -- mint, list and revoke the HTTP transport's tokens.
Standard library only, like :mod:`hhstream_mcp.authorizations` beneath it: the
verbs never open a socket, never talk to the broker and never import the SDK.
**Nothing here may touch the stdio transport.** :func:`hhstream_mcp.server.main`
calls these and returns before ``claim_transport`` runs, which is the same
structural position ``--list-tools`` and ``--list-resources`` already occupy.
It matters more here than there: ``mint`` prints a secret on stdout, and stdout
is the JSON-RPC stream once a server is serving.
Exit codes
----------
These verbs are **outside** the frozen table in :mod:`hhstreamd.protocol`,
because nothing they do reaches the broker and an agent never calls them. Two
of the codes are chosen by analogy with it anyway, so that a reader who knows
the broker's table is not surprised:
==== ===============================================================
0 done
2 usage (argparse's own)
3 no token with that id -- the analogue of ``NO_SUCH_SESSION``
4 the store itself is unusable -- the analogue of ``BAD_STATE``
==== ===============================================================
"""
from __future__ import annotations
import argparse
import sys
from typing import Any
from hhstream_session.format import DASH, dump_json, render_table
from hhstreamd.clock import parse_duration
from . import tools
from .authorizations import Grant, NoSuchGrant, Store, StoreError
EXIT_OK = 0
EXIT_USAGE = 2
EXIT_NO_SUCH_TOKEN = 3
EXIT_STORE = 4
#: What `token list` shows. There is no column for the token, and there never
#: can be: the store holds a digest.
HEADERS = ["ID", "LABEL", "CREATED", "EXPIRES", "LAST USED", "TOOLSETS"]
def add_parser(subparsers: Any) -> argparse.ArgumentParser:
"""Attach the ``token`` command to a subparser set."""
token = subparsers.add_parser(
"token",
help="mint, list and revoke the tokens the --http transport accepts",
description=(
"Manage the bearer tokens `hhstream-mcp serve --http` accepts. "
"These do nothing for the stdio transport, which is unauthenticated "
"on purpose: its gate is the filesystem, since the client spawns "
"the server and already needs the broker's socket."
),
)
token.add_argument(
"--state-dir",
metavar="DIR",
help="where the authorization table lives. Defaults to "
"$HHSTREAM_MCP_STATE_DIR, then /var/lib/hhstream-mcp. This is NOT the "
"broker's sessions.stateDir: that directory is 0700 owned by the "
"broker's user and holds its audit trail.",
)
verbs = token.add_subparsers(dest="verb", metavar="VERB")
mint = verbs.add_parser(
"mint",
help="create a token and print it once",
description=(
"Print a new token on stdout. It is shown HERE AND NOWHERE ELSE: "
"the table on disk keeps a SHA-256 digest, so a token that was not "
"written down cannot be recovered, only revoked and replaced."
),
)
mint.add_argument(
"--label",
default="",
help="a note for the operator reading `token list` later, e.g. a client "
"or a host name. Never sent anywhere.",
)
mint.add_argument(
"--expires-in",
metavar="DURATION",
help="how long the token is valid: 30s, 5m, 3h, 90d, or a bare number "
"of seconds. Omitted, it does not expire.",
)
mint.add_argument(
"--toolsets",
metavar="NAMES",
help="comma-separated families this token may call: "
+ ", ".join(tools.TOOLSETS)
+ ". Omitted, it may call every tool. Unlike the server's own "
"--toolsets this is per CALLER, which is the only place the "
"distinction can live once one process serves several of them.",
)
listing = verbs.add_parser(
"list",
help="show every token's id, label and age",
description="Show the table. The tokens themselves are not in it.",
)
listing.add_argument("--json", action="store_true", help="the records, as JSON")
revoke = verbs.add_parser(
"revoke",
help="remove a token by id",
description=(
"Remove one grant, by the id `token list` shows. By id and not by "
"token, so ending a grant never means pasting a live secret onto a "
"command line that lands in a shell history."
),
)
revoke.add_argument("id", nargs="?", help="the token id, as `token list` shows it")
revoke.add_argument(
"--all", action="store_true", help="remove every token instead of one"
)
verbs.add_parser(
"prune",
help="drop expired tokens from the table",
description=(
"Housekeeping only. An expired token is refused whether or not it "
"is still on disk, so this changes what `token list` shows and "
"nothing about who can connect."
),
)
return token
def row_for(grant: Grant) -> list[str]:
return [
grant.id,
grant.label or DASH,
grant.created or DASH,
grant.expires_iso() or "never",
grant.last_used or "never",
",".join(grant.toolsets) if grant.toolsets else "all",
]
def run(args: argparse.Namespace, parser: argparse.ArgumentParser) -> int:
"""Dispatch one ``token`` verb. ``parser`` is the ``token`` parser itself."""
verb = getattr(args, "verb", None)
if verb is None:
parser.print_help()
return EXIT_USAGE
store = Store.at(getattr(args, "state_dir", None))
try:
if verb == "mint":
return _mint(args, store, parser)
if verb == "list":
return _list(args, store)
if verb == "revoke":
return _revoke(args, store, parser)
if verb == "prune":
return _prune(store)
except NoSuchGrant as exc:
print(f"error: {exc}", file=sys.stderr)
return EXIT_NO_SUCH_TOKEN
except StoreError as exc:
print(f"error: {exc}", file=sys.stderr)
return EXIT_STORE
parser.error(f"unknown verb: {verb}") # pragma: no cover -- argparse refuses first
return EXIT_USAGE # pragma: no cover
def _mint(args: argparse.Namespace, store: Store, parser: argparse.ArgumentParser) -> int:
ttl: float | None = None
if args.expires_in is not None:
try:
ttl = parse_duration(args.expires_in)
except ValueError as exc:
parser.error(f"--expires-in: {exc}")
return EXIT_USAGE # pragma: no cover -- parser.error exits
if ttl == 0:
parser.error("--expires-in: 0 would mint a token that is already expired")
chosen: list[str] | None = None
if args.toolsets is not None:
names = [part.strip() for part in args.toolsets.split(",") if part.strip()]
if not names:
parser.error(
"--toolsets was given no names. Omit it for every tool, or name "
f"some of: {', '.join(tools.TOOLSETS)}"
)
unknown = [name for name in names if name not in tools.TOOLSETS]
if unknown:
parser.error(
f"--toolsets: no such toolset: {', '.join(repr(u) for u in unknown)}. "
f"Known: {', '.join(tools.TOOLSETS)}"
)
chosen = names
grant, token = store.mint(label=args.label, ttl=ttl, toolsets=chosen)
# The token on stdout alone, so `... token mint | pbcopy` and friends get
# exactly it; everything a human reads goes to stderr beside it.
print(token)
print(
f"minted {grant.id}"
+ (f" ({grant.label})" if grant.label else "")
+ f", expires {grant.expires_iso() or 'never'}"
+ f", toolsets {','.join(grant.toolsets) if grant.toolsets else 'all'}."
"\nThis is the only time the token is shown. Store it in the client's "
"Authorization header; the table on disk keeps only its digest.",
file=sys.stderr,
)
return EXIT_OK
def _list(args: argparse.Namespace, store: Store) -> int:
grants = store.load()
if args.json:
dump_json([grant.to_dict() for grant in grants])
return EXIT_OK
if not grants:
print(
f"no tokens in {store.path}. `hhstream-mcp token mint` makes one; "
"until then `serve --http` refuses every request.",
file=sys.stderr,
)
return EXIT_OK
print(render_table([row_for(grant) for grant in grants], HEADERS))
return EXIT_OK
def _revoke(args: argparse.Namespace, store: Store, parser: argparse.ArgumentParser) -> int:
if args.all and args.id:
parser.error("give an id or --all, not both")
if args.all:
gone = store.revoke_all()
print(f"revoked {len(gone)} token(s).", file=sys.stderr)
return EXIT_OK
if not args.id:
parser.error("which token? give an id, or --all for every one")
return EXIT_USAGE # pragma: no cover -- parser.error exits
grant = store.revoke(args.id)
print(
f"revoked {grant.id}" + (f" ({grant.label})" if grant.label else "") + ".",
file=sys.stderr,
)
return EXIT_OK
def _prune(store: Store) -> int:
dead = store.prune()
print(
f"dropped {len(dead)} expired token(s)."
if dead
else "nothing expired; the table is unchanged.",
file=sys.stderr,
)
return EXIT_OK
__all__ = [
"EXIT_NO_SUCH_TOKEN",
"EXIT_OK",
"EXIT_STORE",
"EXIT_USAGE",
"HEADERS",
"add_parser",
"row_for",
"run",
]
+483
View File
@@ -0,0 +1,483 @@
"""The token store, and the token verbs on top of it.
Standard library only, like the module under test: no SDK, no socket, no
Starlette. The HTTP transport that consumes these tokens is
``test_mcp_http.py``.
Two properties get most of the attention here, because both are the kind that
a refactor breaks silently:
* **the file never holds a token.** A store that quietly started writing the
secret would pass every functional test in this file, so there is an explicit
one that reads the bytes on disk and looks for it.
* **a lost update is impossible.** Two processes write this file, and the
atomic replace alone does not prevent one from overwriting a table the other
just added to. The concurrency test does the interleaving by hand.
"""
from __future__ import annotations
import json
import os
import threading
from pathlib import Path
import pytest
from hhstreamd.clock import ManualClock
from hhstream_mcp import authorizations as auth
from hhstream_mcp import server, tokencli
@pytest.fixture
def store(tmp_path):
return auth.Store.at(tmp_path / "state")
# ── the token itself ───────────────────────────────────────────────────────
def test_mint_token_is_prefixed_and_long():
token = auth.mint_token()
assert token.startswith(auth.TOKEN_PREFIX)
# The prefix is for recognising a leak on sight; the rest carries the
# entropy. 32 bytes base64url is 43 characters.
assert len(token) - len(auth.TOKEN_PREFIX) >= 43
def test_mint_token_never_repeats():
assert len({auth.mint_token() for _ in range(200)}) == 200
def test_hash_is_stable_and_not_the_token():
token = auth.mint_token()
digest = auth.hash_token(token)
assert digest == auth.hash_token(token)
assert len(digest) == 64
assert token not in digest
# ── the table ──────────────────────────────────────────────────────────────
def test_missing_file_is_an_empty_table(store):
assert store.load() == []
assert not store.path.exists()
def test_mint_then_load_round_trips_every_field(store):
grant, token = store.mint(label="laptop", ttl=3600, toolsets=["frames", "pointer"])
(loaded,) = store.load()
assert loaded.id == grant.id
assert loaded.label == "laptop"
assert loaded.toolsets == ("frames", "pointer")
assert loaded.expires_at == pytest.approx(grant.expires_at)
assert loaded.hash == auth.hash_token(token)
def test_the_file_does_not_contain_the_token(store):
_, token = store.mint(label="secret-bearing")
raw = store.path.read_bytes()
assert token.encode() not in raw
# Nor any usable prefix of it: a truncated write would be just as bad.
assert token[len(auth.TOKEN_PREFIX) : len(auth.TOKEN_PREFIX) + 20].encode() not in raw
def test_the_file_is_private_and_so_is_its_directory(store):
store.mint()
assert os.stat(store.path).st_mode & 0o777 == 0o600
assert os.stat(store.state_dir).st_mode & 0o777 == 0o700
def test_the_file_is_json_with_a_version(store):
store.mint()
data = json.loads(store.path.read_text())
assert data["version"] == auth.FORMAT_VERSION
assert len(data["grants"]) == 1
def test_ids_are_unique_across_many_mints(store):
for _ in range(50):
store.mint()
ids = [g.id for g in store.load()]
assert len(set(ids)) == 50
# ── verify ─────────────────────────────────────────────────────────────────
def test_verify_accepts_the_minted_token(store):
grant, token = store.mint(label="good")
found = store.verify(token)
assert found is not None
assert found.id == grant.id
@pytest.mark.parametrize(
"bad",
[
"",
"hhsmcp_not-a-real-token",
"completely wrong",
auth.TOKEN_PREFIX,
],
)
def test_verify_refuses_anything_else(store, bad):
store.mint()
assert store.verify(bad) is None
def test_verify_refuses_a_revoked_token(store):
grant, token = store.mint()
store.revoke(grant.id)
assert store.verify(token) is None
def test_verify_refuses_an_expired_token(store):
clock = ManualClock()
store.clock = clock
_, token = store.mint(ttl=60)
assert store.verify(token) is not None
clock.advance(61)
assert store.verify(token) is None
def test_verify_refuses_everything_when_the_table_is_corrupt(store):
_, token = store.mint()
store.path.write_text("{ not json")
# Deny, never allow: a store that cannot be read is not a store that lets
# everyone in.
assert store.verify(token) is None
def test_verify_refuses_a_table_from_a_future_version(store):
_, token = store.mint()
data = json.loads(store.path.read_text())
data["version"] = auth.FORMAT_VERSION + 1
store.path.write_text(json.dumps(data))
assert store.verify(token) is None
with pytest.raises(auth.StoreError, match="format version"):
store.load()
def test_one_token_does_not_verify_as_another(store):
_, first = store.mint(label="a")
second_grant, second = store.mint(label="b")
assert store.verify(first).id != second_grant.id
assert store.verify(second).id == second_grant.id
# ── last_used ──────────────────────────────────────────────────────────────
def test_verify_stamps_last_used(store):
grant, token = store.mint()
assert grant.last_used is None
store.verify(token)
(loaded,) = store.load()
assert loaded.last_used is not None
def test_the_stamp_is_throttled(store):
clock = ManualClock()
store.clock = clock
_, token = store.mint()
store.verify(token)
first = store.load()[0].last_used
clock.advance(auth.LAST_USED_FLUSH_S / 2)
store.verify(token)
assert store.load()[0].last_used == first
clock.advance(auth.LAST_USED_FLUSH_S)
store.verify(token)
assert store.load()[0].last_used != first
def test_stamp_does_not_resurrect_a_token_revoked_underneath_it(store):
grant, token = store.mint()
# The grant object the caller is holding, after it has been removed: this
# is the real interleaving, where a `verify` and a `revoke` race.
store.revoke(grant.id)
store._stamp(grant, store.clock.wall())
assert store.load() == []
def test_verify_can_be_asked_not_to_stamp(store):
_, token = store.mint()
assert store.verify(token, stamp=False) is not None
assert store.load()[0].last_used is None
# ── revoke and prune ───────────────────────────────────────────────────────
def test_revoke_removes_only_the_named_grant(store):
first, _ = store.mint(label="a")
second, _ = store.mint(label="b")
store.revoke(first.id)
assert [g.id for g in store.load()] == [second.id]
def test_revoke_an_unknown_id_raises(store):
store.mint()
with pytest.raises(auth.NoSuchGrant, match="no token with id"):
store.revoke("zzzzzz")
assert len(store.load()) == 1
def test_revoke_all_empties_the_table(store):
for _ in range(3):
store.mint()
gone = store.revoke_all()
assert len(gone) == 3
assert store.load() == []
def test_prune_drops_only_what_expired(store):
clock = ManualClock()
store.clock = clock
store.mint(label="short", ttl=10)
keeper, _ = store.mint(label="forever")
clock.advance(11)
dropped = store.prune()
assert [g.label for g in dropped] == ["short"]
assert [g.id for g in store.load()] == [keeper.id]
def test_prune_is_housekeeping_not_enforcement(store):
"""An expired token is refused whether or not `prune` has run."""
clock = ManualClock()
store.clock = clock
_, token = store.mint(ttl=10)
clock.advance(11)
assert store.verify(token) is None
assert len(store.load()) == 1 # still on disk, and still refused
# ── the lock ───────────────────────────────────────────────────────────────
def test_a_concurrent_mint_is_not_lost(tmp_path):
"""Thirty threads minting into one table; every token must survive.
The atomic replace stops a reader seeing half a file. It does **not** stop
two writers from each loading the same table and the second replacing the
first's addition -- which is why the store takes a flock, and why this test
would fail without it.
"""
store = auth.Store.at(tmp_path / "state")
tokens: list[str] = []
lock = threading.Lock()
errors: list[BaseException] = []
def mint() -> None:
try:
_, token = store.mint()
with lock:
tokens.append(token)
except BaseException as exc: # recorded, asserted below
with lock:
errors.append(exc)
threads = [threading.Thread(target=mint) for _ in range(30)]
for thread in threads:
thread.start()
for thread in threads:
thread.join(30)
assert errors == []
assert len(store.load()) == 30
# And every one of them still works, which is the property that matters.
for token in tokens:
assert store.verify(token, stamp=False) is not None
def test_separate_store_objects_see_each_others_writes(tmp_path):
"""Two `Store` instances on one path, as a unit and a shell would be."""
root = tmp_path / "state"
unit = auth.Store.at(root)
operator = auth.Store.at(root)
_, token = operator.mint(label="minted by the operator")
assert unit.verify(token) is not None
# ── where the store lives ──────────────────────────────────────────────────
def test_the_default_is_not_the_brokers_state_dir():
"""The broker's state dir is 0700 and holds its audit trail; stay out."""
assert auth.DEFAULT_STATE_DIR == "/var/lib/hhstream-mcp"
assert not auth.DEFAULT_STATE_DIR.startswith("/var/lib/hhstream/")
def test_the_environment_names_the_state_dir(tmp_path, monkeypatch):
monkeypatch.setenv(auth.ENV_STATE_DIR, str(tmp_path / "from-env"))
assert auth.Store.at().state_dir == tmp_path / "from-env"
def test_an_explicit_dir_beats_the_environment(tmp_path, monkeypatch):
monkeypatch.setenv(auth.ENV_STATE_DIR, str(tmp_path / "from-env"))
assert auth.Store.at(tmp_path / "explicit").state_dir == tmp_path / "explicit"
# ── the CLI verbs ──────────────────────────────────────────────────────────
@pytest.fixture
def state_dir(tmp_path, monkeypatch):
root = tmp_path / "state"
monkeypatch.setenv(auth.ENV_STATE_DIR, str(root))
return root
def test_mint_prints_the_token_on_stdout_and_nothing_else(state_dir, capsys):
assert server.main(["token", "mint"]) == tokencli.EXIT_OK
captured = capsys.readouterr()
token = captured.out.strip()
# Exactly the token on stdout, so `token mint | ...` is usable.
assert token.startswith(auth.TOKEN_PREFIX)
assert "\n" not in token
# And the prose for the human is on stderr, beside it.
assert "only time the token is shown" in captured.err
assert auth.Store.at(state_dir).verify(token) is not None
def test_mint_accepts_a_label_an_expiry_and_toolsets(state_dir, capsys):
rc = server.main(
["token", "mint", "--label", "laptop", "--expires-in", "30d",
"--toolsets", "frames,pointer"]
)
assert rc == tokencli.EXIT_OK
capsys.readouterr()
(grant,) = auth.Store.at(state_dir).load()
assert grant.label == "laptop"
assert grant.toolsets == ("frames", "pointer")
assert grant.expires_at is not None
@pytest.mark.parametrize(
"argv",
[
["token", "mint", "--expires-in", "not-a-duration"],
["token", "mint", "--expires-in", "0"],
["token", "mint", "--toolsets", "nonesuch"],
["token", "mint", "--toolsets", ","],
],
)
def test_mint_refuses_a_bad_argument(state_dir, argv):
with pytest.raises(SystemExit) as exit_info:
server.main(argv)
assert exit_info.value.code == tokencli.EXIT_USAGE
assert auth.Store.at(state_dir).load() == []
def test_list_shows_the_ids_but_never_a_token(state_dir, capsys):
server.main(["token", "mint", "--label", "one"])
token = capsys.readouterr().out.strip()
assert server.main(["token", "list"]) == tokencli.EXIT_OK
out = capsys.readouterr().out
assert "one" in out
assert auth.Store.at(state_dir).load()[0].id in out
assert token not in out
def test_list_json_is_the_records(state_dir, capsys):
server.main(["token", "mint", "--label", "one"])
capsys.readouterr()
assert server.main(["token", "list", "--json"]) == tokencli.EXIT_OK
records = json.loads(capsys.readouterr().out)
assert [r["label"] for r in records] == ["one"]
assert "hash" in records[0]
def test_list_on_an_empty_table_says_so_and_succeeds(state_dir, capsys):
assert server.main(["token", "list"]) == tokencli.EXIT_OK
assert "no tokens" in capsys.readouterr().err
def test_revoke_by_id(state_dir, capsys):
server.main(["token", "mint"])
capsys.readouterr()
grant_id = auth.Store.at(state_dir).load()[0].id
assert server.main(["token", "revoke", grant_id]) == tokencli.EXIT_OK
assert auth.Store.at(state_dir).load() == []
def test_revoke_an_unknown_id_is_exit_3(state_dir, capsys):
assert server.main(["token", "revoke", "zzzzzz"]) == tokencli.EXIT_NO_SUCH_TOKEN
assert "no token with id" in capsys.readouterr().err
def test_revoke_all(state_dir, capsys):
for _ in range(3):
server.main(["token", "mint"])
capsys.readouterr()
assert server.main(["token", "revoke", "--all"]) == tokencli.EXIT_OK
assert auth.Store.at(state_dir).load() == []
def test_revoke_needs_an_id_or_all(state_dir):
with pytest.raises(SystemExit) as exit_info:
server.main(["token", "revoke"])
assert exit_info.value.code == tokencli.EXIT_USAGE
def test_revoke_refuses_an_id_and_all_together(state_dir):
with pytest.raises(SystemExit) as exit_info:
server.main(["token", "revoke", "abc123", "--all"])
assert exit_info.value.code == tokencli.EXIT_USAGE
def test_prune_through_the_cli(state_dir, capsys):
assert server.main(["token", "prune"]) == tokencli.EXIT_OK
assert "nothing expired" in capsys.readouterr().err
def test_token_with_no_verb_prints_help(state_dir, capsys):
assert server.main(["token"]) == tokencli.EXIT_USAGE
assert "mint" in capsys.readouterr().out
def test_an_unreadable_table_is_exit_4(state_dir, capsys):
state_dir.mkdir(parents=True)
(state_dir / auth.STORE_NAME).write_text("{ not json")
assert server.main(["token", "list"]) == tokencli.EXIT_STORE
assert "not valid JSON" in capsys.readouterr().err
def test_state_dir_flag_overrides_the_environment(state_dir, tmp_path, capsys):
elsewhere = tmp_path / "elsewhere"
assert server.main(["token", "--state-dir", str(elsewhere), "mint"]) == 0
capsys.readouterr()
assert len(auth.Store.at(elsewhere).load()) == 1
assert auth.Store.at(state_dir).load() == []
def test_the_token_verbs_never_claim_the_transport(state_dir, monkeypatch, capsys):
"""`mint` prints a secret on stdout, which on stdio *is* the transport.
So the verb has to return before anything moves file descriptor 1. A fake
`claim_transport` that fails the test if it is called is the only way to
assert that from here.
"""
def forbidden():
raise AssertionError("a token verb claimed the stdio transport")
monkeypatch.setattr(server, "claim_transport", forbidden)
assert server.main(["token", "mint"]) == tokencli.EXIT_OK
assert server.main(["token", "list"]) == tokencli.EXIT_OK
capsys.readouterr()
def test_row_for_renders_the_honest_gaps():
grant = auth.Grant(id="abc123", hash="x" * 64)
row = tokencli.row_for(grant)
assert row[0] == "abc123"
assert row[3] == "never" # expires
assert row[4] == "never" # last used
assert row[5] == "all" # toolsets
assert len(row) == len(tokencli.HEADERS)
+506
View File
@@ -0,0 +1,506 @@
"""The HTTP transport: every way a request is refused, and the one way it is not.
This file imports Starlette as well as the SDK, **unconditionally**, for the
reason ``test_mcp_server.py`` imports the SDK that way: the thing under test is
the wiring, only a real ASGI stack can judge it, and a file that skipped when
its dependency was missing would be green in precisely the build where that
mattered. The store itself is
standard library and tested without any of this
(``test_mcp_authorizations.py``).
The app is driven through ``starlette.testclient``, which speaks to it over
ASGI with no socket. That is enough for everything here -- a status code, a
header, an empty body, which tools a session is shown -- and it is
deliberately *not* enough for "the unit comes up and answers on a port", which
is ``checks/vm-mcp.nix``' job.
``Host: testserver`` is what the test client sends, so every app built here
names it in ``allowed_hosts``. That is not a workaround: DNS-rebinding
protection is on by default and an app that did not name its clients' host
would 421 them, which is the behaviour ``test_rebinding_protection_*`` asserts.
"""
from __future__ import annotations
import json
import pathlib
import pytest
from hhstream_mcp import authorizations as auth
from hhstream_mcp import http as http_transport
from hhstream_mcp import resources, tools
from hhstream_mcp.invoke import Runner
# Imported unconditionally, and not through `importorskip`, for the reason
# `test_mcp_server.py` imports the SDK unconditionally: a file that skipped
# when its dependency was absent would be green in exactly the build where
# that mattered. `checks/unit.nix` declares `starlette` and `httpx` so this
# cannot become a skip, and proves them importable before pytest runs.
from starlette.testclient import TestClient
#: The test client's Host, plus what a loopback deployment derives.
TEST_HOSTS = ["testserver", *http_transport.default_allowed_hosts("127.0.0.1", 9989)]
HEADERS = {
"Accept": "application/json, text/event-stream",
"Content-Type": "application/json",
}
INITIALIZE = {
"jsonrpc": "2.0",
"id": 1,
"method": "initialize",
"params": {
"protocolVersion": "2025-06-18",
"capabilities": {},
"clientInfo": {"name": "test", "version": "0"},
},
}
@pytest.fixture
def store(tmp_path):
return auth.Store.at(tmp_path / "state")
def make_app(store, **kwargs):
"""The app, with a Runner that never reaches a CLI.
``entry`` returns 0 without doing anything: nothing in this file calls a
tool for its effect, and a Runner that could reach `hhstream.cli` would
make these tests depend on a broker socket.
"""
kwargs.setdefault("allowed_hosts", TEST_HOSTS)
kwargs.setdefault("json_response", True)
return http_transport.build_app(Runner(entry=lambda argv: 0), store, **kwargs)
def post(client, body, token=None, headers=None, path="/mcp"):
"""One POST, and **never** following a redirect.
`follow_redirects=False` is load-bearing, not tidiness. The test client
follows redirects by default, which is how a `Mount`-shaped 307 on the
exact path `/mcp` passed this whole file once while failing a real client:
a redirected POST has to be re-sent with its body, and `urllib` will not do
that at all. See `test_the_exact_path_does_not_redirect`.
"""
sent = dict(HEADERS)
if token is not None:
sent["Authorization"] = f"Bearer {token}"
sent.update(headers or {})
return client.post(
path, content=json.dumps(body), headers=sent, follow_redirects=False
)
def session(client, token):
"""Initialize and return the headers that carry the session on."""
response = post(client, INITIALIZE, token=token)
assert response.status_code == 200, response.content
extra = {}
sid = response.headers.get("mcp-session-id")
if sid:
extra["mcp-session-id"] = sid
post(client, {"jsonrpc": "2.0", "method": "notifications/initialized"},
token=token, headers=extra)
return extra
def call(client, token, method, params=None, headers=None, request_id=2):
body = {"jsonrpc": "2.0", "id": request_id, "method": method}
if params is not None:
body["params"] = params
response = post(client, body, token=token, headers=headers)
assert response.status_code == 200, response.content
return json.loads(response.content)
# ── parsing the header ─────────────────────────────────────────────────────
@pytest.mark.parametrize(
("header", "expected"),
[
("Bearer abc", "abc"),
("bearer abc", "abc"), # RFC 9110: the scheme is case-insensitive
("BEARER abc", "abc"),
("Bearer abc ", "abc"),
("", None),
("abc", None), # no scheme
("Basic abc", None), # wrong scheme
("Bearer", None), # no token
("Bearer ", None),
],
)
def test_bearer_token_parsing(header, expected):
assert http_transport.bearer_token(header) == expected
# ── the refusals ───────────────────────────────────────────────────────────
def test_no_header_is_401_with_an_empty_body(store):
store.mint()
with TestClient(make_app(store)) as client:
response = post(client, INITIALIZE)
assert response.status_code == 401
assert response.content == b""
assert response.headers["www-authenticate"].startswith("Bearer ")
assert response.headers["cache-control"] == "no-store"
def test_a_wrong_token_is_401_and_is_not_echoed(store):
store.mint()
wrong = "hhsmcp_definitely-not-a-real-token"
with TestClient(make_app(store)) as client:
response = post(client, INITIALIZE, token=wrong)
assert response.status_code == 401
assert response.content == b""
# Not in the body, and not in any header either.
assert wrong not in "".join(f"{k}{v}" for k, v in response.headers.items())
def test_a_revoked_token_is_401(store):
grant, token = store.mint()
store.revoke(grant.id)
with TestClient(make_app(store)) as client:
assert post(client, INITIALIZE, token=token).status_code == 401
def test_an_expired_token_is_401(store):
from hhstreamd.clock import ManualClock
clock = ManualClock()
store.clock = clock
_, token = store.mint(ttl=60)
clock.advance(61)
with TestClient(make_app(store)) as client:
assert post(client, INITIALIZE, token=token).status_code == 401
def test_the_wrong_scheme_is_401(store):
_, token = store.mint()
with TestClient(make_app(store)) as client:
response = post(client, INITIALIZE, headers={"Authorization": f"Basic {token}"})
assert response.status_code == 401
def test_an_empty_table_refuses_everything(store):
with TestClient(make_app(store)) as client:
assert post(client, INITIALIZE, token="hhsmcp_anything").status_code == 401
def test_the_refusal_does_not_say_which_kind_it_was(store):
"""Unknown, revoked and expired are one answer, byte for byte.
Telling them apart would tell someone who guessed a real id that they had.
"""
grant, revoked = store.mint()
store.revoke(grant.id)
_, live = store.mint()
responses = []
with TestClient(make_app(store)) as client:
for token in ("hhsmcp_unknown", revoked, live[:-1] + "x"):
response = post(client, INITIALIZE, token=token)
responses.append((response.status_code, response.content,
response.headers.get("www-authenticate")))
assert len(set(responses)) == 1
@pytest.mark.parametrize("path", ["/mcp", "/mcp/"])
def test_the_exact_path_does_not_redirect(store, path):
"""Both spellings are the endpoint, and neither is a redirect to the other.
The regression this exists for: `Starlette(routes=[Mount("/mcp", ...)])`
serves `/mcp/` and answers `/mcp` with a 307 — and every MCP client
configuration spells the URL without the trailing slash. The redirect is
invisible to a client that follows it on a POST and fatal to one that does
not, which is why the endpoint matches the path itself instead.
"""
_, token = store.mint()
with TestClient(make_app(store)) as client:
response = post(client, INITIALIZE, token=token, path=path)
assert response.status_code == 200, (
f"{path} answered {response.status_code} "
f"(location: {response.headers.get('location')!r})"
)
def test_a_custom_path_is_honoured(store):
_, token = store.mint()
with TestClient(make_app(store, path="/agent/mcp")) as client:
assert post(client, INITIALIZE, token=token, path="/agent/mcp").status_code == 200
assert post(client, INITIALIZE, token=token, path="/mcp").status_code == 404
def test_an_unknown_path_leaks_nothing_without_a_token(store):
store.mint()
with TestClient(make_app(store)) as client:
# 401 and not 404: an unauthenticated caller does not learn what is
# mounted here, or that anything is.
assert client.get("/elsewhere").status_code == 401
def test_an_unknown_path_is_a_404_once_authenticated(store):
_, token = store.mint()
with TestClient(make_app(store)) as client:
response = client.get(
"/elsewhere",
headers={"Authorization": f"Bearer {token}"},
follow_redirects=False,
)
assert response.status_code == 404
def test_repeated_failures_are_rate_limited(store):
from hhstream_live.server import RateLimiter
store.mint()
app = make_app(store)
app.limiter = RateLimiter(limit=3)
with TestClient(app) as client:
codes = [post(client, INITIALIZE).status_code for _ in range(5)]
assert codes[:3] == [401, 401, 401]
assert codes[3:] == [429, 429]
def test_a_good_token_is_not_rate_limited(store):
from hhstream_live.server import RateLimiter
_, token = store.mint()
app = make_app(store)
app.limiter = RateLimiter(limit=2)
with TestClient(app) as client:
for _ in range(5):
assert post(client, INITIALIZE, token=token).status_code == 200
# ── the one way through ────────────────────────────────────────────────────
def test_a_good_token_completes_the_handshake(store):
_, token = store.mint()
with TestClient(make_app(store)) as client:
response = post(client, INITIALIZE, token=token)
assert response.status_code == 200
result = json.loads(response.content)["result"]
assert result["serverInfo"]["name"] == "hhstream"
assert "instructions" in result
def test_the_full_surface_is_offered_to_an_unrestricted_token(store):
_, token = store.mint()
with TestClient(make_app(store)) as client:
extra = session(client, token)
listed = call(client, token, "tools/list", headers=extra)["result"]["tools"]
offered = call(client, token, "resources/list", headers=extra,
request_id=3)["result"]["resources"]
assert len(listed) == len(tools.TOOLS)
# Counted against the server's own table, never a literal: a resource added
# by a later contract version must not have to be counted here as well. The
# sibling stdio check learned this the hard way.
assert len(offered) == len(resources.FIXED)
def test_using_a_token_stamps_it(store):
grant, token = store.mint()
with TestClient(make_app(store)) as client:
session(client, token)
assert store.load()[0].last_used is not None
assert store.load()[0].id == grant.id
# ── what a token's toolsets do ─────────────────────────────────────────────
def test_a_scoped_token_is_offered_only_its_toolsets(store):
_, token = store.mint(toolsets=["frames"])
with TestClient(make_app(store)) as client:
extra = session(client, token)
listed = call(client, token, "tools/list", headers=extra)["result"]["tools"]
assert {tool["name"] for tool in listed} == set(tools.TOOLSETS["frames"])
def test_a_scoped_token_cannot_call_outside_its_toolsets(store):
_, token = store.mint(toolsets=["frames"])
with TestClient(make_app(store)) as client:
extra = session(client, token)
result = call(
client, token, "tools/call",
{"name": "session_gc", "arguments": {}}, headers=extra,
)["result"]
assert result["isError"] is True
text = result["content"][0]["text"]
assert "session_gc" in text
# The message names both possible narrowings, because the agent cannot tell
# them apart and neither is its to change.
assert "--toolsets" in text
assert "authorization" in text
def test_a_token_cannot_widen_the_processs_own_toolsets(store):
"""The intersection, not the token's list.
A unit started with `--toolsets frames` must not be talked into serving
`session_start` by a token that names `sessions`.
"""
_, token = store.mint(toolsets=["frames", "sessions"])
with TestClient(make_app(store, toolsets=["frames"])) as client:
extra = session(client, token)
listed = call(client, token, "tools/list", headers=extra)["result"]["tools"]
assert {tool["name"] for tool in listed} == set(tools.TOOLSETS["frames"])
def test_two_tokens_on_one_server_see_different_surfaces(store):
"""The property the per-token narrowing exists for.
One process, two callers, two tool budgets. A launch-time flag cannot do
this, which is the whole argument for putting toolsets on the token.
"""
_, broad = store.mint(label="broad")
_, narrow = store.mint(label="narrow", toolsets=["frames"])
with TestClient(make_app(store)) as client:
broad_extra = session(client, broad)
narrow_extra = session(client, narrow)
broad_tools = call(client, broad, "tools/list",
headers=broad_extra)["result"]["tools"]
narrow_tools = call(client, narrow, "tools/list", headers=narrow_extra,
request_id=3)["result"]["tools"]
assert len(broad_tools) == len(tools.TOOLS)
assert {tool["name"] for tool in narrow_tools} == set(tools.TOOLSETS["frames"])
def test_resources_are_not_narrowed_by_a_token(store):
"""A stated limit, not an oversight.
`docs/CONTRACT-mcp.md` keeps resources out of `--toolsets` because they are
the host's context rather than the model's budget. A token is an
authorization rather than a budget, so this is a real gap and it is written
down in `docs/TODO/mcp-tokens-do-not-scope-resources.md` rather than
quietly implied by the tests.
"""
_, token = store.mint(toolsets=["frames"])
with TestClient(make_app(store)) as client:
extra = session(client, token)
listed = call(client, token, "resources/list", headers=extra)["result"]["resources"]
assert any(entry["uri"] == "hhstream://sessions" for entry in listed)
def test_a_token_naming_an_unknown_toolset_offers_nothing(store):
"""A grant from a future build: refuse everything rather than widen."""
grant, token = store.mint()
grant.toolsets = ("a-family-this-build-has-never-heard-of",)
store._write([grant])
with TestClient(make_app(store)) as client:
extra = session(client, token)
listed = call(client, token, "tools/list", headers=extra)["result"]["tools"]
assert listed == []
# ── DNS-rebinding protection ───────────────────────────────────────────────
def test_rebinding_protection_refuses_an_unnamed_host(store):
_, token = store.mint()
app = make_app(store, allowed_hosts=["somewhere.else"])
with TestClient(app) as client:
response = post(client, INITIALIZE, token=token)
# The token was good; the Host was not. 421 Misdirected Request.
assert response.status_code == 421
def test_the_token_is_checked_before_the_host(store):
"""Order matters: an unauthenticated caller learns nothing about the host
allowlist, which is itself a hint about how this server is deployed."""
store.mint()
app = make_app(store, allowed_hosts=["somewhere.else"])
with TestClient(app) as client:
assert post(client, INITIALIZE).status_code == 401
@pytest.mark.parametrize("address", ["127.0.0.1", "::1", "localhost"])
def test_loopback_derives_a_working_allowlist(address):
hosts = http_transport.default_allowed_hosts(address, 9989)
assert "127.0.0.1:9989" in hosts
assert "localhost:9989" in hosts
assert "[::1]:9989" in hosts
# The SDK's wildcard-port form, for a client that reached a proxy on
# another port and was forwarded here.
assert "127.0.0.1:*" in hosts
def test_a_named_address_derives_itself_and_not_loopback():
hosts = http_transport.default_allowed_hosts("10.0.0.5", 9989)
assert hosts == ["10.0.0.5:9989", "10.0.0.5:*", "10.0.0.5"]
@pytest.mark.parametrize("address", ["0.0.0.0", "::", "*"])
def test_a_wildcard_bind_derives_nothing(address):
"""Which is what makes the launch path refuse it.
Nothing can be guessed about the Host a client will send, and an empty
allowlist with protection on refuses every request. So the operator is made
to name the hostnames instead.
"""
assert http_transport.default_allowed_hosts(address, 9989) == []
# ── the grant on the request ───────────────────────────────────────────────
def test_current_grant_is_none_off_a_request():
"""The stdio case: no request context at all, and no restriction either."""
from mcp.server.lowlevel import Server
assert http_transport.current_grant(Server("probe")) is None
def test_offered_for_returns_everything_without_a_grant():
from mcp.server.lowlevel import Server
chooser = http_transport.offered_for(Server("probe"), tools.TOOLS)
assert chooser() == tools.TOOLS
# ── the dependencies this transport needs declared ─────────────────────────
#: `src/hhstream_mcp/pyproject.toml`, when the suite is run from a checkout.
#: Absent in the build sandbox, which copies only `tests/`.
MCP_PYPROJECT = (
pathlib.Path(__file__).resolve().parents[2] / "src" / "hhstream_mcp" / "pyproject.toml"
)
@pytest.mark.skipif(not MCP_PYPROJECT.is_file(), reason="not running from a checkout")
def test_the_distribution_declares_what_this_module_imports():
"""`starlette` and `uvicorn` by name, not by way of the SDK.
The same assertion `tests/pointer/test_vendored_protocol.py` makes about
`pywayland` and `xkbcommon`, and for the same kind of reason. The MCP SDK
propagates both of these today, so leaving them out of the metadata would
*work* — which is exactly what makes it worth pinning. `http.py` imports
`uvicorn` by name and the SDK's `TransportSecuritySettings` sits on
Starlette's `Request`, so an SDK release that moved its HTTP transport
behind an optional extra would break `serve --http` with an ImportError
naming somebody else's package.
"""
pyproject = MCP_PYPROJECT.read_text()
assert "starlette" in pyproject
assert "uvicorn" in pyproject
# And `anyio`, which was already declared on this argument before the HTTP
# transport existed — the precedent this follows.
assert "anyio" in pyproject
+123 -1
View File
@@ -520,7 +520,10 @@ def test_a_tool_outside_the_selection_cannot_be_called_by_name(session):
assert result.isError is True
text = result.content[0].text
assert "session_kill" in text
assert "without its toolset" in text
# Both narrowings are named, because on the HTTP transport the token can be
# the one that excluded it and the agent cannot tell from here which it was.
assert "--toolsets" in text
assert "authorization" in text
assert fake.calls == []
@@ -575,3 +578,122 @@ def test_content_for_a_text_only_result_has_no_image_block():
blocks = server.content_for(tools.Result(text="hello"))
assert [block.type for block in blocks] == ["text"]
assert isinstance(blocks[0], types.TextContent)
# ── the command line, now that it has subcommands ──────────────────────────
#
# `hhstream-mcp` grew `serve` and `token` beneath it, and the forms that
# existed before have to keep working unchanged: an MCP client's `command`
# entry is a configuration someone else wrote, and a flag that silently became
# a subcommand would break it with a usage error nobody can read from the far
# side of a pipe.
@pytest.mark.parametrize(
"argv",
[
[],
["--toolsets", "frames"],
["--toolsets=frames"],
["serve"],
["serve", "--toolsets", "frames"],
],
)
def test_the_pre_subcommand_forms_still_mean_serve_stdio(argv):
args = server.build_parser().parse_args(argv)
assert getattr(args, "command", None) in (None, "serve")
assert getattr(args, "http", False) is False
def test_a_toolsets_given_before_the_subcommand_survives_it():
"""argparse copies a subparser's namespace over the top-level one.
With an ordinary default on the subcommand's copy of `--toolsets`, that
copy would overwrite a value given *before* the subcommand with `None` --
so `hhstream-mcp --toolsets=frames serve` would silently serve all
twenty-one tools. `SUPPRESS` is what stops that, and this is the test that
notices if it is ever removed.
"""
args = server.build_parser().parse_args(["--toolsets=frames", "serve"])
assert args.toolsets == "frames"
def test_the_subcommand_wins_when_both_sides_name_it():
args = server.build_parser().parse_args(["--toolsets=frames", "serve", "--toolsets=keyboard"])
assert args.toolsets == "keyboard"
def test_list_tools_works_on_either_side_of_serve(capsys):
assert server.main(["--list-tools", "--toolsets", "keyboard"]) == 0
before = capsys.readouterr().out
assert server.main(["serve", "--toolsets", "keyboard", "--list-tools"]) == 0
assert capsys.readouterr().out == before
def test_list_resources_still_works(capsys):
assert server.main(["--list-resources"]) == 0
out = capsys.readouterr().out
assert "hhstream://status" in out
def test_http_is_off_unless_asked_for():
args = server.build_parser().parse_args(["serve"])
assert args.http is False
args = server.build_parser().parse_args(["serve", "--http"])
assert args.http is True
assert args.address == "127.0.0.1" # loopback, not a wildcard
assert args.port == 9989
def test_http_refuses_to_start_with_an_empty_table(tmp_path, monkeypatch):
"""A listener that 401s every request is almost always a forgotten mint."""
monkeypatch.setenv("HHSTREAM_MCP_STATE_DIR", str(tmp_path / "state"))
with pytest.raises(SystemExit) as exit_info:
server.main(["serve", "--http"])
assert exit_info.value.code == 2
def test_http_starts_with_an_empty_table_when_told_to(tmp_path, monkeypatch):
"""`--allow-no-tokens` is for provisioning the unit before its first mint.
Patched at `http.serve` rather than actually bound: what is under test is
that the launch path got past its own refusal, with the arguments it was
given.
"""
monkeypatch.setenv("HHSTREAM_MCP_STATE_DIR", str(tmp_path / "state"))
from hhstream_mcp import http as http_transport
seen: dict = {}
def fake_serve(runner, store, **kwargs):
seen.update(kwargs)
monkeypatch.setattr(http_transport, "serve", fake_serve)
assert server.main(["serve", "--http", "--allow-no-tokens"]) == 0
assert seen["address"] == "127.0.0.1"
# The derived allowlist, not an empty one: empty would 421 every request.
assert "127.0.0.1:9989" in seen["allowed_hosts"]
def test_http_refuses_a_wildcard_bind_with_no_allowed_host(tmp_path, monkeypatch):
monkeypatch.setenv("HHSTREAM_MCP_STATE_DIR", str(tmp_path / "state"))
with pytest.raises(SystemExit) as exit_info:
server.main(["serve", "--http", "--allow-no-tokens", "--address", "0.0.0.0"])
assert exit_info.value.code == 2
def test_http_accepts_a_wildcard_bind_that_names_its_hosts(tmp_path, monkeypatch):
monkeypatch.setenv("HHSTREAM_MCP_STATE_DIR", str(tmp_path / "state"))
from hhstream_mcp import http as http_transport
seen: dict = {}
monkeypatch.setattr(http_transport, "serve", lambda runner, store, **kw: seen.update(kw))
rc = server.main([
"serve", "--http", "--allow-no-tokens",
"--address", "0.0.0.0", "--allowed-host", "desk.lan:*",
])
assert rc == 0
assert seen["allowed_hosts"] == ["desk.lan:*"]