Files
skyg-linux-tracker/docs/workstreams/P5-consolidator/ws-c-rest.md
T
2026-10-20 16:35:11 -04:00

15 KiB

WS-C: REST surface

Part of Phase 5: Consolidator. Contracts: contracts.md sections 3 to 5.

Field Value
Status Merged
Owner WS-C agent (2026-10-19)
Depends on B (original plan). In practice: A0 for compiling, B only for the final end-to-end check; develop against FakeTransport + registry_with / static_registry
Blocks WS-D D1 (shares PeerRegistry::get / proxy.rs; D can stub it from A0 until C2 lands), F
Can start when A0 is merged (earlier in a branch against contracts.md)
Owned files src/api/peers.rs, src/peers/proxy.rs (the single proxy implementation: allowlist, header rules, error mapping calls), src/error.rs (only if D9 adds a variant), docs/api.md, src/api/behaviour_tests.rs (peer cases appended)
Do not touch src/api/mod.rs (A0 wired pub mod peers, the conditional merge and the 404/405 fallbacks; stay out), src/api/ping.rs (B), src/peers/{registry,health,client}.rs (B), src/peers/{types,error,transport,testing}.rs (contracts.md only), src/mcp/** (D), src/telemetry/** (E), src/state.rs (A0), nix/**, README.md (F)

Shared-file seam: src/api/mod.rs is edited only by A0. WS-C fills peers::routes() (returns the Router merged when state.peers is Some). The contract for what a route returns is contracts.md section 5.

Goal

Status and pass-through over REST under the existing auth.

Tasks

  • C1 GET /api/v1/peers and /{peer}: status JSON (shape in contracts section 5), sorted by name; unknown peer is NotFound.
  • C2 pass-through GET /api/v1/peers/{peer}/{*path}: allowlist of the peer's documented routes (systemd/..., journal, journal/follow, docker/...; ping and peers/... excluded), reject .., encoded slashes and non-allowlisted query keys; forward only the allowlisted query; never forward the caller's Authorization or cookies, use the peer token; add x-slinuxm-hop: 1 and reject requests that already carry it (no chaining); stream bodies through the size cap; SSE passes through bounded by limits.max_follow_secs and shutdown. Implemented in src/peers/proxy.rs as the body of PeerRegistry::get, which WS-D also calls; the route in peers.rs is a thin adapter. (Axum 0.8: routes use /{peer} and /{*path}.)
  • C3 response header x-slinuxm-peer: <name>; body unchanged so clients can reuse local parsers.
  • C4 (note, no new task from the original) error mapping goes through peer_error_to_app (contracts section 4). Whether to add an extra peer field to the REST error body is optional; if done it is a contracts.md change plus src/error.rs.

Inputs

  • Contracts sections 3 (registry API), 4 (error table), 5 (REST shapes).
  • Fakes: FakeTransport (records calls to assert hop header, no Authorization, filtered query), registry_with, static_registry, fake_state_with_peers.

Done when

Behaviour tests with the fake transport for every error mapping row in D9; 405 on non-GET; traversal and header-smuggling cases rejected; SSE ends on shutdown.

Handoff

  • PeerRegistry::get fully implemented (allowlist, hop, mapping) for WS-D; post in D's Log when C2 merges.
  • docs/api.md section for the peers endpoints for F4.
  • Verified shapes that WS-T asserts (x-slinuxm-peer, 405, 4xx on hop header, 503 naming the peer).

Log

  • 2026-10-19: Split from the single phase plan 2026-10-19. The proxy logic is placed in src/peers/proxy.rs (not api/peers.rs) so REST and MCP share one implementation, as the original plan required.
  • 2026-10-19 (A0): contract stubs landed (src/peers/{types,error,transport,testing,registry,proxy,health}.rs, src/api/peers.rs stub, AppState.peers, Config.peers, tests/peers_config.rs). PeerRegistry::get forwards through PeerTransport and proxy.rs already adds x-slinuxm-hop and traceparent; path/query allowlists, the hop-header rejection and the x-slinuxm-peer response header are still yours. api::peers::routes() is empty and merged only when state.peers.is_some(), so fill it without touching api/mod.rs.
  • 2026-10-19 (C1): GET /api/v1/peers and /peers/{peer} landed in src/api/peers.rs (sorted snapshot, one object for the single-peer route, peer <name>: unknown peer 404), with behaviour tests for shape, sorting, unknown/invalid names, the empty registry, route absence when peers are disabled and local routes staying available. docs/api.md gained the route table row and the status section.
  • 2026-10-19 (C2): the allowlist proxy is the body of PeerRegistry::get in src/peers/proxy.rs; src/api/peers.rs is a thin adapter that parses the raw (still percent-encoded) path, the raw query and the trace/accept headers, then renders PeerResponse. Allowed routes: systemd/units, systemd/units/{name}, systemd/timers, journal, journal/follow, docker/ps, docker/containers/{name}, docker/containers/{name}/logs; an /api/v1 prefix from the caller is stripped. .., encoded slashes, NUL/CRLF, a second encoding layer, empty segments and over-long paths/queries are rejected before any transport call; a backslash is only legal inside a unit name (see the review entry). Query keys are per route, single-valued and character-checked; anything else is peer <name>: unsupported query parameter. The caller's Authorization/Cookie are never copied (the request is rebuilt, the transport attaches the peer token), x-slinuxm-hop: 1 is added and any inbound hop header is 400 peer <name>: chained requests are not allowed. Buffered bodies over peers.max_response_bytes are 503 oversize; streams are cut at the cap, at limits.max_follow_secs or on the shared CancellationToken. SSE is selected by Accept or journal/follow / docker follow=1. A peer Content-Type that is not a valid header value becomes 500 malformed response instead of smuggling a second header.
  • 2026-10-19 (C3): x-slinuxm-peer: <name> is attached to every outcome of the pass-through route, including the consolidator's own 400/404/500/503 for that peer (a 405 is produced by the router fallback in api/mod.rs, which WS-C does not own, and carries no peer header; WS-T does not require one there).
  • 2026-10-19 (C4): mapping stays in peer_error_to_app (WS-A file, unchanged); REST adds no peer field to the error body, so the note in this file is closed as "not done, optional". A behaviour test drives every D9 row through the fake transport and asserts status, code, message, the peer name in the body, the peer header, and that no row is 401, leaks a token-file path or a URL.
  • 2026-10-19 Handoff: C1 to C4 are committed as four task commits on ws/C (9356596, a1ea58a, cf6b81c, c967ad9). cargo fmt --check, cargo clippy --all-targets -- -D warnings, cargo test (154 lib tests) and cargo tree -i openssl (empty) pass. WS-D can call PeerRegistry::get now: it takes /api/v1-prefixed or bare paths and returns PeerResponse or an already-mapped AppError. SSE callers set ProxyOpts.accept_sse and then hand the response to proxy::prepare_response(peer, resp, reg.config().max_response_bytes, deadline, shutdown) and stream PreparedResponse.body: that function (not the REST adapter) owns the 2xx-only status check, the content-type check, the buffered cap and the follow deadline/shutdown bound, so D must not add a second copy. See the review entry below. WS-F/WS-T: the x-slinuxm-peer header, 405, hop-header 400, 503-with-peer-name and query filtering behaviours the VM test asserts are covered by src/api/behaviour_tests.rs. Re-check against the real registry once B lands (real transport must map peer statuses to PeerError before C sees them; the fake returns statuses verbatim). No Nix VM test was run by this workstream.
  • 2026-10-19 (review round 1): fixes.
    • Blocking: the proxy allowlist no longer keeps a second, stricter copy of unit-name validation. systemd/units/{name} decodes the segment and calls sources::systemd::validate_unit_name, so escaped units (foo\x2dbar.mount, mnt-data%5Cx2dbackup.mount) and -.mount/-.slice work through the proxy as they do locally; %5c/\ is now rejected only where the route is not a unit name (an exact route or a docker container). Each segment is decoded on its own, and the forwarded path is re-encoded from the decoded segment (\ becomes %5C), because the transport builds the URL with Url::set_path and a literal backslash would be read as a separator there.
    • Size/stream bounds moved out of the REST adapter: proxy::prepare_response is the shared step for REST and MCP. It applies the buffered cap, the stream cap, the follow deadline and shutdown, and validates the content type.
    • A non-2xx PeerResponse is classified instead of relayed (404 -> NotFound, 401/403 -> Rejected, anything else -> Http(code)), so a transport bug cannot pass a redirect or the peer's error body through.
    • x-slinuxm-peer is attached only when the name parses and the registry knows it; an unknown name is the contract's 404 with no peer header, and the proxy is not called.
    • Proxied text/event-stream responses now carry Cache-Control: no-cache like the local Sse routes.
    • The unreachable fallbacks are gone: the routes are merged only with a registry, so the handler expects one, and the peer name is taken from PeerRegistry::status (one parse and one lookup) instead of being parsed again after a successful proxy.
    • Tests: allowlist and forward-form cases for escaped and --root units (unit and behaviour level), non-2xx mapping rows (404, 401, 403, 500, 302), unknown/malformed peer names with no header and no transport call, no-cache on a proxied follow, and the double-encoding/control-character rejections (%252f, %255c, %00, %0a, %0d) in both bad_paths and bad_queries. Each was checked to fail with the fix reverted (the unit-name, non-2xx status, encoding-rejection and no-cache tests were run against the old code); the unknown-peer header assertion replaces one that asserted the old labelled behaviour.
    • Disagreeing with / not doing:
      • Disabled peers are still proxied in this tree, but the gating belongs to PeerRegistry::get in src/peers/registry.rs, which WS-C does not own and WS-B's review already covers (its finding 2). A test here would have to assert behaviour C cannot change, so no C-side test is added; when B refuses disabled peers, the REST route inherits it unchanged because C only resolves the name via the registry.
      • The raw_peer_path None branch is kept: it is an Option check on the request URI, not an error-shape fallback, and it is cheap insurance if the route pattern changes.
      • 405 on a proxied path is still produced by the router fallback in api/mod.rs (not owned by WS-C), so it carries no x-slinuxm-peer; the contract's "every proxied response" is satisfied for responses that reach a peer handler, and WS-T does not assert the header on 405.
  • 2026-10-19 (review round 2): fixes.
    • Blocking: proxied journal query values now accept what the local route accepts on the unit side. query_value_ok(key, value) takes the key: for unit it calls sources::systemd::validate_unit_name, the helper the local systemd/units/{name} route already uses, so there is still one unit rule and no second validator. Escaped units (mnt-data\x2dbackup.mount, dev-disk-by\x2did.device) therefore work through journal and journal/follow as text, as %5C and as %5c; the value is forwarded decoded, which is what the transport re-encodes as one query pair. Other keys keep the generic \, / and .. rejection, and every key keeps the length bound, the control-character rule and the second-encoding-layer rejection (%252f, %255c, %2e%2e).
    • Tests: journal_unit_values_are_forwarded_like_the_local_route (raw, %5C, %5c, a second key, -.mount, user@100.service, and a decoded unit pair for the MCP path) and journal_unit_values_still_reject_unsafe_text in proxy.rs; behaviour-level proxy_forwards_journal_unit_values_the_local_route_accepts asserts the forwarded query values and that an unknown key or traversal text is still 400. proxy_forwards_unit_names_the_local_route_accepts gained a raw-backslash systemd path next to the encoded ones, so both routes cover raw and encoded input. With the unit branch reverted, all three journal tests fail on the old check (the behaviour one with the reported 400), so they pin the fix.
    • Non-blocking 2: the status table is now proxy::classify_status(u16) -> PeerError, one function with one test (status_classification_is_one_table). prepare_response calls it and keeps the 2xx-only guard as defence in depth. Handoff to WS-B: the real transport classifies a live response with crate::peers::proxy::classify_status instead of copying the table, so the two cannot drift.
    • Handoff to WS-D: PeerRequest.query carries decoded pairs. A unit argument reaches the proxy as the unit name itself (dev-disk-by\x2did.device, backslash included) and needs no encoding on D's side; the tool schema takes the plain name, so an already percent-encoded string such as dev-disk-by%5Cx2did.device is not a unit name and is 400.
    • Non-blocking 3: a .. inside a single segment is not traversal, because segments cannot contain / and the forwarded path is re-encoded from them. Pinned by allowlist cases systemd/units/foo..bar.mount and docker/containers/foo..bar, plus a rejection case for an encoded %2e%2e inside one segment (systemd/units/foo%2e%2ebar.mount), which encoding_forbidden catches.
    • docs/api.md states the unit query rule and the %5C/literal-backslash forms.
  • 2026-10-20 (from WS-F, blocking defect found by the F3 end-to-end test): every proxied call to a healthy peer answers 404 and the consolidator maps it to peer <name>: not found on peer.
    • Cause: proxy::get hands PeerRequest.path to the transport as the stripped route (/systemd/units), and ReqwestTransport::request_url builds {peer.url.path()}/{path}, i.e. http://<peer>:1984/systemd/units. An unmodified peer serves its REST routes only under /api/v1 (server::build_router nests api::router at /api/v1), so the peer's own 404 comes back. Health checks are unaffected: health::probe_api passes /api/v1/ping, which is the PeerRequest.path form contracts.md section 2 specifies ("e.g. /api/v1/systemd/units").
    • Reproduction (real transport, real peer, no VM): in the WS-F worktree, cargo test --test peers — the tests a_plain_peer_serves_its_own_data_through_the_proxy and shutdown_ends_a_proxied_follow are #[ignore]d with this reason and state the contract (proxied units equal a direct call to the peer; the proxied follow ends on shutdown). Removing the attribute is the acceptance check for this fix. It also blocks the proxy subtest of checks.vm-peers (nix/tests/peers.nix), which expects 200 from GET /api/v1/peers/beta/systemd/units.
    • Fix, one place: prefix the path the transport sees with /api/v1 in proxy::get (path: format!("/api/v1{path}") before building PeerRequest), keeping the stripped route for the allowlist and the forward form. Then the proxy.rs module doc ("the transport always sees one form" of the stripped path) and the unit tests asserting forwarded(&fake) == "/systemd/units" follow the new form. Doing it in the transport instead would leave the probes' /api/v1/ping double-prefixed, so the proxy side is the safer place.
    • Not fixed here: src/** is outside WS-F's Owned files.