Files
skyg-linux-tracker/docs/workstreams/WS-G-openapi.md
T
2026-10-20 16:35:11 -04:00

18 KiB

WS-G: Generated OpenAPI documentation

Stand-alone workstream, not part of Phase 5. Request: an autogenerated OpenAPI doc for the REST API and the MCP server.

Field Value
Status Merged
Owner goose
Depends on Phase 5 routes and tools on main (peer routes are part of the spec)
Owned files src/openapi.rs, src/api/**, src/mcp/**, src/main.rs, src/lib.rs, src/cli.rs, Cargo.toml, tests/openapi.rs, docs/openapi.json, docs/api.md, README.md, config.example.toml, TODO.md is NOT owned (merger ticks)

Tasks

  1. G1 REST spec. Add utoipa (check that cargo tree -i openssl stays empty). Derive ToSchema on the response DTOs and annotate every REST handler with #[utoipa::path] (systemd, journal, docker, tokens-free endpoints, /healthz, /api/v1/ping, peers status and the peer proxy as one documented wildcard GET /api/v1/peers/{peer}/{path} listing the allowlist). Declare the bearer security scheme. Single ApiDoc in src/openapi.rs. The peer paths appear only when peers are enabled, matching the real router.
  2. G2 MCP description. MCP is JSON-RPC, so it cannot be modelled as paths. Document POST /mcp (Streamable HTTP) in the same spec and add a vendor extension x-mcp-tools listing every tool with its description and input JSON Schema, generated from the rmcp tool router (the schemars schemas it already holds), not hand-written. Tool count is 8, or 10 with peers; the existing snapshot test must still pass.
  3. G3 Serving and CLI. GET /api/v1/openapi.json behind the normal auth (no new public route), and slinuxm openapi [--peers] printing the spec offline without starting the server or reading any token. Keep the spec generation in one function shared by both.
  4. G4 Drift guard and docs. Commit the generated docs/openapi.json (peers on). tests/openapi.rs regenerates it and fails with the regeneration command if it differs; also validates structure (every route in the real router is documented, every tool in the tool router appears in x-mcp-tools, so an undocumented new route or tool fails a test). Update docs/api.md and README.md briefly.

Rules

Reuse existing DTOs, do not duplicate types for the doc. Read-only, no secrets in the spec or examples. Surfaces call sources/, do not touch D-Bus or Docker from the doc code. Tests in the same commit as the code. No workstream tags in comments.

Log

  • 2026-10-20: G1 REST spec. utoipa 6.0.0 (default features, macros included) added to Cargo.toml; cargo tree -i openssl is still empty. Every REST handler carries #[utoipa::path] with an explicit operationId, tag, params and the real response set (200 body plus 400/401/404/405/503 ErrorBody where they exist); handlers became pub(crate) so the ApiDoc derive in src/openapi.rs can name them. One ApiDoc plus one document(peers: bool) function; the bearer scheme is declared once (bearer_auth) and applied globally, with /healthz opting out via security(()). Peer paths are dropped when peers is false, matching the router (paths.retain). The peer pass-through is documented as one wildcard operation GET /api/v1/peers/{peer}/{path} whose description lists the allowlist.

    Owned-files conflict with G1 as written. G1 says "derive ToSchema on the response DTOs", but those DTOs live in src/sources/mod.rs and src/peers/types.rs, which are not in this workstream's owned files, so no derive could be added there. Resolution, inside src/openapi.rs only: the DTOs already derive schemars::JsonSchema for MCP, so a small bridge (reflected<T>() + a PartialSchema/ToSchema impl macro) reflects each component from the schemars schema of the same type. There is exactly one definition per DTO and a field added to a DTO shows up in the spec with no edit here. peer_status_components...-style payloads that exist only as JSON (error body, /healthz and /api/v1/ping body) or have no schemars derive (PeerStatus, ServerStatus, ApiStatus, AuthState from src/peers/types.rs) are described by hand with utoipa builders and checked against the JSON the server really emits (error_body_and_health_match_the_real_payloads, peer_status_components_cover_the_serialized_fields), so an added field fails a test instead of vanishing from the document. reflected_schemas_match_schemars asserts the reflected component equals the schemars output exactly, which pins the converter (object/array/oneOf/anyOf/allOf/format/enum/default/minimum/maximum/required/properties).

    Translation note for a future merger: if WS-G is ever allowed to touch the DTO files, the derive can replace the bridge for src/sources/mod.rs; the peer types would still need JsonSchema added.

    Tests: 10 unit tests in src/openapi.rs (reflection parity, payload parity, peer gating, operation ids and 2xx, bearer scheme, determinism) and tests/openapi.rs with the router drift guard started here (source scan of every .route(...) literal in src/api/** plus src/server.rs, /api/v1 prefix applied as server.rs nests it, axum {*path} normalised to OpenAPI {path}, and one request per documented path that must not hit the fallback's no such route). cargo fmt --check, cargo clippy --all-targets -- -D warnings and cargo test green.

  • 2026-10-20: G2 MCP description. MCP is JSON-RPC, so it is documented as one operation POST /mcp (declared on a doc-only mcp_endpoint() in src/openapi.rs; the real handler is rmcp's StreamableHttpService, which has no axum handler fn to annotate) with the request body, the application/json + text/event-stream response, and the 400/401/403/404/406/415 codes rmcp and the auth layer really return. x-mcp-tools is generated from the tool router itself: TrackerService::advertised_tools(peers) builds the same ToolRouter that TrackerService::new stores (extracted into router_for, so there is one place that applies the peer gating) and returns name, description and input_schema for every tool, sorted, so the extension cannot drift from tools/list. Tool count stays 8 without peers and 10 with them; the existing peers-off snapshot test is untouched and still passes.

    Tests: mcp_endpoint_is_documented_in_both_modes and mcp_extension_lists_tools_sorted_with_schemas in src/openapi.rs; mcp_tools_match_the_tool_router (every entry equal to the router's name/description/input schema, both modes), mcp_tool_count_follows_the_peers_flag and the /mcp mount added to the route scan in tests/openapi.rs. cargo fmt --check, cargo clippy --all-targets -- -D warnings and cargo test green.

  • 2026-10-20: G3 Serving and CLI. document(peers) now returns a cached Arc<OpenApi> (one OnceLock per mode; generation walks every DTO and the whole tool router, and the result cannot change while the process runs), with build/to_pretty_json behind it so one function is shared by every consumer. GET /api/v1/openapi.json is mounted in api::router, i.e. under /api/v1 and therefore behind the existing auth layer; no new public route, no change to server::build_router or /healthz. It serves the mode of the running instance (state.peers.is_some()), so a peer-less instance advertises no peer paths. New slinuxm openapi [--peers] (src/cli.rs, Command::Openapi) prints the pretty document; main returns before Config::load, so it needs no config file, no token file and no socket and cannot start a server or bind a port. --print-config still wins if combined. The peers::cli import in main.rs is now aliased to peer_cli because cli is the new module.

    Tests: render_is_valid_json_for_both_modes, generation_is_deterministic (bytes, not just values, because the committed file is compared) in the unit tests; openapi_route_is_authenticated_and_serves_the_document (401 without a token, 200 with one, body equals the generated peers-on document, application/json) and cli_prints_the_same_document_offline (real binary, --config /nonexistent/..., both modes, output equals the generated document) in tests/openapi.rs.

  • 2026-10-20: G4 Drift guard and docs. docs/openapi.json is committed (peers on, pretty-printed by slinuxm openapi --peers, 15 paths and 10 tools). tests/openapi.rs now holds the guards: committed_spec_matches_the_code compares the file byte for byte with cli::render(true) and prints the regeneration command (cargo run -- openapi --peers > docs/openapi.json) on failure; every_router_route_is_documented scans every .route(...) literal in src/api/** and src/server.rs plus the /mcp mount and requires each path in the document (verified by adding a throwaway route: the test failed with /api/v1/ping/extra is routed but missing from the OpenAPI document); every_documented_path_is_routed sends one request per documented path through the assembled router and fails if it hits the fallback's no such route; mcp_tools_match_the_tool_router requires every tool of the router in x-mcp-tools with the same description and input schema (a new or gated-off tool changes the spec and then fails the staleness check); every_operation_is_read_only keeps every operation a GET except POST /mcp; operation_ids_and_tags_are_declared catches duplicate ids and undeclared tags. A crate version bump changes info.version, so it also needs the regeneration in the same commit; that is deliberate (the file is the artifact that is published).

    Docs: docs/api.md gains a "Generated reference (OpenAPI)" section (what the file contains, GET /api/v1/openapi.json, slinuxm openapi [--peers], regeneration, what the tests guard) and a route-table row; README.md gets a short paragraph in the REST section. config.example.toml needs nothing (no new key). docs/README.md does not list docs/openapi.json yet; it is outside this workstream's owned files, so the merger should add the row.

    Verification: cargo fmt --check, cargo clippy --all-targets -- -D warnings, cargo test green (10 tests in tests/openapi.rs, 12 unit tests in src/openapi.rs/src/cli.rs); cargo tree -i openssl empty; the committed spec contains no token, token-file path or URL other than the documented default server. No nix build was run; no VM test is involved in this workstream.

  • 2026-10-20: G4 self-review fixes. Three accuracy gaps found on a second read: the ErrorBody.code enum listed only the AppError codes, but two status-only bodies reuse error_response (method_not_allowed from the 405 fallback, forbidden_host from the auth-off Host guard), so both are now documented and every_error_code_is_documented ties the enum to AppError::code() plus those two; the docker logs and the peer pass-through 200s now declare both application/json and text/event-stream instead of JSON only; schema_type no longer maps an unknown JSON Schema type name to object (it leaves the type out), so the converter cannot invent a type the DTO does not have. Spec regenerated.

  • 2026-10-20: Handoff (G1 to G4 complete). The REST API and the MCP server are described by one generated OpenAPI 3.1 document.

    • G1 utoipa 6.0.0; #[utoipa::path] on every handler (explicit operationIds, tags, parameters and the real status set), bearer scheme declared and applied globally with /healthz opting out, single ApiDoc + document(peers) in src/openapi.rs. Commits d21e5e7, plus the ownership-conflict resolution described above: DTO components are reflected from the existing schemars derives instead of a new ToSchema derive in files WS-G does not own; the peer status types and the JSON-only bodies are handwritten and parity-tested.
    • G2 POST /mcp documented (request body, JSON/SSE 200, the real 400/401/403/404/406/415), x-mcp-tools generated from TrackerService::advertised_tools (the same router that serves tools/list), 8 or 10 tools. Commit f7a18dc.
    • G3 GET /api/v1/openapi.json behind the existing auth, cached per mode, and slinuxm openapi [--peers], which returns before Config::load so it needs no config, token or socket. Commit f1cb210.
    • G4 docs/openapi.json committed (peers on), staleness test printing the regeneration command, route-scan and path-probe guards, tool-table guard, read-only and operationId/tag checks; docs/api.md and README.md updated. Commits 539651c, 646e5d4.

    Verification. cargo fmt --check, cargo clippy --all-targets -- -D warnings and cargo test are green on the final tree (273 unit tests, of which 12 in src/openapi.rs and 1 in src/cli.rs; 10 integration tests in tests/openapi.rs; the existing peers/MCP/TLS/plain suites unchanged). cargo tree -i openssl is empty; utoipa brings only utoipa and utoipa-gen, no crypto. The route scan was proved live by adding a throwaway route (test failed naming it) and the staleness guard by appending a byte to the committed file (test failed with the regeneration command). No write path, no secrets, no nix build, no VM test in this workstream.

    Note on running the checks here. devenv shell -- dev-test … does not start in this worktree: devenv aborts with "Failed assertions: - To use 'git-hooks', run the following command: $ devenv inputs add git-hooks github:cachix/git-hooks.nix --follows nixpkgs" (devenv.yaml in this worktree lists only the nixpkgs input). The equivalent commands were run directly (cargo fmt --check, cargo clippy --all-targets -- -D warnings, cargo test). dev/hooks/select.sh already maps src/openapi.rs to test-unit:openapi + test-it:openapi, so the hooks pick the new tests up without a change.

    For the merger. (1) Tick the G1 to G4 boxes in TODO.md (not owned here). (2) docs/README.md has no row for docs/openapi.json yet; add one under "Reference" (not owned here). (3) AGENTS.md "The MCP tool count is 8" can now point at x-mcp-tools and tests/openapi.rs, which pin both counts (8 peer-less, 10 with peers); not owned here. (4) config.example.toml needs nothing: the workstream adds no config key. (5) A crate version bump changes info.version, so docs/openapi.json must be regenerated in the same commit; the test message says how. If that turns out to be noisy, the alternative is to leave info.version out of the committed file, which weakens it. (6) If WS-G is ever allowed to touch src/sources/mod.rs and src/peers/types.rs, the ToSchema derives from the G1 task can replace the schemars bridge and the handwritten peer schemas; until then the bridge is the cheaper way to keep one definition per DTO.

  • 2026-10-20: Review-1 fixes. Each finding from loop/G/review-1.md (verdict line ignored) was addressed and committed on its own:

    • F1 every_documented_path_is_routed probed with auth on, so require_token answered 401 before routing and the test could never fail. It now builds the router with auth.enabled = false, sends the method each operation actually declares (not GET for everything), asserts the body is not the fallback's no such route, and a new documented_peer_paths_are_absent_without_peers checks every document(false) path through a peer-less router plus three peer paths that must answer the fallback. Commit f8595b6.
    • F2 The peer status types now derive schemars::JsonSchema (PeerName is #[schemars(transparent)]), so PeerStatus, ServerStatus, ApiStatus and AuthState join the reflected components; the ~150-line handwritten block, check_properties, nullable, date_time and SINCE_DOC are gone. assert_covers now checks both directions (serialized keys equal documented properties), and the exact schemars parity test covers the peer types too. Commit 9449499.
    • F3 JournalParams, PsParams, LogParams and UnitsQuery derive IntoParams (parameter_in = Query) and the path macros name the structs, so a new query key cannot drift. This also corrected since to format: date-time and set minimum: 1 on lines and tail. Commit b1b39e8.
    • F4 The route scan now parses .route calls independently of formatting (the newlines in .route(\n "/path" no longer hide a route), reads the method builders and compares them with the document both ways, requires exactly 15 routes, requires every file that builds a Router to contribute a route, walks src/api recursively and rejects an unscanned .nest/.nest_service in src/api. A unit test pins a multiline get(...).post(...) call. Commit 0f07ee4.
    • F5 /mcp now documents all three operations the default LocalSessionManager mode serves: POST (JSON-RPC), GET (standalone SSE, 400/404/406) and DELETE (session end, 202). every_operation_is_read_only allows the session DELETE because it changes no host state, and docs/api.md says the same. Commit 8f6c94a.
    • F6 The health tag no longer calls /api/v1/ping unauthenticated; the auth layer's 503 is documented on ping, both peer reads, openapi.json and /mcp; the info.description explains that [auth] enabled = false disables the bearer requirement while the host guard can still answer 403 forbidden_host. The since format and lines/tail bounds landed with F3. Commit 83b4546.
    • F7 docs/README.md gains the docs/openapi.json row under Reference. Commit e17d537.

    Deviation from the owned files. F2 required src/peers/types.rs, which the header does not own; the reviewer and the task explicitly allowed this, and it is a strictly smaller deviation than the schemars bridge's handwritten copy. F7 required docs/README.md, also outside the owned list; the previous handoff had left that row to the merger. No src/sources file was touched. TODO.md needs no change: the G1 to G4 tasks were never migrated into it, so there are no boxes to tick, and the merger still owns that move.

    Handoff (2026-10-20). All seven findings are fixed and committed separately. docs/openapi.json was regenerated with the documented command (cargo run -- openapi --peers > docs/openapi.json) after the last spec change. cargo fmt --check, cargo clippy --all-targets -- -D warnings and cargo test are green on the final tree (273 unit tests, 12 in src/openapi.rs; 12 integration tests in tests/openapi.rs; the peers, MCP, TLS, plain and binary suites unchanged). cargo tree -i openssl is still empty. The remaining merger items from the previous handoff are unchanged; the docs/README.md row is now done.