Fold Phase 5 into docs

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
ariandClaude Sonnet 5.5 committed 2026-10-20 16:35:11 -04:00
1 parent 6be774e98b
commit f5e5f752e3
17 files changed
+71 -43

No files matched your search

+74
View File
@@ -0,0 +1,74 @@
# WS-G: Generated OpenAPI documentation
Stand-alone workstream, not part of Phase 5. Request: an autogenerated OpenAPI doc for the REST API and the MCP server.
| Field | Value |
|---|---|
| Status | Merged |
| Owner | goose |
| Depends on | Phase 5 routes and tools on main (peer routes are part of the spec) |
| Owned files | `src/openapi.rs`, `src/api/**`, `src/mcp/**`, `src/main.rs`, `src/lib.rs`, `src/cli.rs`, `Cargo.toml`, `tests/openapi.rs`, `docs/openapi.json`, `docs/api.md`, `README.md`, `config.example.toml`, `TODO.md` is NOT owned (merger ticks) |
## Tasks
1. **G1 REST spec.** Add `utoipa` (check that `cargo tree -i openssl` stays empty). Derive `ToSchema` on the response DTOs and annotate every REST handler with `#[utoipa::path]` (systemd, journal, docker, tokens-free endpoints, `/healthz`, `/api/v1/ping`, peers status and the peer proxy as one documented wildcard `GET /api/v1/peers/{peer}/{path}` listing the allowlist). Declare the bearer security scheme. Single `ApiDoc` in `src/openapi.rs`. The peer paths appear only when peers are enabled, matching the real router.
2. **G2 MCP description.** MCP is JSON-RPC, so it cannot be modelled as paths. Document `POST /mcp` (Streamable HTTP) in the same spec and add a vendor extension `x-mcp-tools` listing every tool with its description and input JSON Schema, generated from the rmcp tool router (the schemars schemas it already holds), not hand-written. Tool count is 8, or 10 with peers; the existing snapshot test must still pass.
3. **G3 Serving and CLI.** `GET /api/v1/openapi.json` behind the normal auth (no new public route), and `slinuxm openapi [--peers]` printing the spec offline without starting the server or reading any token. Keep the spec generation in one function shared by both.
4. **G4 Drift guard and docs.** Commit the generated `docs/openapi.json` (peers on). `tests/openapi.rs` regenerates it and fails with the regeneration command if it differs; also validates structure (every route in the real router is documented, every tool in the tool router appears in `x-mcp-tools`, so an undocumented new route or tool fails a test). Update `docs/api.md` and `README.md` briefly.
## Rules
Reuse existing DTOs, do not duplicate types for the doc. Read-only, no secrets in the spec or examples. Surfaces call `sources/`, do not touch D-Bus or Docker from the doc code. Tests in the same commit as the code. No workstream tags in comments.
## Log
- 2026-10-20: **G1 REST spec.** `utoipa` 6.0.0 (default features, macros included) added to `Cargo.toml`; `cargo tree -i openssl` is still empty. Every REST handler carries `#[utoipa::path]` with an explicit `operationId`, tag, params and the real response set (200 body plus 400/401/404/405/503 `ErrorBody` where they exist); handlers became `pub(crate)` so the `ApiDoc` derive in `src/openapi.rs` can name them. One `ApiDoc` plus one `document(peers: bool)` function; the bearer scheme is declared once (`bearer_auth`) and applied globally, with `/healthz` opting out via `security(())`. Peer paths are dropped when `peers` is false, matching the router (`paths.retain`). The peer pass-through is documented as one wildcard operation `GET /api/v1/peers/{peer}/{path}` whose description lists the allowlist.
**Owned-files conflict with G1 as written.** G1 says "derive `ToSchema` on the response DTOs", but those DTOs live in `src/sources/mod.rs` and `src/peers/types.rs`, which are not in this workstream's owned files, so no derive could be added there. Resolution, inside `src/openapi.rs` only: the DTOs already derive `schemars::JsonSchema` for MCP, so a small bridge (`reflected<T>()` + a `PartialSchema`/`ToSchema` impl macro) reflects each component from the schemars schema of the same type. There is exactly one definition per DTO and a field added to a DTO shows up in the spec with no edit here. `peer_status_components...`-style payloads that exist only as JSON (error body, `/healthz` and `/api/v1/ping` body) or have no schemars derive (`PeerStatus`, `ServerStatus`, `ApiStatus`, `AuthState` from `src/peers/types.rs`) are described by hand with utoipa builders and checked against the JSON the server really emits (`error_body_and_health_match_the_real_payloads`, `peer_status_components_cover_the_serialized_fields`), so an added field fails a test instead of vanishing from the document. `reflected_schemas_match_schemars` asserts the reflected component equals the schemars output exactly, which pins the converter (object/array/oneOf/anyOf/allOf/format/enum/default/minimum/maximum/required/properties).
Translation note for a future merger: if WS-G is ever allowed to touch the DTO files, the derive can replace the bridge for `src/sources/mod.rs`; the peer types would still need `JsonSchema` added.
Tests: 10 unit tests in `src/openapi.rs` (reflection parity, payload parity, peer gating, operation ids and 2xx, bearer scheme, determinism) and `tests/openapi.rs` with the router drift guard started here (source scan of every `.route(...)` literal in `src/api/**` plus `src/server.rs`, `/api/v1` prefix applied as `server.rs` nests it, axum `{*path}` normalised to OpenAPI `{path}`, and one request per documented path that must not hit the fallback's `no such route`). `cargo fmt --check`, `cargo clippy --all-targets -- -D warnings` and `cargo test` green.
- 2026-10-20: **G2 MCP description.** MCP is JSON-RPC, so it is documented as one operation `POST /mcp` (declared on a doc-only `mcp_endpoint()` in `src/openapi.rs`; the real handler is rmcp's `StreamableHttpService`, which has no axum handler fn to annotate) with the request body, the `application/json` + `text/event-stream` response, and the 400/401/403/404/406/415 codes rmcp and the auth layer really return. `x-mcp-tools` is generated from the tool router itself: `TrackerService::advertised_tools(peers)` builds the same `ToolRouter` that `TrackerService::new` stores (extracted into `router_for`, so there is one place that applies the peer gating) and returns `name`, `description` and `input_schema` for every tool, sorted, so the extension cannot drift from `tools/list`. Tool count stays 8 without peers and 10 with them; the existing peers-off snapshot test is untouched and still passes.
Tests: `mcp_endpoint_is_documented_in_both_modes` and `mcp_extension_lists_tools_sorted_with_schemas` in `src/openapi.rs`; `mcp_tools_match_the_tool_router` (every entry equal to the router's name/description/input schema, both modes), `mcp_tool_count_follows_the_peers_flag` and the `/mcp` mount added to the route scan in `tests/openapi.rs`. `cargo fmt --check`, `cargo clippy --all-targets -- -D warnings` and `cargo test` green.
- 2026-10-20: **G3 Serving and CLI.** `document(peers)` now returns a cached `Arc<OpenApi>` (one `OnceLock` per mode; generation walks every DTO and the whole tool router, and the result cannot change while the process runs), with `build`/`to_pretty_json` behind it so one function is shared by every consumer. `GET /api/v1/openapi.json` is mounted in `api::router`, i.e. under `/api/v1` and therefore behind the existing auth layer; no new public route, no change to `server::build_router` or `/healthz`. It serves the mode of the running instance (`state.peers.is_some()`), so a peer-less instance advertises no peer paths. New `slinuxm openapi [--peers]` (`src/cli.rs`, `Command::Openapi`) prints the pretty document; `main` returns before `Config::load`, so it needs no config file, no token file and no socket and cannot start a server or bind a port. `--print-config` still wins if combined. The `peers::cli` import in `main.rs` is now aliased to `peer_cli` because `cli` is the new module.
Tests: `render_is_valid_json_for_both_modes`, `generation_is_deterministic` (bytes, not just values, because the committed file is compared) in the unit tests; `openapi_route_is_authenticated_and_serves_the_document` (401 without a token, 200 with one, body equals the generated peers-on document, `application/json`) and `cli_prints_the_same_document_offline` (real binary, `--config /nonexistent/...`, both modes, output equals the generated document) in `tests/openapi.rs`.
- 2026-10-20: **G4 Drift guard and docs.** `docs/openapi.json` is committed (peers on, pretty-printed by `slinuxm openapi --peers`, 15 paths and 10 tools). `tests/openapi.rs` now holds the guards: `committed_spec_matches_the_code` compares the file byte for byte with `cli::render(true)` and prints the regeneration command (`cargo run -- openapi --peers > docs/openapi.json`) on failure; `every_router_route_is_documented` scans every `.route(...)` literal in `src/api/**` and `src/server.rs` plus the `/mcp` mount and requires each path in the document (verified by adding a throwaway route: the test failed with `/api/v1/ping/extra is routed but missing from the OpenAPI document`); `every_documented_path_is_routed` sends one request per documented path through the assembled router and fails if it hits the fallback's `no such route`; `mcp_tools_match_the_tool_router` requires every tool of the router in `x-mcp-tools` with the same description and input schema (a new or gated-off tool changes the spec and then fails the staleness check); `every_operation_is_read_only` keeps every operation a GET except `POST /mcp`; `operation_ids_and_tags_are_declared` catches duplicate ids and undeclared tags. A crate version bump changes `info.version`, so it also needs the regeneration in the same commit; that is deliberate (the file is the artifact that is published).
Docs: `docs/api.md` gains a "Generated reference (OpenAPI)" section (what the file contains, `GET /api/v1/openapi.json`, `slinuxm openapi [--peers]`, regeneration, what the tests guard) and a route-table row; `README.md` gets a short paragraph in the REST section. `config.example.toml` needs nothing (no new key). `docs/README.md` does not list `docs/openapi.json` yet; it is outside this workstream's owned files, so the merger should add the row.
Verification: `cargo fmt --check`, `cargo clippy --all-targets -- -D warnings`, `cargo test` green (10 tests in `tests/openapi.rs`, 12 unit tests in `src/openapi.rs`/`src/cli.rs`); `cargo tree -i openssl` empty; the committed spec contains no token, token-file path or URL other than the documented default server. No nix build was run; no VM test is involved in this workstream.
- 2026-10-20: **G4 self-review fixes.** Three accuracy gaps found on a second read: the `ErrorBody.code` enum listed only the `AppError` codes, but two status-only bodies reuse `error_response` (`method_not_allowed` from the 405 fallback, `forbidden_host` from the auth-off Host guard), so both are now documented and `every_error_code_is_documented` ties the enum to `AppError::code()` plus those two; the docker `logs` and the peer pass-through 200s now declare both `application/json` and `text/event-stream` instead of JSON only; `schema_type` no longer maps an unknown JSON Schema type name to `object` (it leaves the type out), so the converter cannot invent a type the DTO does not have. Spec regenerated.
- 2026-10-20: **Handoff (G1 to G4 complete).** The REST API and the MCP server are described by one generated OpenAPI 3.1 document.
- **G1** `utoipa` 6.0.0; `#[utoipa::path]` on every handler (explicit `operationId`s, tags, parameters and the real status set), bearer scheme declared and applied globally with `/healthz` opting out, single `ApiDoc` + `document(peers)` in `src/openapi.rs`. Commits `d21e5e7`, plus the ownership-conflict resolution described above: DTO components are reflected from the existing `schemars` derives instead of a new `ToSchema` derive in files WS-G does not own; the peer status types and the JSON-only bodies are handwritten and parity-tested.
- **G2** `POST /mcp` documented (request body, JSON/SSE 200, the real 400/401/403/404/406/415), `x-mcp-tools` generated from `TrackerService::advertised_tools` (the same router that serves `tools/list`), 8 or 10 tools. Commit `f7a18dc`.
- **G3** `GET /api/v1/openapi.json` behind the existing auth, cached per mode, and `slinuxm openapi [--peers]`, which returns before `Config::load` so it needs no config, token or socket. Commit `f1cb210`.
- **G4** `docs/openapi.json` committed (peers on), staleness test printing the regeneration command, route-scan and path-probe guards, tool-table guard, read-only and operationId/tag checks; `docs/api.md` and `README.md` updated. Commits `539651c`, `646e5d4`.
**Verification.** `cargo fmt --check`, `cargo clippy --all-targets -- -D warnings` and `cargo test` are green on the final tree (273 unit tests, of which 12 in `src/openapi.rs` and 1 in `src/cli.rs`; 10 integration tests in `tests/openapi.rs`; the existing peers/MCP/TLS/plain suites unchanged). `cargo tree -i openssl` is empty; `utoipa` brings only `utoipa` and `utoipa-gen`, no crypto. The route scan was proved live by adding a throwaway route (test failed naming it) and the staleness guard by appending a byte to the committed file (test failed with the regeneration command). No write path, no secrets, no nix build, no VM test in this workstream.
**Note on running the checks here.** `devenv shell -- dev-test …` does not start in this worktree: `devenv` aborts with "Failed assertions: - To use 'git-hooks', run the following command: $ devenv inputs add git-hooks github:cachix/git-hooks.nix --follows nixpkgs" (`devenv.yaml` in this worktree lists only the nixpkgs input). The equivalent commands were run directly (`cargo fmt --check`, `cargo clippy --all-targets -- -D warnings`, `cargo test`). `dev/hooks/select.sh` already maps `src/openapi.rs` to `test-unit:openapi` + `test-it:openapi`, so the hooks pick the new tests up without a change.
**For the merger.** (1) Tick the G1 to G4 boxes in `TODO.md` (not owned here). (2) `docs/README.md` has no row for `docs/openapi.json` yet; add one under "Reference" (not owned here). (3) `AGENTS.md` "The MCP tool count is 8" can now point at `x-mcp-tools` and `tests/openapi.rs`, which pin both counts (8 peer-less, 10 with peers); not owned here. (4) `config.example.toml` needs nothing: the workstream adds no config key. (5) A crate version bump changes `info.version`, so `docs/openapi.json` must be regenerated in the same commit; the test message says how. If that turns out to be noisy, the alternative is to leave `info.version` out of the committed file, which weakens it. (6) If WS-G is ever allowed to touch `src/sources/mod.rs` and `src/peers/types.rs`, the `ToSchema` derives from the G1 task can replace the schemars bridge and the handwritten peer schemas; until then the bridge is the cheaper way to keep one definition per DTO.
- 2026-10-20: **Review-1 fixes.** Each finding from `loop/G/review-1.md` (verdict line ignored) was addressed and committed on its own:
- **F1** `every_documented_path_is_routed` probed with auth on, so `require_token` answered 401 before routing and the test could never fail. It now builds the router with `auth.enabled = false`, sends the method each operation actually declares (not GET for everything), asserts the body is not the fallback's `no such route`, and a new `documented_peer_paths_are_absent_without_peers` checks every `document(false)` path through a peer-less router plus three peer paths that must answer the fallback. Commit `f8595b6`.
- **F2** The peer status types now derive `schemars::JsonSchema` (`PeerName` is `#[schemars(transparent)]`), so `PeerStatus`, `ServerStatus`, `ApiStatus` and `AuthState` join the reflected components; the ~150-line handwritten block, `check_properties`, `nullable`, `date_time` and `SINCE_DOC` are gone. `assert_covers` now checks both directions (serialized keys equal documented properties), and the exact schemars parity test covers the peer types too. Commit `9449499`.
- **F3** `JournalParams`, `PsParams`, `LogParams` and `UnitsQuery` derive `IntoParams` (`parameter_in = Query`) and the path macros name the structs, so a new query key cannot drift. This also corrected `since` to `format: date-time` and set `minimum: 1` on `lines` and `tail`. Commit `b1b39e8`.
- **F4** The route scan now parses `.route` calls independently of formatting (the newlines in `.route(\n "/path"` no longer hide a route), reads the method builders and compares them with the document both ways, requires exactly 15 routes, requires every file that builds a `Router` to contribute a route, walks `src/api` recursively and rejects an unscanned `.nest`/`.nest_service` in `src/api`. A unit test pins a multiline `get(...).post(...)` call. Commit `0f07ee4`.
- **F5** `/mcp` now documents all three operations the default `LocalSessionManager` mode serves: `POST` (JSON-RPC), `GET` (standalone SSE, 400/404/406) and `DELETE` (session end, 202). `every_operation_is_read_only` allows the session `DELETE` because it changes no host state, and `docs/api.md` says the same. Commit `8f6c94a`.
- **F6** The `health` tag no longer calls `/api/v1/ping` unauthenticated; the auth layer's 503 is documented on `ping`, both peer reads, `openapi.json` and `/mcp`; the `info.description` explains that `[auth] enabled = false` disables the bearer requirement while the host guard can still answer 403 `forbidden_host`. The `since` format and `lines`/`tail` bounds landed with F3. Commit `83b4546`.
- **F7** `docs/README.md` gains the `docs/openapi.json` row under Reference. Commit `e17d537`.
**Deviation from the owned files.** F2 required `src/peers/types.rs`, which the header does not own; the reviewer and the task explicitly allowed this, and it is a strictly smaller deviation than the schemars bridge's handwritten copy. F7 required `docs/README.md`, also outside the owned list; the previous handoff had left that row to the merger. No `src/sources` file was touched. `TODO.md` needs no change: the G1 to G4 tasks were never migrated into it, so there are no boxes to tick, and the merger still owns that move.
**Handoff (2026-10-20).** All seven findings are fixed and committed separately. `docs/openapi.json` was regenerated with the documented command (`cargo run -- openapi --peers > docs/openapi.json`) after the last spec change. `cargo fmt --check`, `cargo clippy --all-targets -- -D warnings` and `cargo test` are green on the final tree (273 unit tests, 12 in `src/openapi.rs`; 12 integration tests in `tests/openapi.rs`; the peers, MCP, TLS, plain and binary suites unchanged). `cargo tree -i openssl` is still empty. The remaining merger items from the previous handoff are unchanged; the `docs/README.md` row is now done.