added live redoc and slight refactors to tokens
This commit is contained in:
1 parent
435a54a02d
commit
b38c66bcc8
19 files changed
+836
-157
No files matched your search
@@ -6,23 +6,23 @@ _Last updated: 2026-10-20_
|
||||
|
||||
## Status
|
||||
|
||||
The build is functionally complete and merged to `main`. REST, MCP, OTel, TLS and the NixOS module are implemented. Verification #1–#4 and #6 passed live on minipc1 (2026-10-19). The WS-H nix gate passed on 2026-10-19. Phase 5 (consolidator, optional peers) and the generated OpenAPI doc are merged; the Phase 5 record is archived in [`docs/workstreams/P5-consolidator/`](docs/workstreams/P5-consolidator/README.md). Everything open is in the Backlog.
|
||||
The build is functionally complete and merged to `main`. REST, MCP, OTel, TLS and the NixOS module are implemented. Verification #1–#4 and #6 passed live on minipc1 (2026-10-19). The WS-H nix gate passed on 2026-10-19. Phase 5 (consolidator, optional peers) and the generated OpenAPI doc are merged, and the binary serves it as a Redoc page at `/docs` (DEC-16); the Phase 5 record is archived in [`docs/workstreams/P5-consolidator/`](docs/workstreams/P5-consolidator/README.md). Everything open is in the Backlog.
|
||||
|
||||
| Milestone | Gate | Status |
|
||||
|---|---|---|
|
||||
| M0 Skeleton | `slinuxm serve --print-config` works. Clippy and tests are clean. No openssl. | Done |
|
||||
| M1 Sources live | All four Phase 1 examples run against a real host | Done (systemd, journal, docker verified live) |
|
||||
| M2 Surfaces | Verification #3, #4 and #5 pass | Partial: #3 and #4 pass live (minipc1, 2026-10-19, all 8 MCP tools); #5 OTel deferred to I6-5 (collector on minipc1 is down) |
|
||||
| M3 Release | Verification #1–#6 pass. Unit file and README shipped. | Partial: #1–#4 and #6 pass, unit installed on minipc1; needs #5 (I6-5) |
|
||||
| M4 NixOS | `nix flake check` passes including the VM test. `:1984` serves and a token is minted with the system-installed CLI. | ✅ 2026-10-19: `nix build` and `nix flake check` (incl. VM test, plain + TLS) pass; see [WS-H log](docs/workstreams/WS-H-nixos.md#log) |
|
||||
| M5 TLS | HTTPS serves REST and `/mcp`, SIGHUP reloads certs, no openssl. | Done |
|
||||
| M6 Consolidator | Phase 5 exit criteria; peer-less instance unchanged (8 MCP tools); `nix flake check` incl. the peers VM test | Met, except two live checks still open: **P5-1** minipc1 cross-check needs a user-minted token, **P5-2** OTel peer-gauge check via `devenv up` not yet recorded (both in the Backlog) |
|
||||
| Milestone | Gate | Status |
|
||||
| --------------- | ------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| M0 Skeleton | `slinuxm serve --print-config` works. Clippy and tests are clean. No openssl. | Done |
|
||||
| M1 Sources live | All four Phase 1 examples run against a real host | Done (systemd, journal, docker verified live) |
|
||||
| M2 Surfaces | Verification #3, #4 and #5 pass | Partial: #3 and #4 pass live (minipc1, 2026-10-19, all 8 MCP tools); #5 OTel deferred to I6-5 (collector on minipc1 is down) |
|
||||
| M3 Release | Verification #1–#6 pass. Unit file and README shipped. | Partial: #1–#4 and #6 pass, unit installed on minipc1; needs #5 (I6-5) |
|
||||
| M4 NixOS | `nix flake check` passes including the VM test. `:1984` serves and a token is minted with the system-installed CLI. | ✅ 2026-10-19: `nix build` and `nix flake check` (incl. VM test, plain + TLS) pass; see [WS-H log](docs/workstreams/WS-H-nixos.md#log) |
|
||||
| M5 TLS | HTTPS serves REST and `/mcp`, SIGHUP reloads certs, no openssl. | Done |
|
||||
| M6 Consolidator | Phase 5 exit criteria; peer-less instance unchanged (8 MCP tools); `nix flake check` incl. the peers VM test | Met, except two live checks still open: **P5-1** minipc1 cross-check needs a user-minted token, **P5-2** OTel peer-gauge check via `devenv up` not yet recorded (both in the Backlog) |
|
||||
|
||||
## In progress / Review
|
||||
|
||||
| Item | Owner | State |
|
||||
|---|---|---|
|
||||
| — | | Nothing in flight |
|
||||
| Item | Owner | State |
|
||||
| ---- | ----- | ----------------- |
|
||||
| — | | Nothing in flight |
|
||||
|
||||
- [x] **WS-H Gate:** `nix build` and `nix flake check` pass. `ldd` shows no openssl. A fresh NixOS host serves `:1984` and a token can be minted from a login shell.
|
||||
|
||||
@@ -30,34 +30,38 @@ The build is functionally complete and merged to `main`. REST, MCP, OTel, TLS an
|
||||
|
||||
Parked, not scheduled. Each has a reason. Nothing here blocks the current work. Tick here when picked up and finished.
|
||||
|
||||
| ID | Item | Reason parked |
|
||||
|---|---|---|
|
||||
| **C7** | *(optional)* Smoke test against the Podman socket (WS-C) | Nice-to-have; no Podman host in use |
|
||||
| **I6-5** | Verification #5: OTel smoke test end to end (P3) | Waiting on the minipc1 collector fix (**X-1**); `slinuxm` itself exports fine |
|
||||
| **B-1** | Live check of docker log-follow SSE (`/docker/containers/{name}/logs?follow=1`) | Not exercised in the live runs so far |
|
||||
| **B-2** | Live check of REST, bearer auth and TLS against minipc1 | The MCP-tool run did not cover them |
|
||||
| **X-1** | *(external)* Investigate failed `opentelemetry-collector.service` and `var-lib-loki.mount` on minipc1 | Outside this repo; blocks I6-5 |
|
||||
| **F-1** | Cap concurrent follow streams | Security review 2026-10-16; not scheduled |
|
||||
| **F-2** | Bound line and message length | Security review 2026-10-16; not scheduled |
|
||||
| **F-3** | Unit hardening extras | Security review 2026-10-16; needs a clean-host test first |
|
||||
| **P5-1** | minipc1 cross-check of the consolidator against a real peer | Needs a read-only token minted by the user on minipc1; agents never mint tokens on real hosts |
|
||||
| **P5-2** | Record the OTel peer-gauge check (`devenv up`, Verification step 5 with peers) | Not yet run and recorded; no automated test asserts the exported peer metrics end to end |
|
||||
| **D8** | Fleet aggregation (merged views across peers) | Out of scope for v1 (DEC-14: no aggregation, no chaining); needs a design for partial failure and ordering |
|
||||
| **P5-3** | Scoped tokens (per-peer or read-class scopes) | Any valid inbound token may read peer data (DEC-14); scopes need a token schema change and a policy decision |
|
||||
| **P5-4** | Peer health history | State is in memory only by design; history needs storage or an external TSDB, and the OTel gauges already cover it |
|
||||
| **P5-5** | Runtime peer management (add/remove without editing config) | Peers are declarative so no write path or DB table exists (DEC-14); would need a write path against the read-only rule |
|
||||
| ID | Item | Reason parked |
|
||||
| --------- | ----------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **C7** | _(optional)_ Smoke test against the Podman socket (WS-C) | Nice-to-have; no Podman host in use |
|
||||
| **I6-5** | Verification #5: OTel smoke test end to end (P3) | Waiting on the minipc1 collector fix (**X-1**); `slinuxm` itself exports fine |
|
||||
| **B-1** | Live check of docker log-follow SSE (`/docker/containers/{name}/logs?follow=1`) | Not exercised in the live runs so far |
|
||||
| **B-2** | Live check of REST, bearer auth and TLS against minipc1 | The MCP-tool run did not cover them |
|
||||
| **X-1** | _(external)_ Investigate failed `opentelemetry-collector.service` and `var-lib-loki.mount` on minipc1 | Outside this repo; blocks I6-5 |
|
||||
| **F-1** | Cap concurrent follow streams | Security review 2026-10-16; not scheduled |
|
||||
| **F-2** | Bound line and message length | Security review 2026-10-16; not scheduled |
|
||||
| **F-3** | Unit hardening extras | Security review 2026-10-16; needs a clean-host test first |
|
||||
| **P5-1** | minipc1 cross-check of the consolidator against a real peer | Needs a read-only token minted by the user on minipc1; agents never mint tokens on real hosts |
|
||||
| **P5-2** | Record the OTel peer-gauge check (`devenv up`, Verification step 5 with peers) | Not yet run and recorded; no automated test asserts the exported peer metrics end to end |
|
||||
| **D8** | Fleet aggregation (merged views across peers) | Out of scope for v1 (DEC-14: no aggregation, no chaining); needs a design for partial failure and ordering |
|
||||
| **P5-3** | Scoped tokens (per-peer or read-class scopes) | Any valid inbound token may read peer data (DEC-14); scopes need a token schema change and a policy decision |
|
||||
| **P5-4** | Peer health history | State is in memory only by design; history needs storage or an external TSDB, and the OTel gauges already cover it |
|
||||
| **P5-5** | Runtime peer management (add/remove without editing config) | Peers are declarative so no write path or DB table exists (DEC-14); would need a write path against the read-only rule |
|
||||
| **DOC-1** | Show the MCP tools (`x-mcp-tools`) in the Redoc page | Redoc renders REST operations only; needs the tool list generated into a tag description (or a section) in `openapi.rs`, which changes the spec |
|
||||
| **DOC-2** | Config switch to turn the `/docs` viewer off | Static, public and data-free (DEC-16), so not needed yet; adds a config key, README and `config.example.toml` entries |
|
||||
|
||||
- [ ] **C7** *(nice-to-have)* Smoke test against the Podman socket
|
||||
- [ ] **C7** _(nice-to-have)_ Smoke test against the Podman socket
|
||||
- [ ] **I6-5** Verification #5 OTel smoke test against a working collector (see the [verification checklist](ARCHITECTURE.md#verification))
|
||||
- [ ] **B-1** Docker log-follow SSE live check
|
||||
- [ ] **B-2** REST, auth and TLS live check against minipc1
|
||||
- [ ] **X-1** *(external)* Fix or investigate `opentelemetry-collector.service` / `var-lib-loki.mount` on minipc1
|
||||
- [ ] **X-1** _(external)_ Fix or investigate `opentelemetry-collector.service` / `var-lib-loki.mount` on minipc1
|
||||
- [ ] **P5-1** minipc1 cross-check: run a consolidator against minipc1 with a read-only token the user mints there (`list_peers`, `get_peer`, one pass-through)
|
||||
- [ ] **P5-2** OTel peer gauges: run `devenv up`, enable `[otel]` with peers, confirm `peer.up`, `peer.api.auth_ok`, `peer.check.duration_seconds`, `peer.last_change_timestamp_seconds` reach the collector, and record the result
|
||||
- [ ] **D8** Fleet aggregation across peers
|
||||
- [ ] **P5-3** Scoped tokens
|
||||
- [ ] **P5-4** Peer health history
|
||||
- [ ] **P5-5** Runtime peer management
|
||||
- [ ] **DOC-1** MCP tools in the Redoc page
|
||||
- [ ] **DOC-2** Config switch for the `/docs` viewer
|
||||
- [ ] **F-1** Cap concurrent follow streams. Each journal follow, `tail_journal` call or docker follow pins a `spawn_blocking` thread for up to `limits.max_follow_secs`. Add a global semaphore shared by REST and MCP (suggested 16) returning 429/503 when full. A client that stops reading may hold the thread past the deadline (untested); add a write timeout or a hard `tokio::time::timeout` around the producer.
|
||||
- [ ] **F-2** Bound line and message length. Docker `LineSplitter` and journal `MESSAGE` are buffered without limit before the 256 KiB result cap applies. Truncate per line (suggested 64 KiB, with a visible marker) and cap total bytes collected.
|
||||
- [ ] **F-3** Unit hardening: `ProtectKernelLogs`, `ProtectHostname`, `RemoveIPC`, `LimitNOFILE`, `TasksMax`, `MemoryMax`, narrower `RestrictAddressFamilies`, `PrivateUsers` (check journal group access). Test under the hardened unit on a clean host before enabling. Document how the service user gets read access to the TLS key.
|
||||
@@ -66,22 +70,22 @@ Parked, not scheduled. Each has a reason. Nothing here blocks the current work.
|
||||
|
||||
Design notes, owned files and logs for each are archived under [`docs/workstreams/`](docs/workstreams/). Task IDs below are stable references.
|
||||
|
||||
| ID | What | Key outcome | Tasks | Completed |
|
||||
|---|---|---|---|---|
|
||||
| [P0](docs/workstreams/P0-foundation.md) | Foundation: devenv, Cargo, config, DTOs, source traits + fakes, errors, `AppState`, stubs | `serve --print-config` works; build, clippy, fmt, test clean; `cargo tree -i openssl` empty | P0-1..P0-11, Gate | merged to `main` |
|
||||
| [WS-A](docs/workstreams/WS-A-systemd.md) | systemd source (zbus) | Units, timers (via `*.timer` + `Timer` props), read-only `GetUnit` status | A1..A7 | merged to `main` |
|
||||
| [WS-B](docs/workstreams/WS-B-journal.md) | journal source (sd-journal) | Snapshot + follow in `spawn_blocking`; follow thread exits on receiver drop | B1..B6 | merged to `main` |
|
||||
| [WS-C](docs/workstreams/WS-C-docker.md) | docker source (bollard) | Lazy socket, bounded concurrent inspect, log streams; missing socket maps to `Unavailable` | C1..C6 (C7 in Backlog) | merged to `main` |
|
||||
| [WS-D](docs/workstreams/WS-D-auth.md) | auth + SQLite token DB | sha256 tokens, `slinuxm token create`, `list`, `revoke`, bearer middleware, 401 + `WWW-Authenticate` | D1..D7 | merged to `main` |
|
||||
| [WS-E](docs/workstreams/WS-E-rest.md) | REST API | All `/api/v1` routes, SSE follow, `/healthz`, 503 on docker-only failures | E1..E9 | merged to `main` |
|
||||
| [WS-F](docs/workstreams/WS-F-mcp.md) | MCP server | 8 tools, 2 prompts, 2 resources, instructions, Origin/Host validation | F1..F9 | merged to `main` |
|
||||
| [WS-G](docs/workstreams/WS-G-otel.md) | OTel export | Traces, observable-gauge metrics, journal/docker log fan-out with reconcile, flush on shutdown, `dev/otelcol.yaml` | G1..G7 | merged to `main` |
|
||||
| [P3](docs/workstreams/P3-integration.md) | Integration and hardening | One axum server, negative tests, hardened unit, README; I6 verified on minipc1 (#1–#4, #6) | I1..I6 (I6-5 in Backlog) | 2026-10-19 |
|
||||
| [WS-I](docs/workstreams/WS-I-tls.md) | Native TLS (rustls) | `server.tls.*`, SIGHUP reload, HTTPS tests, live `curl --cacert` check | T1..T8 | merged to `main` |
|
||||
| [WS-H](docs/workstreams/WS-H-nixos.md) | NixOS package, module, VM test | H1..H8 merged (PR #1, 34da657); gate passed 2026-10-19 | H1..H8 | merged; gate passed 2026-10-19 |
|
||||
| [P5](docs/workstreams/P5-consolidator/README.md) | Consolidator: optional peers (WS-T, A..F) | Declarative peers with TCP and API health checks, `/api/v1/peers` status and GET pass-through, `peer` argument and `list_peers`/`get_peer` (10 MCP tools with peers, 8 without), four OTel gauges, NixOS module option, SSRF guard, peers VM test; peer-less instance unchanged | T1..T7, A0..A4, B1..B5, C1..C4, D1..D3, E1..E2, F1..F4 (P5-1, P5-2 in Backlog) | 2026-10-20 |
|
||||
| [WS-G (OpenAPI)](docs/workstreams/WS-G-openapi.md) | Generated OpenAPI 3.1 doc (stand-alone, after P5) | `utoipa` paths plus schemars bridge, `x-mcp-tools`, `GET /api/v1/openapi.json`, `slinuxm openapi [--peers]`, drift test, `docs/openapi.json` | G1..G4 | 2026-10-20 |
|
||||
| [WS-H (final fixes)](docs/workstreams/WS-H-final-fixes.md) | Phase 5 final-review fixes | One token loader, one set of peer helpers, IPv4-mapped deny check, IPv6 `deny_cidrs`, client span kind, flaky test fixes, docs corrections | review items | 2026-10-20 |
|
||||
| ID | What | Key outcome | Tasks | Completed |
|
||||
| ---------------------------------------------------------- | ----------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ | ------------------------------ |
|
||||
| [P0](docs/workstreams/P0-foundation.md) | Foundation: devenv, Cargo, config, DTOs, source traits + fakes, errors, `AppState`, stubs | `serve --print-config` works; build, clippy, fmt, test clean; `cargo tree -i openssl` empty | P0-1..P0-11, Gate | merged to `main` |
|
||||
| [WS-A](docs/workstreams/WS-A-systemd.md) | systemd source (zbus) | Units, timers (via `*.timer` + `Timer` props), read-only `GetUnit` status | A1..A7 | merged to `main` |
|
||||
| [WS-B](docs/workstreams/WS-B-journal.md) | journal source (sd-journal) | Snapshot + follow in `spawn_blocking`; follow thread exits on receiver drop | B1..B6 | merged to `main` |
|
||||
| [WS-C](docs/workstreams/WS-C-docker.md) | docker source (bollard) | Lazy socket, bounded concurrent inspect, log streams; missing socket maps to `Unavailable` | C1..C6 (C7 in Backlog) | merged to `main` |
|
||||
| [WS-D](docs/workstreams/WS-D-auth.md) | auth + SQLite token DB | sha256 tokens, `slinuxm token create`, `list`, `revoke`, bearer middleware, 401 + `WWW-Authenticate` | D1..D7 | merged to `main` |
|
||||
| [WS-E](docs/workstreams/WS-E-rest.md) | REST API | All `/api/v1` routes, SSE follow, `/healthz`, 503 on docker-only failures | E1..E9 | merged to `main` |
|
||||
| [WS-F](docs/workstreams/WS-F-mcp.md) | MCP server | 8 tools, 2 prompts, 2 resources, instructions, Origin/Host validation | F1..F9 | merged to `main` |
|
||||
| [WS-G](docs/workstreams/WS-G-otel.md) | OTel export | Traces, observable-gauge metrics, journal/docker log fan-out with reconcile, flush on shutdown, `dev/otelcol.yaml` | G1..G7 | merged to `main` |
|
||||
| [P3](docs/workstreams/P3-integration.md) | Integration and hardening | One axum server, negative tests, hardened unit, README; I6 verified on minipc1 (#1–#4, #6) | I1..I6 (I6-5 in Backlog) | 2026-10-19 |
|
||||
| [WS-I](docs/workstreams/WS-I-tls.md) | Native TLS (rustls) | `server.tls.*`, SIGHUP reload, HTTPS tests, live `curl --cacert` check | T1..T8 | merged to `main` |
|
||||
| [WS-H](docs/workstreams/WS-H-nixos.md) | NixOS package, module, VM test | H1..H8 merged (PR #1, 34da657); gate passed 2026-10-19 | H1..H8 | merged; gate passed 2026-10-19 |
|
||||
| [P5](docs/workstreams/P5-consolidator/README.md) | Consolidator: optional peers (WS-T, A..F) | Declarative peers with TCP and API health checks, `/api/v1/peers` status and GET pass-through, `peer` argument and `list_peers`/`get_peer` (10 MCP tools with peers, 8 without), four OTel gauges, NixOS module option, SSRF guard, peers VM test; peer-less instance unchanged | T1..T7, A0..A4, B1..B5, C1..C4, D1..D3, E1..E2, F1..F4 (P5-1, P5-2 in Backlog) | 2026-10-20 |
|
||||
| [WS-G (OpenAPI)](docs/workstreams/WS-G-openapi.md) | Generated OpenAPI 3.1 doc (stand-alone, after P5) | `utoipa` paths plus schemars bridge, `x-mcp-tools`, `GET /api/v1/openapi.json`, `slinuxm openapi [--peers]`, drift test, `docs/openapi.json` | G1..G4 | 2026-10-20 |
|
||||
| [WS-H (final fixes)](docs/workstreams/WS-H-final-fixes.md) | Phase 5 final-review fixes | One token loader, one set of peer helpers, IPv4-mapped deny check, IPv6 `deny_cidrs`, client span kind, flaky test fixes, docs corrections | review items | 2026-10-20 |
|
||||
|
||||
### Done (archive)
|
||||
|
||||
|
||||
Reference in new issue
Block a user