Fold Phase 5 into docs
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
6be774e98b
commit
f5e5f752e3
17 files changed
+71
-43
No files matched your search
@@ -2,11 +2,11 @@
|
||||
|
||||
The single place for work tracking. **Tick boxes here only.** Design lives in [`ARCHITECTURE.md`](ARCHITECTURE.md). Archived workstream design notes and logs are in [`docs/workstreams/`](docs/workstreams/). Plans for the next big thing go in [`plan/`](plan/README.md).
|
||||
|
||||
_Last updated: 2026-10-19_
|
||||
_Last updated: 2026-10-20_
|
||||
|
||||
## Status
|
||||
|
||||
The build is functionally complete and merged to `main`. REST, MCP, OTel, TLS and the NixOS module are implemented. Verification #1–#4 and #6 passed live on minipc1 (2026-10-19). The WS-H nix gate passed on 2026-10-19. Phase 5 (consolidator) is planned and starting; everything else open is in the Backlog.
|
||||
The build is functionally complete and merged to `main`. REST, MCP, OTel, TLS and the NixOS module are implemented. Verification #1–#4 and #6 passed live on minipc1 (2026-10-19). The WS-H nix gate passed on 2026-10-19. Phase 5 (consolidator, optional peers) and the generated OpenAPI doc are merged; the Phase 5 record is archived in [`docs/workstreams/P5-consolidator/`](docs/workstreams/P5-consolidator/README.md). Everything open is in the Backlog.
|
||||
|
||||
| Milestone | Gate | Status |
|
||||
|---|---|---|
|
||||
@@ -16,25 +16,7 @@ The build is functionally complete and merged to `main`. REST, MCP, OTel, TLS an
|
||||
| M3 Release | Verification #1–#6 pass. Unit file and README shipped. | Partial: #1–#4 and #6 pass, unit installed on minipc1; needs #5 (I6-5) |
|
||||
| M4 NixOS | `nix flake check` passes including the VM test. `:1984` serves and a token is minted with the system-installed CLI. | ✅ 2026-10-19: `nix build` and `nix flake check` (incl. VM test, plain + TLS) pass; see [WS-H log](docs/workstreams/WS-H-nixos.md#log) |
|
||||
| M5 TLS | HTTPS serves REST and `/mcp`, SIGHUP reloads certs, no openssl. | Done |
|
||||
|
||||
## Phase 5: Consolidator (in progress)
|
||||
|
||||
Plan: [`plan/phase-5-consolidator/`](plan/phase-5-consolidator/README.md). Decisions accepted 2026-10-19 (DEC-13, DEC-14). Work loop: `.claude/skills/phase-loop`. Task text lives in each `ws-*.md`; tick here only.
|
||||
|
||||
| Milestone | Gate | Status |
|
||||
|---|---|---|
|
||||
| M6 Consolidator | Phase 5 exit criteria; peer-less instance unchanged (8 MCP tools); `nix flake check` incl. the peers VM test | Not started |
|
||||
|
||||
- [ ] **WS-T** tests: T1 T2 T3 T4 T5 T6 (peers.nix written) · [x] T7 `dev/test.nu` runner
|
||||
- [x] T1..T6 written in `nix/tests/peers.nix`; promoted to `checks` by F2
|
||||
- [ ] **WS-A** config: [x] A0 stubs · [x] A1 config · [x] A2 validation · [x] A3 secrets loader · [x] A4 `peer` CLI
|
||||
- [ ] **WS-B** registry: [x] B1 client · [x] B2 health loops · [x] B3 status model · [x] B4 SIGHUP reload · [x] B5 transport + SSRF guard
|
||||
- [ ] **WS-C** REST: [x] C1 status routes · [x] C2 pass-through · [x] C3 `x-slinuxm-peer` header · [x] C4 error mapping
|
||||
- [ ] **WS-D** MCP: [x] D1 `peer` arg · [x] D2 `list_peers`/`get_peer` · [x] D3 instructions
|
||||
- [ ] **WS-E** telemetry: [x] E1 gauges · [x] E2 client spans and propagation
|
||||
- [x] **WS-F** NixOS and docs: [x] F1 module · [x] F2 promote VM test · [x] F3 `tests/peers.rs` · [x] F4 README, `docs/consolidator.md`
|
||||
- [x] **WS-G** (stand-alone) generated OpenAPI doc for REST and MCP: [x] G1 REST spec · [x] G2 MCP `x-mcp-tools` · [x] G3 `/api/v1/openapi.json` + `slinuxm openapi` · [x] G4 drift guard, `docs/openapi.json`
|
||||
- [ ] **Final review** and fold-in: move `plan/phase-5-consolidator` to `docs/workstreams/`, collapse into the Completed table, add Backlog entries (D8 fleet aggregation, scoped tokens, health history, runtime peer management)
|
||||
| M6 Consolidator | Phase 5 exit criteria; peer-less instance unchanged (8 MCP tools); `nix flake check` incl. the peers VM test | Met, except two live checks still open: **P5-1** minipc1 cross-check needs a user-minted token, **P5-2** OTel peer-gauge check via `devenv up` not yet recorded (both in the Backlog) |
|
||||
|
||||
## In progress / Review
|
||||
|
||||
@@ -58,12 +40,24 @@ Parked, not scheduled. Each has a reason. Nothing here blocks the current work.
|
||||
| **F-1** | Cap concurrent follow streams | Security review 2026-10-16; not scheduled |
|
||||
| **F-2** | Bound line and message length | Security review 2026-10-16; not scheduled |
|
||||
| **F-3** | Unit hardening extras | Security review 2026-10-16; needs a clean-host test first |
|
||||
| **P5-1** | minipc1 cross-check of the consolidator against a real peer | Needs a read-only token minted by the user on minipc1; agents never mint tokens on real hosts |
|
||||
| **P5-2** | Record the OTel peer-gauge check (`devenv up`, Verification step 5 with peers) | Not yet run and recorded; no automated test asserts the exported peer metrics end to end |
|
||||
| **D8** | Fleet aggregation (merged views across peers) | Out of scope for v1 (DEC-14: no aggregation, no chaining); needs a design for partial failure and ordering |
|
||||
| **P5-3** | Scoped tokens (per-peer or read-class scopes) | Any valid inbound token may read peer data (DEC-14); scopes need a token schema change and a policy decision |
|
||||
| **P5-4** | Peer health history | State is in memory only by design; history needs storage or an external TSDB, and the OTel gauges already cover it |
|
||||
| **P5-5** | Runtime peer management (add/remove without editing config) | Peers are declarative so no write path or DB table exists (DEC-14); would need a write path against the read-only rule |
|
||||
|
||||
- [ ] **C7** *(nice-to-have)* Smoke test against the Podman socket
|
||||
- [ ] **I6-5** Verification #5 OTel smoke test against a working collector (see the [verification checklist](ARCHITECTURE.md#verification))
|
||||
- [ ] **B-1** Docker log-follow SSE live check
|
||||
- [ ] **B-2** REST, auth and TLS live check against minipc1
|
||||
- [ ] **X-1** *(external)* Fix or investigate `opentelemetry-collector.service` / `var-lib-loki.mount` on minipc1
|
||||
- [ ] **P5-1** minipc1 cross-check: run a consolidator against minipc1 with a read-only token the user mints there (`list_peers`, `get_peer`, one pass-through)
|
||||
- [ ] **P5-2** OTel peer gauges: run `devenv up`, enable `[otel]` with peers, confirm `peer.up`, `peer.api.auth_ok`, `peer.check.duration_seconds`, `peer.last_change_timestamp_seconds` reach the collector, and record the result
|
||||
- [ ] **D8** Fleet aggregation across peers
|
||||
- [ ] **P5-3** Scoped tokens
|
||||
- [ ] **P5-4** Peer health history
|
||||
- [ ] **P5-5** Runtime peer management
|
||||
- [ ] **F-1** Cap concurrent follow streams. Each journal follow, `tail_journal` call or docker follow pins a `spawn_blocking` thread for up to `limits.max_follow_secs`. Add a global semaphore shared by REST and MCP (suggested 16) returning 429/503 when full. A client that stops reading may hold the thread past the deadline (untested); add a write timeout or a hard `tokio::time::timeout` around the producer.
|
||||
- [ ] **F-2** Bound line and message length. Docker `LineSplitter` and journal `MESSAGE` are buffered without limit before the 256 KiB result cap applies. Truncate per line (suggested 64 KiB, with a visible marker) and cap total bytes collected.
|
||||
- [ ] **F-3** Unit hardening: `ProtectKernelLogs`, `ProtectHostname`, `RemoveIPC`, `LimitNOFILE`, `TasksMax`, `MemoryMax`, narrower `RestrictAddressFamilies`, `PrivateUsers` (check journal group access). Test under the hardened unit on a clean host before enabling. Document how the service user gets read access to the TLS key.
|
||||
@@ -85,6 +79,9 @@ Design notes, owned files and logs for each are archived under [`docs/workstream
|
||||
| [P3](docs/workstreams/P3-integration.md) | Integration and hardening | One axum server, negative tests, hardened unit, README; I6 verified on minipc1 (#1–#4, #6) | I1..I6 (I6-5 in Backlog) | 2026-10-19 |
|
||||
| [WS-I](docs/workstreams/WS-I-tls.md) | Native TLS (rustls) | `server.tls.*`, SIGHUP reload, HTTPS tests, live `curl --cacert` check | T1..T8 | merged to `main` |
|
||||
| [WS-H](docs/workstreams/WS-H-nixos.md) | NixOS package, module, VM test | H1..H8 merged (PR #1, 34da657); gate passed 2026-10-19 | H1..H8 | merged; gate passed 2026-10-19 |
|
||||
| [P5](docs/workstreams/P5-consolidator/README.md) | Consolidator: optional peers (WS-T, A..F) | Declarative peers with TCP and API health checks, `/api/v1/peers` status and GET pass-through, `peer` argument and `list_peers`/`get_peer` (10 MCP tools with peers, 8 without), four OTel gauges, NixOS module option, SSRF guard, peers VM test; peer-less instance unchanged | T1..T7, A0..A4, B1..B5, C1..C4, D1..D3, E1..E2, F1..F4 (P5-1, P5-2 in Backlog) | 2026-10-20 |
|
||||
| [WS-G (OpenAPI)](docs/workstreams/WS-G-openapi.md) | Generated OpenAPI 3.1 doc (stand-alone, after P5) | `utoipa` paths plus schemars bridge, `x-mcp-tools`, `GET /api/v1/openapi.json`, `slinuxm openapi [--peers]`, drift test, `docs/openapi.json` | G1..G4 | 2026-10-20 |
|
||||
| [WS-H (final fixes)](docs/workstreams/WS-H-final-fixes.md) | Phase 5 final-review fixes | One token loader, one set of peer helpers, IPv4-mapped deny check, IPv6 `deny_cidrs`, client span kind, flaky test fixes, docs corrections | review items | 2026-10-20 |
|
||||
|
||||
### Done (archive)
|
||||
|
||||
@@ -101,5 +98,8 @@ All ticked. One line per workstream; the full task text is on each card.
|
||||
- **P3** — serve wiring and shutdown (I1); negative tests (I2); systemd unit, verified with `systemd-analyze security --offline` (1.5 OK) and installed as a system unit on minipc1, 2026-10-19 (I3); README (I4); release build, `ldd` clean (I5); manual Verification #1–#4 and #6 on minipc1, 2026-10-19; #5 deferred as I6-5 (I6).
|
||||
- **WS-I** — `server.tls.*` config (T1); rustls with pinned provider (T2); SIGHUP reload (T3); `https://` MCP origins (T4); non-loopback-without-TLS warning (T5); self-signed cert tests (T6); README TLS section (T7); live check (T8).
|
||||
- **WS-H** — `nix/package.nix` (H1); `flake.nix` (H2); module options (H3); service definition (H4); system-wide CLI with root guard (H5); firewall options and eval assertion (H6); README NixOS section (H7); VM test (H8). Gate passed 2026-10-19 (build, flake check, VM test).
|
||||
- **P5** — runner `dev/dev-test.nu` and peers VM test (WS-T); config, validation, secrets loader, `peer` CLI (WS-A); registry, health loops, status model, SIGHUP reload, transport with SSRF guard (WS-B); status routes, pass-through, `x-slinuxm-peer`, error mapping (WS-C); `peer` argument, `list_peers`/`get_peer`, instructions (WS-D); gauges, client spans and propagation (WS-E); NixOS module, promoted VM test, `tests/peers.rs`, README and `docs/consolidator.md` (WS-F).
|
||||
- **WS-G (OpenAPI)** — REST spec (G1); MCP `x-mcp-tools` (G2); route and `slinuxm openapi` (G3); drift guard and `docs/openapi.json` (G4).
|
||||
- **WS-H (final fixes)** — see the [record](docs/workstreams/WS-H-final-fixes.md).
|
||||
|
||||
Early decisions (DEC-1..DEC-8) were accepted before work started; all decisions are recorded in [`ARCHITECTURE.md`](ARCHITECTURE.md#decisions).
|
||||
Reference in new issue
Block a user