105 lines
11 KiB
Markdown
105 lines
11 KiB
Markdown
# TODO — skyg-linux-tracker
|
||
|
||
The single place for work tracking. **Tick boxes here only.** Design lives in [`ARCHITECTURE.md`](ARCHITECTURE.md). Archived workstream design notes and logs are in [`docs/workstreams/`](docs/workstreams/). Plans for the next big thing go in [`plan/`](plan/README.md).
|
||
|
||
_Last updated: 2026-10-19_
|
||
|
||
## Status
|
||
|
||
The build is functionally complete and merged to `main`. REST, MCP, OTel, TLS and the NixOS module are implemented. Verification #1–#4 and #6 passed live on minipc1 (2026-10-19). The WS-H nix gate passed on 2026-10-19. Phase 5 (consolidator) is planned and starting; everything else open is in the Backlog.
|
||
|
||
| Milestone | Gate | Status |
|
||
|---|---|---|
|
||
| M0 Skeleton | `slinuxm serve --print-config` works. Clippy and tests are clean. No openssl. | Done |
|
||
| M1 Sources live | All four Phase 1 examples run against a real host | Done (systemd, journal, docker verified live) |
|
||
| M2 Surfaces | Verification #3, #4 and #5 pass | Partial: #3 and #4 pass live (minipc1, 2026-10-19, all 8 MCP tools); #5 OTel deferred to I6-5 (collector on minipc1 is down) |
|
||
| M3 Release | Verification #1–#6 pass. Unit file and README shipped. | Partial: #1–#4 and #6 pass, unit installed on minipc1; needs #5 (I6-5) |
|
||
| M4 NixOS | `nix flake check` passes including the VM test. `:1984` serves and a token is minted with the system-installed CLI. | ✅ 2026-10-19: `nix build` and `nix flake check` (incl. VM test, plain + TLS) pass; see [WS-H log](docs/workstreams/WS-H-nixos.md#log) |
|
||
| M5 TLS | HTTPS serves REST and `/mcp`, SIGHUP reloads certs, no openssl. | Done |
|
||
|
||
## Phase 5: Consolidator (in progress)
|
||
|
||
Plan: [`plan/phase-5-consolidator/`](plan/phase-5-consolidator/README.md). Decisions accepted 2026-10-19 (DEC-13, DEC-14). Work loop: `.claude/skills/phase-loop`. Task text lives in each `ws-*.md`; tick here only.
|
||
|
||
| Milestone | Gate | Status |
|
||
|---|---|---|
|
||
| M6 Consolidator | Phase 5 exit criteria; peer-less instance unchanged (8 MCP tools); `nix flake check` incl. the peers VM test | Not started |
|
||
|
||
- [ ] **WS-T** tests: T1 T2 T3 T4 T5 T6 (peers.nix written) · [x] T7 `dev/test.nu` runner
|
||
- [x] T1..T6 written in `nix/tests/peers.nix`; promoted to `checks` by F2
|
||
- [ ] **WS-A** config: [x] A0 stubs · [x] A1 config · [x] A2 validation · [x] A3 secrets loader · [x] A4 `peer` CLI
|
||
- [ ] **WS-B** registry: [x] B1 client · [x] B2 health loops · [x] B3 status model · [x] B4 SIGHUP reload · [x] B5 transport + SSRF guard
|
||
- [ ] **WS-C** REST: [x] C1 status routes · [x] C2 pass-through · [x] C3 `x-slinuxm-peer` header · [x] C4 error mapping
|
||
- [ ] **WS-D** MCP: [x] D1 `peer` arg · [x] D2 `list_peers`/`get_peer` · [x] D3 instructions
|
||
- [ ] **WS-E** telemetry: [x] E1 gauges · [x] E2 client spans and propagation
|
||
- [ ] **WS-F** NixOS and docs: [ ] F1 module · [ ] F2 promote VM test · [ ] F3 `tests/peers.rs` · [ ] F4 README, `docs/consolidator.md`
|
||
- [ ] **Final review** and fold-in: move `plan/phase-5-consolidator` to `docs/workstreams/`, collapse into the Completed table, add Backlog entries (D8 fleet aggregation, scoped tokens, health history, runtime peer management)
|
||
|
||
## In progress / Review
|
||
|
||
| Item | Owner | State |
|
||
|---|---|---|
|
||
| — | | Nothing in flight |
|
||
|
||
- [x] **WS-H Gate:** `nix build` and `nix flake check` pass. `ldd` shows no openssl. A fresh NixOS host serves `:1984` and a token can be minted from a login shell.
|
||
|
||
## Backlog
|
||
|
||
Parked, not scheduled. Each has a reason. Nothing here blocks the current work. Tick here when picked up and finished.
|
||
|
||
| ID | Item | Reason parked |
|
||
|---|---|---|
|
||
| **C7** | *(optional)* Smoke test against the Podman socket (WS-C) | Nice-to-have; no Podman host in use |
|
||
| **I6-5** | Verification #5: OTel smoke test end to end (P3) | Waiting on the minipc1 collector fix (**X-1**); `slinuxm` itself exports fine |
|
||
| **B-1** | Live check of docker log-follow SSE (`/docker/containers/{name}/logs?follow=1`) | Not exercised in the live runs so far |
|
||
| **B-2** | Live check of REST, bearer auth and TLS against minipc1 | The MCP-tool run did not cover them |
|
||
| **X-1** | *(external)* Investigate failed `opentelemetry-collector.service` and `var-lib-loki.mount` on minipc1 | Outside this repo; blocks I6-5 |
|
||
| **F-1** | Cap concurrent follow streams | Security review 2026-10-16; not scheduled |
|
||
| **F-2** | Bound line and message length | Security review 2026-10-16; not scheduled |
|
||
| **F-3** | Unit hardening extras | Security review 2026-10-16; needs a clean-host test first |
|
||
|
||
- [ ] **C7** *(nice-to-have)* Smoke test against the Podman socket
|
||
- [ ] **I6-5** Verification #5 OTel smoke test against a working collector (see the [verification checklist](ARCHITECTURE.md#verification))
|
||
- [ ] **B-1** Docker log-follow SSE live check
|
||
- [ ] **B-2** REST, auth and TLS live check against minipc1
|
||
- [ ] **X-1** *(external)* Fix or investigate `opentelemetry-collector.service` / `var-lib-loki.mount` on minipc1
|
||
- [ ] **F-1** Cap concurrent follow streams. Each journal follow, `tail_journal` call or docker follow pins a `spawn_blocking` thread for up to `limits.max_follow_secs`. Add a global semaphore shared by REST and MCP (suggested 16) returning 429/503 when full. A client that stops reading may hold the thread past the deadline (untested); add a write timeout or a hard `tokio::time::timeout` around the producer.
|
||
- [ ] **F-2** Bound line and message length. Docker `LineSplitter` and journal `MESSAGE` are buffered without limit before the 256 KiB result cap applies. Truncate per line (suggested 64 KiB, with a visible marker) and cap total bytes collected.
|
||
- [ ] **F-3** Unit hardening: `ProtectKernelLogs`, `ProtectHostname`, `RemoveIPC`, `LimitNOFILE`, `TasksMax`, `MemoryMax`, narrower `RestrictAddressFamilies`, `PrivateUsers` (check journal group access). Test under the hardened unit on a clean host before enabling. Document how the service user gets read access to the TLS key.
|
||
|
||
## Completed workstreams
|
||
|
||
Design notes, owned files and logs for each are archived under [`docs/workstreams/`](docs/workstreams/). Task IDs below are stable references.
|
||
|
||
| ID | What | Key outcome | Tasks | Completed |
|
||
|---|---|---|---|---|
|
||
| [P0](docs/workstreams/P0-foundation.md) | Foundation: devenv, Cargo, config, DTOs, source traits + fakes, errors, `AppState`, stubs | `serve --print-config` works; build, clippy, fmt, test clean; `cargo tree -i openssl` empty | P0-1..P0-11, Gate | merged to `main` |
|
||
| [WS-A](docs/workstreams/WS-A-systemd.md) | systemd source (zbus) | Units, timers (via `*.timer` + `Timer` props), read-only `GetUnit` status | A1..A7 | merged to `main` |
|
||
| [WS-B](docs/workstreams/WS-B-journal.md) | journal source (sd-journal) | Snapshot + follow in `spawn_blocking`; follow thread exits on receiver drop | B1..B6 | merged to `main` |
|
||
| [WS-C](docs/workstreams/WS-C-docker.md) | docker source (bollard) | Lazy socket, bounded concurrent inspect, log streams; missing socket maps to `Unavailable` | C1..C6 (C7 in Backlog) | merged to `main` |
|
||
| [WS-D](docs/workstreams/WS-D-auth.md) | auth + SQLite token DB | sha256 tokens, `slinuxm token create`, `list`, `revoke`, bearer middleware, 401 + `WWW-Authenticate` | D1..D7 | merged to `main` |
|
||
| [WS-E](docs/workstreams/WS-E-rest.md) | REST API | All `/api/v1` routes, SSE follow, `/healthz`, 503 on docker-only failures | E1..E9 | merged to `main` |
|
||
| [WS-F](docs/workstreams/WS-F-mcp.md) | MCP server | 8 tools, 2 prompts, 2 resources, instructions, Origin/Host validation | F1..F9 | merged to `main` |
|
||
| [WS-G](docs/workstreams/WS-G-otel.md) | OTel export | Traces, observable-gauge metrics, journal/docker log fan-out with reconcile, flush on shutdown, `dev/otelcol.yaml` | G1..G7 | merged to `main` |
|
||
| [P3](docs/workstreams/P3-integration.md) | Integration and hardening | One axum server, negative tests, hardened unit, README; I6 verified on minipc1 (#1–#4, #6) | I1..I6 (I6-5 in Backlog) | 2026-10-19 |
|
||
| [WS-I](docs/workstreams/WS-I-tls.md) | Native TLS (rustls) | `server.tls.*`, SIGHUP reload, HTTPS tests, live `curl --cacert` check | T1..T8 | merged to `main` |
|
||
| [WS-H](docs/workstreams/WS-H-nixos.md) | NixOS package, module, VM test | H1..H8 merged (PR #1, 34da657); gate passed 2026-10-19 | H1..H8 | merged; gate passed 2026-10-19 |
|
||
|
||
### Done (archive)
|
||
|
||
All ticked. One line per workstream; the full task text is on each card.
|
||
|
||
- **P0** — devenv scripts and libsystemd (P0-1); `Cargo.toml` with `slinuxm` bin, rustls-only, LTO+strip (P0-2); API spike (P0-3); figment config with `otel.headers`, `otel.metrics.*`, `limits.*` and validation (P0-4); `config.example.toml` (P0-5); DTOs with serde/schemars (P0-6); source traits + fakes (P0-7); `AppError` mapping (P0-8); `AppState` (P0-9); stubs and `main.rs` dispatch (P0-10); config tests (P0-11); gate.
|
||
- **WS-A** — bus connection (A1); `list_units` (A2); `list_timers` (A3); `unit_status` via `GetUnit` (A4); name validation, `NoSuchUnit` to NotFound (A5); `examples/systemd_dump.rs` (A6); mapping tests (A7).
|
||
- **WS-B** — `spawn_blocking` access (B1); `query` (B2); `follow` (B3); lossy UTF-8 mapping (B4); `examples/journal_tail.rs` (B5); follow-exit test (B6).
|
||
- **WS-C** — lazy bollard client (C1); `ps` (C2); `container_status` (C3); `logs` stream (C4); `tail` cap (C5); `examples/docker_ps.rs` (C6).
|
||
- **WS-D** — `db.rs` open-or-create, WAL (D1); tokens migration (D2); token CRUD (D3); CLI (D4); bearer middleware (D5); protected-router layer, `/healthz` public (D6); tests (D7).
|
||
- **WS-E** — router with `{name}` syntax (E1); systemd routes (E2); journal snapshot (E3); journal SSE (E4); docker routes (E5); `/healthz` (E6); JSON errors (E7); handler tests (E8); docker 503 (E9).
|
||
- **WS-F** — `TrackerService`, 8 tools (F1); `tail_journal` cap (F2); `get_container_logs` cap (F3); `get_info` instructions (F4); tool descriptions (F5); prompts (F6); resources (F7); `mcp::service` with Origin validation (F8); tests (F9).
|
||
- **WS-G** — `telemetry::init` (G1); tracing subscriber (G2); snapshot + observable gauges (G3); metric set and allowlists (G4); log export with reconcile (G5); shutdown flush (G6); `dev/otelcol.yaml` (G7).
|
||
- **P3** — serve wiring and shutdown (I1); negative tests (I2); systemd unit, verified with `systemd-analyze security --offline` (1.5 OK) and installed as a system unit on minipc1, 2026-10-19 (I3); README (I4); release build, `ldd` clean (I5); manual Verification #1–#4 and #6 on minipc1, 2026-10-19; #5 deferred as I6-5 (I6).
|
||
- **WS-I** — `server.tls.*` config (T1); rustls with pinned provider (T2); SIGHUP reload (T3); `https://` MCP origins (T4); non-loopback-without-TLS warning (T5); self-signed cert tests (T6); README TLS section (T7); live check (T8).
|
||
- **WS-H** — `nix/package.nix` (H1); `flake.nix` (H2); module options (H3); service definition (H4); system-wide CLI with root guard (H5); firewall options and eval assertion (H6); README NixOS section (H7); VM test (H8). Gate passed 2026-10-19 (build, flake check, VM test).
|
||
|
||
Early decisions (DEC-1..DEC-8) were accepted before work started; all decisions are recorded in [`ARCHITECTURE.md`](ARCHITECTURE.md#decisions).
|