initial setup

This commit is contained in:
ari committed 2026-10-15 17:07:03 -04:00
1 parent 58ebcaa97f
commit 389c4f998a
19 files changed
+1206 -40

No files matched your search

+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
eval "$(devenv direnvrc)"
use devenv
+3
View File
@@ -8,3 +8,6 @@ devenv.local.yaml
# pre-commit
.pre-commit-config.yaml
.commandcode
.claude-account
+104
View File
@@ -0,0 +1,104 @@
# TODO — skyg-linux-tracker
Master checklist. **Tick boxes here only.** Card status lives on [`plan/BOARD.md`](plan/BOARD.md). Scope, owned files and design notes for each workstream live in [`plan/workstreams/`](plan/workstreams/).
Design: [`plan.md`](plan.md) · Review fixes: [`plan/review.md`](plan/review.md) · Open questions: [`plan/decisions.md`](plan/decisions.md)
## Before starting
- [x] Resolve **DEC-1**: binary and CLI are `slinuxm`
- [ ] Accept or override the proposed decisions DEC-2 to DEC-7
## Phase 0 — Foundation · [card](plan/workstreams/P0-foundation.md)
- [ ] **P0-1** devenv: `pkg-config`, `systemd` (libsystemd.pc), `build`/`run`/`token` scripts. No openssl. `pkg-config --libs libsystemd` works in the shell.
- [ ] **P0-2** `Cargo.toml` with `[[bin]] name = "slinuxm"` and every dependency pinned for all phases. rustls-only features. Release profile set to lto + strip.
- [ ] **P0-3** Time-boxed API spike to confirm rmcp, zbus_systemd, systemd-journal and bollard APIs. Findings go in the card. Update DEC-8.
- [ ] **P0-4** `config.rs`: figment layering (defaults < TOML < `SKYG_*` env with `__` nesting < CLI). Add `otel.headers`, `otel.metrics.*` allowlists and `limits.*`. Validate the auth-disabled + non-loopback combination.
- [ ] **P0-5** `config.example.toml` covering every key, with comments
- [ ] **P0-6** Shared DTOs in `src/sources/mod.rs` with serde + schemars derives and RFC3339 timestamps. Service-only fields are `Option`.
- [ ] **P0-7** Source traits plus in-memory fakes (`src/sources/fake.rs`)
- [ ] **P0-8** `error.rs`: `AppError` mapped to HTTP status + JSON body and to MCP `ErrorData`
- [ ] **P0-9** `src/state.rs`: `AppState` (sources, optional DB pool, config, `CancellationToken`)
- [ ] **P0-10** Stub every module with its final signatures. `main.rs` dispatches `serve` / `token` / `--print-config` to them.
- [ ] **P0-11** Unit tests for config precedence and validation
- [ ] **Gate:** `slinuxm serve --print-config` prints effective config. Build, clippy, fmt and test are clean. `cargo tree -i openssl` is empty.
## Phase 1 — Sources + auth (parallel)
### WS-A systemd · [card](plan/workstreams/WS-A-systemd.md)
- [ ] **A1** Shared system-bus `zbus::Connection` + `ManagerProxy`
- [ ] **A2** `list_units(state)` mapped to `UnitSummary`
- [ ] **A3** `list_timers()` via `*.timer` units + `TimerProxy` props (no `ListTimers` method exists)
- [ ] **A4** `unit_status(name)`: `GetUnit` first, `LoadUnit` as fallback. `Unit` props always, `Service` props for `.service` only.
- [ ] **A5** Unit-name validation. `NoSuchUnit` maps to `NotFound`.
- [ ] **A6** `examples/systemd_dump.rs` prints JSON from the live bus
- [ ] **A7** Unit tests for timestamp conversion and property mapping
### WS-B journal · [card](plan/workstreams/WS-B-journal.md)
- [ ] **B1** All sd-journal access inside `spawn_blocking`. Optional namespace, depending on the spike.
- [ ] **B2** `query(filter)`: unit, priority ≤ N, `lines` capped by `limits.max_lines`, returned in chronological order
- [ ] **B3** `follow(filter)` returns an mpsc receiver. `wait()` with a timeout. Exits on receiver drop or shutdown.
- [ ] **B4** Entry mapping with lossy UTF-8 for binary `MESSAGE`
- [ ] **B5** `examples/journal_tail.rs`
- [ ] **B6** Test: the follow thread exits after its receiver is dropped
### WS-C docker · [card](plan/workstreams/WS-C-docker.md)
- [ ] **C1** Lazy `bollard::Docker` on the configured socket. Missing or denied socket maps to `Unavailable`.
- [ ] **C2** `ps()`: list + concurrent bounded inspect (health, uptime, restarts, image)
- [ ] **C3** `container_status(name)`
- [ ] **C4** `logs(name, follow, tail, since)` stream that stops on drop or shutdown
- [ ] **C5** `tail` capped by config
- [ ] **C6** `examples/docker_ps.rs` + log follow
- [ ] **C7** *(nice-to-have)* Smoke test against the Podman socket
### WS-D auth + DB · [card](plan/workstreams/WS-D-auth.md)
- [ ] **D1** `db.rs`: open-or-create, dir `0700`, WAL, `busy_timeout`, embedded migrations
- [ ] **D2** Migration for `tokens(id, name UNIQUE, token_hash UNIQUE, created_at, revoked_at, last_used_at)`
- [ ] **D3** `tokens.rs`: create (32 random bytes, `slinuxm_` prefix, printed once), list, revoke. Uses runtime queries.
- [ ] **D4** `slinuxm token create|list|revoke` CLI wired via the Phase 0 stub
- [ ] **D5** Bearer middleware: sha256 + indexed lookup of non-revoked tokens. 401 + `WWW-Authenticate` JSON on failure.
- [ ] **D6** Middleware exposed as a layer for the protected router only. `/healthz` stays public.
- [ ] **D7** Tests: create, accept, revoke, reject. Unknown token is rejected. Startup refuses auth-off + non-loopback.
## Phase 2 — Surfaces (parallel)
### WS-E REST API · [card](plan/workstreams/WS-E-rest.md)
- [ ] **E1** `api::router(state)` using axum 0.8 `{name}` path syntax
- [ ] **E2** `GET /systemd/units?state=`, `/systemd/units/{name}`, `/systemd/timers`
- [ ] **E3** `GET /journal?unit=&priority=&lines=` with validation
- [ ] **E4** `GET /journal/follow` SSE with keep-alive. Ends on disconnect or shutdown.
- [ ] **E5** `GET /docker/ps`, `/docker/containers/{name}`, `/docker/containers/{name}/logs` (JSON, or SSE with `follow=1`)
- [ ] **E6** `GET /healthz` on the public router
- [ ] **E7** Consistent JSON error body from `AppError`
- [ ] **E8** Handler tests against fake sources (`tower::ServiceExt::oneshot`)
- [ ] **E9** Docker unavailable returns 503 on docker routes only
### WS-F MCP · [card](plan/workstreams/WS-F-mcp.md)
- [ ] **F1** `TrackerService` with `#[tool_router]` and 8 tools
- [ ] **F2** `tail_journal` capped by `limits.max_follow_secs` and an entry cap
- [ ] **F3** `get_container_logs` tail cap
- [ ] **F4** `ServerHandler::get_info`: instructions + capabilities (tools, prompts, resources)
- [ ] **F5** Tool descriptions that explain systemd states, health statuses and priorities
- [ ] **F6** Prompts `diagnose_unit { name }` and `container_health_report {}`
- [ ] **F7** Resources `systemd://units/failed` and `docker://containers`, read live, no subscriptions
- [ ] **F8** `mcp::service(state)` returns `StreamableHttpService` with Origin validation (DEC-8)
- [ ] **F9** Tests: tool list snapshot + each tool against fakes
### WS-G OTel · [card](plan/workstreams/WS-G-otel.md)
- [ ] **G1** `telemetry::init(cfg)`: OTLP grpc/http with headers. Resource has `service.name`, `service.version`, `host.name`. Returns a guard.
- [ ] **G2** `tracing` subscriber: fmt to stderr + `tracing-opentelemetry` layer when enabled
- [ ] **G3** Metrics poll loop feeds a snapshot. Observable gauges read it, so vanished series drop out.
- [ ] **G4** Metric set from `plan.md` with semconv attribute names, allowlists and the DEC-7 default scope
- [ ] **G5** Log export: journal follow + per-container docker follow, reconciled via the Docker events stream, with backoff
- [ ] **G6** Shutdown cancels tasks and flushes all providers
- [ ] **G7** `dev/otelcol.yaml` with a debug exporter, plus a devenv process to run it
## Phase 3 — Integration & hardening · [card](plan/workstreams/P3-integration.md)
- [ ] **I1** `main.rs` wiring: protected router (`/api/v1` + `/mcp` + auth) merged with the public `/healthz`, `TraceLayer`, SIGTERM/SIGINT leading to cancel, drain and OTel flush
- [ ] **I2** Automated negative tests (Verification #6)
- [ ] **I3** `systemd/skyg-linux-tracker.service` running `slinuxm serve`: `StateDirectory=`, `ProtectSystem=strict`, `NoNewPrivileges`, `SupplementaryGroups=systemd-journal`, docker group commented (DEC-6)
- [ ] **I4** README: install, config reference, token workflow, MCP client setup, hardened docker option
- [ ] **I5** Release build: `ldd` shows only libc + libsystemd and its deps. Record binary size.
- [ ] **I6** Full manual Verification #1–#6 from `plan.md` passes on a real host
+65
View File
@@ -0,0 +1,65 @@
{
"nodes": {
"devenv": {
"locked": {
"dir": "src/modules",
"lastModified": 1792071910,
"narHash": "sha256-DUCfod12b6e/r9FCjE3+x7mJlVEmIpC10BZKGz+Nrsw=",
"owner": "cachix",
"repo": "devenv",
"rev": "1566050ec14ce238983376e118af9fb220454d39",
"type": "github"
},
"original": {
"dir": "src/modules",
"owner": "cachix",
"repo": "devenv",
"type": "github"
}
},
"nixpkgs": {
"inputs": {
"nixpkgs-src": "nixpkgs-src"
},
"locked": {
"lastModified": 1791812882,
"narHash": "sha256-4AdYH2HoD21mys5lk9NoDcM2Z7iPeEUjxJeGE3Pw278=",
"owner": "cachix",
"repo": "devenv-nixpkgs",
"rev": "760b85220489f69855e6df78a6d1dad5873f8bf6",
"type": "github"
},
"original": {
"owner": "cachix",
"ref": "rolling",
"repo": "devenv-nixpkgs",
"type": "github"
}
},
"nixpkgs-src": {
"flake": false,
"locked": {
"lastModified": 1791487292,
"narHash": "sha256-tqM7xDEsaeK1p6VvRygeZ9LkQRpUUkbRT02BHv6hnTM=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "8edc0c72e3a38faf5434e40d1f19431125fe4b30",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixpkgs-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"root": {
"inputs": {
"devenv": "devenv",
"nixpkgs": "nixpkgs"
}
}
},
"root": "root",
"version": 7
}
+3 -40
View File
@@ -1,46 +1,9 @@
{ pkgs, lib, config, inputs, ... }:
{ pkgs, ... }:
{
# https://devenv.sh/basics/
env.GREET = "devenv";
# https://devenv.sh/packages/
packages = [ pkgs.git ];
# https://devenv.sh/languages/
# languages.rust.enable = true;
# https://devenv.sh/processes/
# processes.dev.exec = "${lib.getExe pkgs.watchexec} -n -- ls -la";
# https://devenv.sh/services/
# services.postgres.enable = true;
# https://devenv.sh/scripts/
scripts.hello.exec = ''
echo hello from $GREET
'';
# https://devenv.sh/basics/
languages.rust.enable = true;
enterShell = ''
hello # Run scripts directly
git --version # Use packages
export CLAUDE_CONFIG_DIR="$(pwd)/.claude-account"
'';
# https://devenv.sh/tasks/
# tasks = {
# "myproj:setup".exec = "mytool build";
# "devenv:enterShell".after = [ "myproj:setup" ];
# };
# https://devenv.sh/tests/
enterTest = ''
echo "Running tests"
git --version | grep --color=auto "${pkgs.git.version}"
'';
# https://devenv.sh/git-hooks/
# git-hooks.hooks.shellcheck.enable = true;
# See full reference at https://devenv.sh/reference/options/
}
+262
View File
@@ -0,0 +1,262 @@
# skyg-linux-tracker — Design Doc
A read-only Linux system monitor in Rust, distributed as a single binary. Exposes systemd state, journald logs, and Docker container state/logs over three surfaces that share one core: a **REST API**, an **MCP server** (Streamable HTTP), and **OpenTelemetry export**. All data is fetched live from the source on every request — no metric/log collection or buffering, except a small SQLite database that stores API access tokens.
## Confirmed requirements
- **Single Rust binary**, read-only against the host.
- **MCP over Streamable HTTP with auth** (mounts on the same axum server as the REST API).
- **Config via TOML file + CLI flags + env vars** (precedence: CLI > env > file > defaults).
- **OTel**: systemd unit / container status exported as **metrics** (gauges); journald / docker logs exported as **OTel logs**. Traces for the API itself are included since they come nearly free.
- **MCP and REST expose the same data**; MCP adds agent-facing context (tool descriptions, instructions, curated resources/prompts).
- **State**: `/var/lib/skyg-linux-tracker/db.sqlite` (SQLite via `sqlx`), access tokens only.
## Tech choices
| Area | Crate | Why |
|---|---|---|
| HTTP | `axum` + `tokio` | Industry standard; rmcp's HTTP transport is a Tower service that mounts directly onto an axum router |
| MCP | `rmcp` (official `modelcontextprotocol/rust-sdk`), features `server`, `macros`, `schemars`, `transport-streamable-http-server` | Official SDK; `StreamableHttpService` mounts at `/mcp` on the same axum router; tools via `#[tool]` macros |
| systemd | `zbus_systemd` (feature `systemd1`, tokio executor) | Pure-Rust generated bindings for `org.freedesktop.systemd1` — `ListUnits`, `ListTimers`, unit properties. No libsystemd C dependency |
| journald | `systemd` crate, `journal` feature (sd-journal FFI) | Only maintained option supporting live tail (`seek_tail` + `next` + `wait`). Links `libsystemd` at build time (present on any systemd host; the devenv shell provides it) |
| Docker | `bollard` (rustls, no openssl) | Async Docker/Podman API over the unix socket; `list_containers`, `inspect_container` (health), `logs(follow=true)` streams |
| OTel | `opentelemetry`, `opentelemetry_sdk`, `opentelemetry-otlp` (metrics + logs both **Stable** in current opentelemetry-rust; traces Beta) | One OTLP exporter config (endpoint, protocol, headers) drives all three signals |
| Auth DB | `sqlx` + `sqlite` (bundled) | Async, compile-time-checked queries, pure-Rust bundled SQLite — keeps the binary self-contained |
| Config | `figment` (TOML + env) + `clap` | Layered config with clear precedence |
## Repo layout
```
linux-system-monitor/
├── Cargo.toml
├── devenv.nix # + languages.rust, pkg-config/openssl/libsystemd build deps
├── devenv.yaml
├── config.example.toml
├── systemd/
│ └── skyg-linux-tracker.service # example unit file (hardened, read-only)
└── src/
├── main.rs # entry: config load → telemetry init → DB → server
├── config.rs # figment layering: defaults < TOML < env(SKYG_) < CLI
├── error.rs # thiserror AppError → axum + MCP error mappings
├── auth/
│ ├── mod.rs # bearer middleware + token verify (sha256 hash lookup)
│ └── tokens.rs # token CRUD against SQLite (`slinuxm token create/revoke/list` CLI)
├── db.rs # sqlx pool, embedded migrations, open-or-create
├── sources/ # === the core: one trait per domain, live reads only ===
│ ├── mod.rs
│ ├── systemd.rs # list_units(), list_timers(), unit_status(name)
│ ├── journal.rs # query(filter) snapshot, follow(filter) → mpsc stream
│ └── docker.rs # ps(), container_status(name), logs(name, follow)
├── api/ # REST layer (thin): DTOs + handlers calling sources/
│ ├── mod.rs # router assembly, auth middleware layer
│ ├── systemd.rs # GET /api/v1/systemd/{units,timers,units/:name}
│ ├── journal.rs # GET /api/v1/journal (snapshot), GET .../journal/follow (SSE)
│ └── docker.rs # GET /api/v1/docker/{ps,containers/:name,containers/:name/logs}
├── mcp/
│ ├── mod.rs # TrackerService: #[tool_router] tools wrapping sources/
│ └── prompts.rs # agent-facing prompts ("diagnose failing units", etc.)
└── telemetry/
├── mod.rs # OTel providers init (traces/metrics/logs), shutdown
├── metrics_export.rs # background loop → observable gauges
└── logs_export.rs # journal/docker follow → OTel log records fan-out
```
The key structural decision: **`sources/` is the only place that talks to D-Bus/journald/Docker.** `api/`, `mcp/`, and `telemetry/` are three thin projections over it, which is what makes "MCP and API expose the same data" true by construction.
## Implementation detail
### Config (`config.rs`, `config.example.toml`)
```toml
[server]
bind = "127.0.0.1" # set 0.0.0.0 only with auth enabled
port = 9109
[auth]
enabled = true # hard-required when bind != localhost
db_path = "/var/lib/skyg-linux-tracker/db.sqlite"
[otel]
enabled = false
endpoint = "http://localhost:4317" # OTLP gRPC; http://…:4318 for HTTP/proto
protocol = "grpc"
service_name = "skyg-linux-tracker" # OTel resource attr service.name
export_unit_metrics = true # poll loop, interval below
export_logs = false # journal/docker log fan-out to OTLP
metrics_interval_secs = 30
[otel.logs]
journal_units = [] # empty = all units
docker_containers = [] # empty = all containers
[sources]
docker_socket = "/var/run/docker.sock"
journal_namespace = "" # default journal
```
Precedence: built-in defaults < `--config /etc/skyg-linux-tracker.toml` < `SKYG_*` env vars < CLI flags. CLI subcommands: `serve` (default), `token create|revoke|list`.
### Sources (live reads only)
- **systemd** (`sources/systemd.rs`): one shared `zbus::Connection` to the system bus. `list_units()` → `ManagerProxy::list_units()`; `list_timers()` → `list_timers()` (next-elapse realtime converted to RFC3339); `unit_status(name)` → `load_unit` + properties (`ActiveState`, `SubState`, `ExecMainStatus`, `NRestarts`, `ActiveEnterTimestamp`). No caching — every call hits D-Bus.
- **journal** (`sources/journal.rs`): `Journal::open()`, `match_add("_SYSTEMD_UNIT", …)`, `seek_tail()`. Snapshot queries seek backward N entries. `follow()` wraps `Journal::wait()` in a `tokio::task::spawn_blocking` bridge (sd-journal is blocking) feeding a `tokio::sync::mpsc` — consumers (SSE endpoint, MCP tool, OTel log exporter) all read from channels.
- **docker** (`sources/docker.rs`): shared `bollard::Docker` over the unix socket. `ps()` → `list_containers` + `inspect_container` per container for `State.Health`, `StartedAt` (→ uptime), restart count. `logs(name, follow)` → `bollard` log stream (`stdout`, `stderr`, `timestamps`, `since`) as a `Stream<Item = LogOutput>`.
### REST API (`api/`)
All under `/api/v1`, all behind the auth middleware when enabled, all read-only (GET only):
| Endpoint | Source |
|---|---|
| `GET /api/v1/systemd/units?state=failed` | `list_units()` |
| `GET /api/v1/systemd/units/:name` | `unit_status()` |
| `GET /api/v1/systemd/timers` | `list_timers()` |
| `GET /api/v1/journal?unit=X&priority=err&lines=100` | journal snapshot |
| `GET /api/v1/journal/follow?unit=X` (SSE) | journal `follow()` |
| `GET /api/v1/docker/ps` | `ps()` |
| `GET /api/v1/docker/containers/:name` | `container_status()` |
| `GET /api/v1/docker/containers/:name/logs?follow=1` (SSE) | `logs(follow)` |
| `GET /healthz` | unauthenticated liveness |
Streaming endpoints use SSE (`axum::response::Sse`) over the mpsc channels from `sources/`.
### MCP server (`mcp/`)
`rmcp`'s `StreamableHttpService` is a Tower service mounted on the same axum router:
```rust
let mcp_service = StreamableHttpService::new(
|| Ok(TrackerService::new(app_state.clone())), // per-session handler, shared state
LocalSessionManager::default().into(),
StreamableHttpServerConfig::default().with_json_response(false),
);
let app = Router::new()
.nest("/api/v1", api_router)
.nest_service("/mcp", mcp_service)
.layer(auth_middleware); // same bearer auth covers both
```
Tools (each a thin wrapper over `sources/`, params derive `schemars::JsonSchema`):
- `list_systemd_units { state? }` / `get_systemd_unit { name }` / `list_systemd_timers {}`
- `get_journal_logs { unit?, priority?, lines }` / `tail_journal { unit, seconds }` (bounded follow — collects for N seconds, returns; no infinite MCP streams)
- `list_containers {}` / `get_container { name }` / `get_container_logs { name, tail }`
The "bit more context for the agent" lives in:
- `ServerHandler` **instructions**: how the tools relate ("check `list_systemd_units state=failed` first, then pull journal logs for that unit").
- Rich `#[tool(description = …)]` strings explaining systemd states, health statuses, priorities.
- **Prompts** (`mcp/prompts.rs`): `diagnose_unit { name }`, `container_health_report {}` — canned investigation workflows.
- **Resources**: `systemd://units/failed`, `docker://containers` as subscribable snapshot resources.
### OpenTelemetry (`telemetry/`)
One OTLP exporter config → three signals:
1. **Traces** (Beta in opentelemetry-rust): `tracing-opentelemetry` layer + `tower-http` `TraceLayer` — spans per HTTP/MCP request.
2. **Metrics** (Stable): `metrics_export.rs` runs a poll loop every `metrics_interval_secs`:
- `systemd.unit.active` (u64 gauge 0/1, attrs: `unit`, `sub_state`, `load_state`)
- `systemd.unit.failed` (u64 0/1), `systemd.unit.restarts` (u64 counter-ish gauge)
- `systemd.timer.next_elapse_seconds` (f64, attr `timer`)
- `docker.container.running` (0/1, attrs `container`, `image`), `docker.container.healthy` (0/1, `health_status` attr), `docker.container.uptime_seconds`, `docker.container.restarts`
- Configurable unit/container allowlists so cardinality stays bounded.
3. **Logs** (Stable): when `otel.export_logs = true`, `logs_export.rs` spawns journal `follow()` + per-container docker `logs(follow=true)` tasks, converts each entry to an OTel `LogRecord` (severity from journal priority / docker stderr-vs-stdout, attrs: `unit`, `container`, `host`) and emits through the SDK `LoggerProvider`.
"Same information via OTel" is satisfied by reusing the identical `sources/` functions; the OTel path is the only one that does periodic push (metrics poll) and continuous fan-out (logs), because OTLP is a push model — everything else stays pull/live.
### Auth & database (`auth/`, `db.rs`)
- SQLite at `/var/lib/skyg-linux-tracker/db.sqlite` (dir created `0700` on first run; `sqlx` embedded migrations).
- Table `tokens(id, name, token_hash TEXT UNIQUE, created_at, revoked_at)` — only **sha256 hashes** stored, plaintext shown once at `slinuxm token create`.
- Middleware: constant-time compare of `Authorization: Bearer <token>` hash; 401 JSON body on REST, JSON-RPC error on MCP.
- `auth.enabled = false` is rejected at startup if `bind` isn't loopback.
## Build & packaging
- **devenv.nix**: enable `languages.rust.enable = true`; add `pkg-config`, `systemd` (libsystemd headers for the journal FFI) to `packages`. Add scripts: `build`, `run`, `token` wrappers.
- Static-ish single binary: everything pure-Rust except libsystemd (acceptable — target hosts run systemd by definition) and bundled SQLite. rustls everywhere (no openssl).
- Example `systemd/skyg-linux-tracker.service`: hardening flags (`ProtectSystem=strict`, `NoNewPrivileges`, `ReadWritePaths=/var/lib/skyg-linux-tracker`), supplementary group `systemd-journal` + optional `docker` group for socket access.
## Phases & workstreams
Work is split so that after a small shared foundation, four tracks run in parallel, then three surfaces run in parallel. Each workstream lists its dependencies, the files it owns (to avoid merge conflicts), and its done-criteria.
### Phase 0 — Foundation (blocking, single track)
Everything depends on this; keep it deliberately small.
| Task | Files |
|---|---|
| Cargo.toml + devenv.nix setup (rust toolchain, libsystemd build dep, all deps pinned) | `Cargo.toml`, `devenv.nix` |
| Config layering (defaults < TOML < `SKYG_*` env < CLI), `config.example.toml`, clap CLI skeleton with `serve` + `token` subcommands | `src/config.rs`, `src/main.rs`, `config.example.toml` |
| Shared DTOs + error types: the response shapes (`UnitStatus`, `TimerInfo`, `JournalEntry`, `ContainerSummary`, …) every surface returns | `src/sources/mod.rs` (DTO definitions), `src/error.rs` |
**Gate:** `cargo run -- serve` starts, loads layered config, prints effective config. No sources or surfaces yet.
### Phase 1 — Sources + auth (4 parallel tracks)
Once DTOs exist, each track is fully independent.
**WS-A: systemd source**
- Files: `src/sources/systemd.rs`
- zbus connection + `list_units` / `list_timers` / `unit_status` returning shared DTOs.
- Done: an example binary calls each fn against the live bus and prints JSON.
**WS-B: journal source**
- Files: `src/sources/journal.rs`
- `query()` snapshot + `follow()` → mpsc (spawn_blocking bridge).
- Done: example tails the journal for a unit and prints entries; snapshot respects `lines`/`priority` filters.
**WS-C: docker source**
- Files: `src/sources/docker.rs`
- `ps()` / `container_status()` / `logs(follow)`.
- Done: example lists containers with health/uptime; log follow streams to stdout.
**WS-D: auth + DB**
- Files: `src/db.rs`, `src/auth/`
- sqlx pool + migration, token CRUD, `slinuxm token create/revoke/list` CLI, bearer middleware (axum-compatible).
- Done: token round-trips CLI → DB → middleware check; startup rejects `auth.enabled=false` + non-loopback bind.
### Phase 2 — Surfaces (3 parallel tracks, after Phase 1)
Each surface consumes `sources/` + the auth middleware; they never touch each other's files.
**WS-E: REST API** (`src/api/`)
- All endpoints from the table above, SSE streaming, `/healthz`.
- Depends on: A, B, C, D.
- Done: curl checks pass (Verification #3).
**WS-F: MCP server** (`src/mcp/`)
- `StreamableHttpService` at `/mcp`, 9 tools, prompts, resources, server instructions.
- Depends on: A, B, C, D.
- Done: MCP inspector checks pass (Verification #4).
**WS-G: OTel export** (`src/telemetry/`)
- Provider init (traces/metrics/logs), metrics poll loop, log fan-out tasks, graceful shutdown.
- Depends on: A, B, C.
- Done: otelcol debug exporter shows gauges + log records (Verification #5).
### Phase 3 — Integration & hardening (single track, after Phase 2)
- Wire all surfaces into one axum server in `main.rs` (the only place surfaces meet).
- End-to-end negative tests (Verification #6), example systemd unit file, README/config docs, binary size + `ldd` check for the single-binary claim.
### Dependency graph
```
Phase 0 (config + DTOs + skeleton)
├── WS-A systemd ──┐
├── WS-B journal ──┼── WS-E REST ──┐
├── WS-C docker ── │── WS-F MCP ── ┼── Phase 3 integration
└── WS-D auth ─────┘── WS-G OTel ──┘
```
Solo, do it sequentially: 0 → A → B → C → D → E → F → G → 3. With multiple contributors (or parallel agents), Phase 1's four tracks and Phase 2's three tracks are fully parallel — disjoint files, DTO contract fixed in Phase 0.
## Verification
1. `cargo build` inside `devenv shell`; unit tests for config precedence and DTO mapping.
2. Run locally: `cargo run -- serve --bind 127.0.0.1`; create a token via `cargo run -- token create`.
3. REST: `curl -H "Authorization: Bearer …" localhost:9109/api/v1/systemd/units?state=failed`, SSE follow endpoints via `curl -N`.
4. MCP: point a client at `http://localhost:9109/mcp` (e.g. `npx @modelcontextprotocol/inspector`) — verify tool list, call each tool, check auth rejection without the header.
5. OTel: run `otelcol` with a debug exporter, enable `[otel]`, confirm unit/container gauges appear on the poll interval and journal/docker log records flow with correct severity/attributes.
6. Negative tests: no token → 401; `auth.enabled=false` + `bind=0.0.0.0` → startup refuses; docker socket absent → clean error on docker endpoints, rest of API unaffected.
+57
View File
@@ -0,0 +1,57 @@
# Board
_Last updated: 2026-10-15_
Move a card by cutting its row into another column. Keep the card's own **Status** field in sync. Rules are in [`README.md`](README.md).
## Blocked
| Card | Owner | Reason |
|---|---|---|
| — | | |
## Ready
| Card | Owner | Notes |
|---|---|---|
| [P0 Foundation](workstreams/P0-foundation.md) | — | Confirm proposed decisions DEC-2 to DEC-7 first |
## In progress
| Card | Owner | Branch |
|---|---|---|
| — | | |
## Review
| Card | Owner | PR |
|---|---|---|
| — | | |
## Done
| Card | Completed |
|---|---|
| — | |
## Backlog
| Card | Waiting on |
|---|---|
| [WS-A systemd](workstreams/WS-A-systemd.md) | P0 |
| [WS-B journal](workstreams/WS-B-journal.md) | P0 |
| [WS-C docker](workstreams/WS-C-docker.md) | P0 |
| [WS-D auth + DB](workstreams/WS-D-auth.md) | P0 |
| [WS-E REST](workstreams/WS-E-rest.md) | P0 to start; A, B, C, D to finish |
| [WS-F MCP](workstreams/WS-F-mcp.md) | P0 to start; A, B, C, D to finish |
| [WS-G OTel](workstreams/WS-G-otel.md) | P0 to start; A, B, C to finish |
| [P3 Integration](workstreams/P3-integration.md) | E, F, G |
## Milestones
| Milestone | Gate | Status |
|---|---|---|
| M0 Skeleton | `slinuxm serve --print-config` works. Clippy and tests are clean. No openssl. | ⬜ |
| M1 Sources live | All four Phase 1 examples run against a real host | ⬜ |
| M2 Surfaces | Verification #3, #4 and #5 pass | ⬜ |
| M3 Release | Verification #1–#6 pass. Unit file and README shipped. | ⬜ |
+50
View File
@@ -0,0 +1,50 @@
# Workboard
Markdown workboard for building skyg-linux-tracker (CLI: `slinuxm`). The design lives in [`../plan.md`](../plan.md). This folder tracks the work.
| File | Purpose |
|---|---|
| [`BOARD.md`](BOARD.md) | Kanban view. One row per workstream card, moved between columns. |
| [`../TODO.md`](../TODO.md) | Master checklist. The only place task checkboxes are ticked. |
| [`workstreams/`](workstreams/) | One card per workstream: goal, owned files, dependencies, design notes, done criteria, log. |
| [`review.md`](review.md) | Design review of `plan.md`. Corrections and gaps, each mapped to a task. |
| [`decisions.md`](decisions.md) | Decision log. Open questions and proposed defaults. |
## Cards
| Card | Phase | Depends on | Owned files |
|---|---|---|---|
| [P0 Foundation](workstreams/P0-foundation.md) | 0 | — | `Cargo.toml`, `devenv.nix`, `config.example.toml`, `src/main.rs`, `src/config.rs`, `src/error.rs`, `src/state.rs`, `src/sources/mod.rs`, `src/sources/fake.rs`, all stubs |
| [WS-A systemd](workstreams/WS-A-systemd.md) | 1 | P0 | `src/sources/systemd.rs`, `examples/systemd_dump.rs` |
| [WS-B journal](workstreams/WS-B-journal.md) | 1 | P0 | `src/sources/journal.rs`, `examples/journal_tail.rs` |
| [WS-C docker](workstreams/WS-C-docker.md) | 1 | P0 | `src/sources/docker.rs`, `examples/docker_ps.rs` |
| [WS-D auth + DB](workstreams/WS-D-auth.md) | 1 | P0 | `src/db.rs`, `src/auth/`, `migrations/` |
| [WS-E REST](workstreams/WS-E-rest.md) | 2 | P0 (fakes); A, B, C, D to finish | `src/api/` |
| [WS-F MCP](workstreams/WS-F-mcp.md) | 2 | P0 (fakes); A, B, C, D to finish | `src/mcp/` |
| [WS-G OTel](workstreams/WS-G-otel.md) | 2 | P0 (fakes); A, B, C to finish | `src/telemetry/`, `dev/otelcol.yaml` |
| [P3 Integration](workstreams/P3-integration.md) | 3 | E, F, G | `src/main.rs` (wiring), `systemd/`, `README.md`, `tests/` |
## Dependency graph
```
P0 Foundation (config, DTOs, traits+fakes, AppState, stubs)
├── WS-A systemd ──┐
├── WS-B journal ──┤
├── WS-C docker ───┼──► WS-E REST ──┐
└── WS-D auth ─────┤ WS-F MCP ───┼──► P3 Integration
└──► WS-G OTel ──┘
```
With source traits and fakes (DEC-2), Phase 2 cards can **start** as soon as P0 is done. They **finish** only once their real sources are merged and verified against a live host.
**Solo order:** P0 → A → B → C → D → E → F → G → P3.
## Rules
1. **Status flow:** `Backlog` → `Ready` → `In progress` → `Review` → `Done`, plus `Blocked` with a reason. A card becomes `Ready` when everything in its "Depends on" is `Done`, or when it can run against fakes.
2. **One owner per card.** Put the name (or agent/branch) in the card's Owner field and on the board.
3. **File ownership is strict.** A card edits only its owned files. If it needs a change in someone else's file, typically a DTO in `src/sources/mod.rs`, raise it in the card log and make the change as a small separate commit agreed with that file's owner.
4. **Tick tasks in `TODO.md`.** Cards describe tasks but do not carry checkboxes, so there is one source of truth.
5. **A card moves to `Done`** when every task is ticked, its "Done when" criteria are demonstrated, and `cargo clippy -- -D warnings` + `cargo test` pass.
6. **Decisions go in `decisions.md`.** Record the outcome there and link it from the card log.
7. **Branch naming:** `ws-a/systemd-source`, `p0/foundation`, and so on.
+107
View File
@@ -0,0 +1,107 @@
# Decision log
Decisions that shape the work. **Open** ones need a human call before the listed task starts. **Proposed** ones have a recommended default that the board already assumes. Override them here if you disagree.
| ID | Decision | Status | Blocks |
|---|---|---|---|
| DEC-1 | Binary / CLI name: `slinuxm` | Decided | — |
| DEC-2 | Source traits + fakes in Phase 0 | Proposed | P0-7 |
| DEC-3 | sqlx runtime queries instead of macros | Proposed | D3 |
| DEC-4 | MCP auth failure = HTTP 401 | Proposed | D5 |
| DEC-5 | MCP resources not subscribable in v1 | Proposed | F7 |
| DEC-6 | Docker socket access model | Proposed | I3 |
| DEC-7 | Default metrics scope | Proposed | G4 |
| DEC-8 | Origin validation for `/mcp` | Open until spike | F8 |
---
## DEC-1 — Binary / CLI name
**Status:** Decided 2026-10-15
The plan used three names. The repo is `linux-system-monitor`, the service is `skyg-linux-tracker`, and the CLI examples said `skyg token create`.
**Decision:** The binary and CLI are **`slinuxm`**. Examples are `slinuxm serve` and `slinuxm token create <name>`. The `[[bin]]` name in `Cargo.toml` is `slinuxm`. Issued tokens use the prefix `slinuxm_`.
**Assumed unchanged:** the product name `skyg-linux-tracker`, the unit file `skyg-linux-tracker.service`, the state dir `/var/lib/skyg-linux-tracker`, the config path `/etc/skyg-linux-tracker.toml`, and the `SKYG_` env prefix. Revisit here if those should become `slinuxm` too.
---
## DEC-2 — Source traits + fakes in Phase 0
**Status:** Proposed (board assumes yes)
Define `SystemdSource`, `JournalSource`, `DockerSource` traits in `src/sources/mod.rs`, with in-memory fakes in `src/sources/fake.rs`. `AppState` holds `Arc<dyn …>`.
- Pro: WS-E/F/G can start right after Phase 0 against fakes. Handler and tool tests run in CI without systemd or Docker.
- Con: About 100 extra lines in Phase 0. Needs `async-trait`, or native async fn in traits with boxed futures for `dyn` use.
**Decision:**
---
## DEC-3 — sqlx runtime queries instead of macros
**Status:** Proposed
`sqlx::query!` macros need `DATABASE_URL` at build time or a committed `.sqlx/` cache via `cargo sqlx prepare`. With one table and about five queries, runtime `sqlx::query_as` plus tests gives the same safety with no build-time DB.
**Decision:**
---
## DEC-4 — MCP auth failure = HTTP 401
**Status:** Proposed
The plan says auth failures on MCP return a JSON-RPC error. The MCP authorization spec puts auth at the HTTP layer. Clients expect `401 Unauthorized` with `WWW-Authenticate: Bearer`. One middleware gives the same response for `/api/v1` and `/mcp`.
**Decision:**
---
## DEC-5 — MCP resources not subscribable in v1
**Status:** Proposed
Subscriptions need the server to detect changes and push notifications, which means background polling. That conflicts with "live reads only". v1 exposes `systemd://units/failed` and `docker://containers` as list/read resources resolved live on each read.
**Decision:**
---
## DEC-6 — Docker socket access model
**Status:** Proposed
Membership in the `docker` group is root-equivalent. Our code only reads, but a compromised process could do anything.
Options:
1. `SupplementaryGroups=docker` in the unit file, documented as a trade-off. This is the plan's default.
2. Point `sources.docker_socket` at a read-only socket proxy that only allows container list, inspect and logs.
**Recommendation:** Ship option 1 commented out in the unit file. Document option 2 in the README as the hardened setup. No code changes needed either way.
**Decision:**
---
## DEC-7 — Default metrics scope
**Status:** Proposed
"Empty allowlist = all units" exports every loaded unit, often more than 300 including devices and mounts.
**Recommendation:** With empty `otel.metrics.units`, export `*.service` and `*.timer` units only. A non-empty list is an explicit allowlist that accepts glob patterns. Containers default to all, since counts are usually small.
**Decision:**
---
## DEC-8 — Origin validation for `/mcp`
**Status:** Open until spike P0-3
The MCP spec requires Origin validation on Streamable HTTP servers. If the pinned `rmcp` version has a built-in allowed-hosts or Origin option, use it. Otherwise add a small tower layer on `/mcp` that rejects requests whose `Origin` is present and not in an allowlist. Default allowlist: loopback origins plus `server.bind`.
**Decision:**
+59
View File
@@ -0,0 +1,59 @@
# Design review — `plan.md`
Reviewed 2026-10-15 against `plan.md` (skyg-linux-tracker design doc).
**Verdict: sound, proceed.** The core architecture holds up. `sources/` is the only layer that talks to the host, and REST, MCP and OTel are thin projections over it. That makes the "same data on every surface" requirement true by construction and makes the parallel workstreams real. The crate choices are reasonable.
The items below are corrections or gaps. Each is folded into a task in [`../TODO.md`](../TODO.md) (task IDs in brackets). Items needing a human call are tracked in [`decisions.md`](decisions.md).
## Correctness issues (plan is wrong as written)
| # | Issue | Fix | Task |
|---|---|---|---|
| R1 | systemd's D-Bus `Manager` has **no `ListTimers` method**. `systemctl list-timers` lists `*.timer` units, then reads the `org.freedesktop.systemd1.Timer` interface on each. | Use `ListUnitsByPatterns(["*.timer"])` (or filter `ListUnits`), then `TimerProxy` properties `NextElapseUSecRealtime`, `LastTriggerUSec`, `Unit`. Treat `0` / `u64::MAX` as "not scheduled". | A3 |
| R2 | `ExecMainStatus` and `NRestarts` live on the **`Service`** interface, not `Unit`. They do not exist for timers, sockets, mounts. | Read `Unit` props for every unit and `Service` props only for `.service` units. Make those DTO fields `Option`. | A4, P0-6 |
| R3 | `.layer(auth_middleware)` on the whole router also protects **`/healthz`**, contradicting "unauthenticated liveness". | Build a protected router (`/api/v1`, `/mcp`) with the auth layer, then `.merge()` a public router holding `/healthz`. | I1, D6 |
| R4 | Route table uses `:name`. **axum 0.8 uses `/{name}`** and panics at startup on the old `:name` syntax. | Use `{name}` in code. The table in `plan.md` is fine as notation. | E1 |
| R5 | Tool count mismatch. The tool list has **8** tools (3 systemd, 2 journal, 3 docker) but WS-F says "9 tools". | Board uses 8. Add a 9th only deliberately. | F1 |
| R6 | `config.example.toml` has no `otel.headers` (the tech table relies on it) and no **metrics** allowlists (the OTel section relies on them). Only log allowlists exist. | Add `otel.headers` and `[otel.metrics] units/containers`. | P0-4, P0-5 |
| R7 | Repo layout comment says devenv needs `openssl`, which contradicts "rustls everywhere". | Drop openssl. Gate on `cargo tree -i openssl` returning nothing. | P0-1, P0-2 |
| R8 | "Compile-time-checked queries" with `sqlx` needs `DATABASE_URL` at build time or a committed `.sqlx/` offline cache. Neither is in the plan. | For one table, use runtime `sqlx::query` / `query_as`. See decision DEC-3. | D3 |
## Gaps (missing from the plan, will bite during implementation)
| # | Gap | Fix | Task |
|---|---|---|---|
| G1 | **Blocking journal calls.** Only `follow()` mentions `spawn_blocking`. Snapshot `query()` is also blocking sd-journal FFI and would stall a tokio worker. | Run all sd-journal work in `spawn_blocking`. Never hold a `Journal` across an `.await`. | B1, B2 |
| G2 | **Follow-stream leaks.** A blocking `wait()` loop does not notice an SSE client disconnecting, so threads pile up. | `wait()` with a short timeout and exit when `tx.is_closed()` or the shutdown token fires. Add a test. | B3, B6 |
| G3 | **Graceful shutdown with open SSE.** axum's graceful shutdown waits for open connections. Long-lived SSE streams block it forever. | One `CancellationToken` in `AppState`. Every stream and background task selects on it. | P0-9, E4, G6, I1 |
| G4 | **Docker containers come and go.** OTel log fan-out spawns per-container follow tasks at startup only. New containers are missed and dead ones leave dangling tasks. | Reconcile from the Docker events stream (`start`/`die`/`destroy`) or a periodic re-list. | G5 |
| G5 | **File-ownership conflicts.** Phase 0 owns `main.rs` and `sources/mod.rs`, but WS-D must add the `token` CLI dispatch to `main.rs`, and each source track must add `pub mod x;` to `sources/mod.rs`. | Phase 0 creates every module as a stub with final function signatures. Workstreams then only fill in their own files. | P0-10 |
| G6 | **No shared `AppState` defined.** Every surface needs the same handles: sources, DB pool, config, shutdown token. | Define it in Phase 0 (`src/state.rs`). | P0-9 |
| G7 | **Input bounds.** `lines`, `tail`, and MCP `tail_journal { seconds }` are unbounded. A large value means a huge response or a hung MCP call. | Config-driven caps (`limits.max_lines`, `limits.max_follow_secs`). Validate unit and container names. | P0-4, A5, B2, F2 |
| G8 | **MCP Origin validation.** The MCP Streamable HTTP spec requires servers to validate `Origin` to block DNS-rebinding attacks. The plan does not mention it. | Check what the pinned `rmcp` provides. Otherwise add a small Origin/Host check layer on `/mcp`. | P0-3, F8 |
| G9 | **Docker absent at startup.** The plan wants a clean error on docker endpoints but doesn't say how the source is constructed. | Construct lazily. A missing or denied socket maps to `AppError::Unavailable` (503) per request and never fails startup. | C1, E9 |
| G10 | **Metric cardinality default.** "Empty = all" for metrics means every loaded unit (often 300+), including inactive mounts and devices. | Default the metrics scope to `*.service` and `*.timer`. Allowlists narrow it further. See DEC-7. | G4 |
| G11 | **Stale gauge series.** A unit or container that disappears keeps its last exported value with synchronous OTel gauges. | Use observable gauges fed from the latest poll snapshot, so vanished series stop being reported. | G3 |
## Recommendations (improvements, not errors)
- **Define the source traits in Phase 0 and ship fakes.** The layout already says "one trait per domain". With `SystemdSource` / `JournalSource` / `DockerSource` traits and in-memory fakes, WS-E/F/G can start right after Phase 0 against fakes instead of waiting for Phase 1. Handler and MCP tool tests also run without a live host. (DEC-2, P0-7)
- **Move the "auth disabled + non-loopback bind" check into config validation.** It is pure config logic and `config.rs` is a Phase 0 file. WS-D keeps it in its done-criteria as a verification. (P0-4)
- **MCP auth failures should be HTTP 401, not a JSON-RPC error.** The MCP authorization spec handles auth at the HTTP transport layer. Clients expect `401` with `WWW-Authenticate: Bearer`. Use the same middleware response for REST and MCP. (DEC-4, D5)
- **MCP resources: no subscriptions in v1.** Subscribable resources need change detection, which means polling. That contradicts "live reads only". Expose them as list/read resources resolved live. (DEC-5, F7)
- **Use OTel semantic-convention attribute names where they exist.** Examples are `container.name`, `container.id`, `container.image.name`, and `host.name` as a resource attribute. Keep `systemd.unit` for systemd, which has no convention. (G4)
- **`unit_status` should prefer `GetUnit`.** `LoadUnit` can load a unit file into the manager, a small side effect for a "read-only" tool. Fall back to `LoadUnit` only to report on unloaded units. (A4)
- **Token DB lookup by sha256 is already the check.** Constant-time comparison adds nothing once you do an indexed lookup on a high-entropy token's hash. Keep the lookup, skip the extra compare. (D5)
- **Use SQLite WAL mode plus `busy_timeout`.** The `token` CLI and the running server write and read the same DB at the same time. (D1)
- **Use systemd `StateDirectory=` in the unit file.** It creates `/var/lib/skyg-linux-tracker` with the right owner and mode, so the app does not need root to create it. The app keeps its "create if missing" fallback for dev runs. (I3)
- **Docker group is root-equivalent.** Read-only in our code, but socket access grants full control of the daemon. Document it. Optionally support a read-only socket proxy (for example `docker-socket-proxy` with only `CONTAINERS=1`), which works as-is since `sources.docker_socket` is configurable. (DEC-6, I3, I4)
- **Add a time-boxed API spike to Phase 0.** Several plan details depend on exact crate APIs: rmcp's `StreamableHttpServerConfig` fields, zbus_systemd proxy names, whether the `systemd` crate can open a journal namespace, and bollard's feature flags. Confirm them while pinning versions, before parallel work fixes the contracts. (P0-3)
## Things that are fine as-is
- One axum server for REST and MCP, with `rmcp`'s `StreamableHttpService` mounted via `nest_service`.
- Bounded `tail_journal` for MCP instead of infinite streams.
- Tokens stored as sha256 hashes, plaintext shown once.
- Config precedence: CLI > env > file > defaults.
- libsystemd as the one dynamic dependency. Every target host runs systemd anyway.
- The Phase 0 → 1 → 2 → 3 structure and the dependency graph.
+82
View File
@@ -0,0 +1,82 @@
# P0 — Foundation
| Field | Value |
|---|---|
| Status | Ready |
| Owner | — |
| Phase | 0 (blocking, single track) |
| Depends on | — |
| Blocks | Every other card |
| Owned files | `Cargo.toml`, `devenv.nix`, `config.example.toml`, `src/main.rs`, `src/config.rs`, `src/error.rs`, `src/state.rs`, `src/sources/mod.rs`, `src/sources/fake.rs`, initial stubs of every other module |
## Goal
Fix every contract that parallel work depends on: dependencies, config shape, DTOs, source traits, error type, `AppState`, and module signatures. After P0, no other card should need to touch `Cargo.toml`, `main.rs` or `sources/mod.rs`, except by agreed exception.
Keep it small. Stubs contain `todo!()` and no logic.
## Tasks
Tick these in [`TODO.md`](../../TODO.md).
- **P0-1 devenv.** Keep `languages.rust.enable` and the existing `CLAUDE_CONFIG_DIR` `enterShell`. Add `pkgs.pkg-config` and `pkgs.systemd` (its `.dev` output provides `libsystemd.pc`). Add `scripts.build`, `scripts.run` and `scripts.token`, which wraps `cargo run -- token`. Do not add openssl. Optionally add `pkgs.opentelemetry-collector-contrib` for G7.
- **P0-2 Cargo.toml.** Package `skyg-linux-tracker` with `[[bin]] name = "slinuxm"` (DEC-1). Pin every crate needed by all phases:
- Runtime: `tokio` (full), `tokio-util` (`CancellationToken`), `futures`.
- HTTP: `axum` 0.8, `tower`, `tower-http` (`trace`).
- Data: `serde`, `serde_json`, `schemars`, `thiserror`, `jiff` or `chrono`.
- Config/CLI: `figment` (`toml`, `env`), `clap` (`derive`, `env`).
- MCP: `rmcp` (`server`, `macros`, `schemars`, `transport-streamable-http-server`).
- systemd/journal: `zbus`, `zbus_systemd` (`systemd1`), `systemd` (`journal`, default features off where possible).
- Docker: `bollard`, no TLS features needed for a unix socket.
- OTel: `opentelemetry`, `opentelemetry_sdk`, `opentelemetry-otlp` (`grpc-tonic`, `http-proto`, rustls), `tracing`, `tracing-subscriber`, `tracing-opentelemetry`.
- Auth: `sqlx` (`runtime-tokio`, `sqlite`), `sha2`, `rand`, `base64`.
- Optional: `async-trait`.
- Release profile: `lto = true`, `strip = true`, `codegen-units = 1`.
- **P0-3 API spike, max half a day.** Confirm against the pinned versions:
- The rmcp `StreamableHttpService::new` signature, `StreamableHttpServerConfig` fields, and whether it validates Origin or Host (DEC-8).
- zbus_systemd `ManagerProxy`, `UnitProxy`, `ServiceProxy`, `TimerProxy` names and property types.
- The `systemd` crate journal open API, `wait` with timeout, namespace support, and whether `Journal` is `Send`.
- bollard feature flags that avoid pulling openssl or rustls when only the unix socket is used.
- The opentelemetry-otlp builder API for headers and protocol selection.
- Record findings in the log below.
- **P0-4 config.rs.** Use figment `Serialized::defaults` < `Toml::file` < `Env::prefixed("SKYG_").split("__")` < CLI overrides. Key additions over `plan.md`:
- `otel.headers` (map)
- `otel.metrics.units` / `otel.metrics.containers` (glob allowlists)
- `limits.max_lines` (default 1000), `limits.max_tail` (default 1000), `limits.max_follow_secs` (default 60)
- Validation: reject `auth.enabled = false` when `bind` is not loopback.
- **P0-5 config.example.toml.** Document every key with a comment and its env-var name.
- **P0-6 DTOs** in `src/sources/mod.rs`:
- `UnitSummary`, `UnitStatus` (service fields `Option`), `TimerInfo`
- `JournalQuery`, `JournalEntry`, `Priority` (0–7, parse from name or number)
- `ContainerSummary`, `ContainerStatus`, `LogLine { stream: Stdout|Stderr, ts, message }`
- All derive `Serialize`, `JsonSchema` and `Debug`. Timestamps serialize as RFC3339.
- **P0-7 Traits + fakes.** `SystemdSource`, `JournalSource`, `DockerSource`. Follow methods return `mpsc::Receiver` or a boxed `Stream`. `fake.rs` provides fixture-backed implementations for tests (DEC-2).
- **P0-8 error.rs.** `AppError { NotFound, BadRequest, Unauthorized, Unavailable, Internal }`. Implement `IntoResponse` (status + `{"error": {"code", "message"}}`) and `From<AppError> for rmcp::ErrorData`.
- **P0-9 state.rs.** `AppState { config: Arc<Config>, systemd: Arc<dyn SystemdSource>, journal: Arc<dyn JournalSource>, docker: Arc<dyn DockerSource>, db: Option<SqlitePool>, shutdown: CancellationToken }`. Must be `Clone`.
- **P0-10 Stubs with final signatures:**
- `api::router(AppState) -> Router`
- `mcp::service(AppState) -> StreamableHttpService<…>`
- `telemetry::init(&Config) -> Result<TelemetryGuard>`
- `auth::layer(AppState)`
- `auth::tokens::run(TokenCmd, &Config) -> Result<()>`
- `db::open(&Config) -> Result<SqlitePool>`
- Source constructors in `sources/{systemd,journal,docker}.rs`.
- `main.rs` dispatches `slinuxm serve` (default), `slinuxm token …`, and `--print-config`.
- **P0-11 Tests.** Defaults only, file overrides defaults, env overrides file, CLI overrides env. The auth-off + `0.0.0.0` combination is an error.
## Done when
- `cargo run -- serve --print-config` prints the merged config, and precedence works by hand.
- `cargo build` produces `target/debug/slinuxm`.
- `cargo clippy -- -D warnings`, `cargo fmt --check` and `cargo test` are clean.
- `cargo tree -i openssl` and `cargo tree -i openssl-sys` print nothing.
- Spike findings are recorded below and DEC-8 is updated.
## Risks
- Changing DTOs after Phase 1 starts causes merge pain in four places. Spend the time here.
- rmcp moves fast. Pin the exact version and note it in the log.
## Log
- 2026-10-15: Card created from `plan.md` review. DEC-1 decided: CLI is `slinuxm`.
+59
View File
@@ -0,0 +1,59 @@
# P3 — Integration & hardening
| Field | Value |
|---|---|
| Status | Backlog |
| Owner | — |
| Phase | 3 (single track) |
| Depends on | WS-E, WS-F, WS-G (and transitively A–D) |
| Blocks | Release |
| Owned files | `src/main.rs` (serve wiring), `systemd/skyg-linux-tracker.service`, `README.md`, `tests/` |
## Goal
Wire every surface into one axum server, prove the negative paths, and ship the unit file and docs. This is the only place the surfaces meet.
## Tasks
Tick these in [`TODO.md`](../../TODO.md).
- **I1** `slinuxm serve` wiring in `main.rs`:
1. Load config, then `telemetry::init`, then `db::open` when auth is enabled, then construct sources and `AppState`.
2. Build the protected router: `.nest("/api/v1", api::router(state))` and `.nest_service("/mcp", mcp::service(state))`, then `.layer(auth::layer(state))`.
3. Build the public router with `/healthz`.
4. `public.merge(protected).layer(TraceLayer::new_for_http())` (review R3).
5. Spawn the metrics and log export tasks when enabled.
6. On SIGINT or SIGTERM: cancel `state.shutdown` so SSE streams and background tasks end, run `axum::serve(...).with_graceful_shutdown`, then drop the telemetry guard to flush OTel (review G3).
- **I2** Integration tests in `tests/` covering Verification #6:
- A missing or invalid token gets 401 on `/api/v1/*` and `/mcp`. `/healthz` returns 200 without a token.
- `auth.enabled = false` with `bind = 0.0.0.0` makes startup exit non-zero with a clear message.
- A missing Docker socket gives 503 on docker routes while systemd routes still return 200.
- SIGTERM with an open SSE client exits within a few seconds.
- **I3** `systemd/skyg-linux-tracker.service`:
- `ExecStart=/usr/local/bin/slinuxm serve --config /etc/skyg-linux-tracker.toml`
- A dedicated `User=` (for example `skyg`), or `DynamicUser=yes` if the token CLI story still works. Decide and note it.
- `StateDirectory=skyg-linux-tracker`, `StateDirectoryMode=0700`
- `ProtectSystem=strict`, `ProtectHome=yes`, `PrivateTmp=yes`, `NoNewPrivileges=yes`, `ProtectKernelTunables=yes`, `ProtectControlGroups=yes`, `RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6`, `LockPersonality=yes`, `MemoryDenyWriteExecute=yes`
- `SupplementaryGroups=systemd-journal`. Add `docker` commented out with a root-equivalence warning (DEC-6).
- **I4** `README.md`:
- What it is and how to build it (`devenv shell`, `cargo build --release`).
- Install: binary, config, unit file.
- Config reference, or a pointer to `config.example.toml` and the env-var naming.
- Token workflow: `sudo -u skyg slinuxm token create <name>`.
- REST endpoint list, and MCP client setup with a bearer header example.
- OTel setup and the hardened Docker socket-proxy option.
- **I5** Release build checks:
- `cargo build --release`.
- `ldd target/release/slinuxm` lists only libc, libsystemd and libsystemd's own deps.
- Record the binary size in the log.
- **I6** Run Verification #1–#6 from `plan.md` by hand on a real systemd + Docker host and record the results in the log.
## Done when
- I2 tests pass in `cargo test`.
- I6 results are logged with all six checks passing.
- The unit file starts the service on a clean host with only the binary, config and unit installed.
## Log
- 2026-10-15: Card created.
+42
View File
@@ -0,0 +1,42 @@
# WS-A — systemd source
| Field | Value |
|---|---|
| Status | Backlog |
| Owner | — |
| Phase | 1 (parallel) |
| Depends on | P0 |
| Blocks | WS-E, WS-F, WS-G (to finish) |
| Owned files | `src/sources/systemd.rs`, `examples/systemd_dump.rs` |
## Goal
Implement `SystemdSource` over D-Bus with `zbus_systemd`. Every call hits the bus live, with no caching.
## Tasks
Tick these in [`TODO.md`](../../TODO.md).
- **A1** Create one `zbus::Connection::system()` at construction and share `ManagerProxy`.
- **A2** `list_units(state: Option<…>)`. Use `ListUnitsFiltered(states)` when a state is given, otherwise `ListUnits`. Map the tuple to `UnitSummary` (name, description, load/active/sub state).
- **A3** `list_timers()`. The Manager has no `ListTimers` method (review R1). Call `ListUnitsByPatterns([], ["*.timer"])`, then for each timer read `TimerProxy`: `NextElapseUSecRealtime`, `LastTriggerUSec`, `Unit` (the triggered unit). Convert µs since epoch to RFC3339. `0` or `u64::MAX` means `None`.
- **A4** `unit_status(name)`. Try `GetUnit(name)` first, falling back to `LoadUnit` only on `NoSuchUnit`. The fallback lets inactive unloaded units still report without side effects on loaded ones.
- `Unit` props: `ActiveState`, `SubState`, `LoadState`, `Description`, `ActiveEnterTimestamp`, `FragmentPath`.
- `Service` props, only when the name ends in `.service`: `ExecMainStatus`, `NRestarts`, `MainPID`.
- **A5** Validate names against systemd's charset (`[A-Za-z0-9:_.@\-\\]`, max 255) and return `BadRequest`. Map D-Bus `NoSuchUnit` to `NotFound`. Map bus connection failure to `Unavailable`.
- **A6** `examples/systemd_dump.rs` prints JSON for units, failed units, timers and one unit's status.
- **A7** Unit tests for µs-to-RFC3339 conversion, sentinel handling and tuple mapping.
## Done when
- `cargo run --example systemd_dump` prints sane JSON on a real host. Timers show next elapse times that match `systemctl list-timers`.
- `unit_status("nonexistent.service")` returns `NotFound` and `unit_status("../etc")` returns `BadRequest`.
## Notes
- Property reads cost one D-Bus round-trip each. Fetch with `GetAll` where zbus_systemd exposes it, or `join!` the reads.
- Unprivileged reads on the system bus are allowed. No polkit involvement is needed for these calls.
## Log
- 2026-10-15: Card created.
+47
View File
@@ -0,0 +1,47 @@
# WS-B — journal source
| Field | Value |
|---|---|
| Status | Backlog |
| Owner | — |
| Phase | 1 (parallel) |
| Depends on | P0 (incl. spike findings on the `systemd` crate) |
| Blocks | WS-E, WS-F, WS-G (to finish) |
| Owned files | `src/sources/journal.rs`, `examples/journal_tail.rs` |
## Goal
Implement `JournalSource` over sd-journal via the `systemd` crate. It provides a bounded snapshot query and a cancellable live follow.
## Tasks
Tick these in [`TODO.md`](../../TODO.md).
- **B1** Run every sd-journal call inside `tokio::task::spawn_blocking` (review G1). Open the journal inside the blocking closure and never hold it across `.await`. Use `sources.journal_namespace` if the spike found namespace support. Otherwise log a warning when it is set and use the default journal.
- **B2** `query(JournalQuery)`:
- Unit filter via `match_add("_SYSTEMD_UNIT", unit)`.
- Priority filter `≤ N` by adding one `PRIORITY=k` match for each k in 0..=N. Matches on the same field are OR'ed by sd-journal.
- `seek_tail`, then step backward `lines` entries. `lines` is clamped to `limits.max_lines`.
- Reverse the result so it returns oldest-first.
- **B3** `follow(JournalQuery, CancellationToken) -> mpsc::Receiver<JournalEntry>`:
- Seek to tail and run a blocking loop of `wait(Some(500ms))`, then drain new entries with `next`.
- Each iteration checks `tx.is_closed()` and `token.is_cancelled()` and exits if either is true (review G2).
- Use `blocking_send` with a bounded channel, around 1024 entries. Decide on drop-or-block for slow consumers and document it.
- **B4** Map `__REALTIME_TIMESTAMP`, `PRIORITY`, `_SYSTEMD_UNIT`, `SYSLOG_IDENTIFIER`, `_PID`, `_HOSTNAME` and `MESSAGE` to `JournalEntry`. `MESSAGE` may be binary, so convert it with lossy UTF-8.
- **B5** `examples/journal_tail.rs --unit X [--priority err] [--lines N] [--follow]`.
- **B6** Test that dropping the receiver makes the blocking thread exit within about one second. Use a counter or `JoinHandle` to prove it.
## Done when
- The example snapshot output matches `journalctl -u X -p err -n N -o json` for the same filters.
- Follow prints new entries live. `logger -t test hello` shows up in under a second.
- B6 passes.
## Notes
- Reading the full system journal needs the `systemd-journal` group (or root). Without it you only see your own user's entries. Say so in the example's error output.
- If the `systemd` crate turns out unmaintained or broken during the spike, record the alternative in the P0 log before starting.
## Log
- 2026-10-15: Card created.
+44
View File
@@ -0,0 +1,44 @@
# WS-C — docker source
| Field | Value |
|---|---|
| Status | Backlog |
| Owner | — |
| Phase | 1 (parallel) |
| Depends on | P0 |
| Blocks | WS-E, WS-F, WS-G (to finish) |
| Owned files | `src/sources/docker.rs`, `examples/docker_ps.rs` |
## Goal
Implement `DockerSource` over the Docker or Podman API with `bollard` on a unix socket. A missing Docker never breaks the rest of the service.
## Tasks
Tick these in [`TODO.md`](../../TODO.md).
- **C1** Construct with `Docker::connect_with_unix(socket, timeout, API_DEFAULT_VERSION)`. Construction must not fail when the socket is absent (review G9). Map connection-refused, not-found and permission-denied to `AppError::Unavailable` with a clear message on each call.
- **C2** `ps()`:
- `list_containers(all = true)`.
- Then `inspect_container` for each one concurrently, bounded with `buffer_unordered(8)`.
- Map `State.Health.Status`, `State.StartedAt` (to uptime seconds), `RestartCount`, image, and status.
- **C3** `container_status(name)` uses inspect only. A 404 maps to `NotFound`.
- **C4** `logs(name, follow, tail, since, token)` returns a boxed `Stream<Item = Result<LogLine>>`. Use `stdout`, `stderr` and `timestamps = true`, parse the timestamp prefix, and keep the stream type (stdout or stderr). The stream ends when dropped or when the token fires.
- **C5** Clamp `tail` to `limits.max_tail`.
- **C6** `examples/docker_ps.rs` prints `ps` as JSON. `--logs NAME [--follow]` streams logs.
- **C7** *(nice-to-have)* Run the example against `/run/podman/podman.sock` and note any field differences.
## Done when
- The example matches `docker ps -a` and shows health and uptime for containers with healthchecks.
- Following logs streams live, and Ctrl-C exits cleanly.
- With `docker_socket` pointed at a missing path, every call returns `Unavailable` and nothing panics.
## Notes
- Container names from the API have a leading `/`. Strip it in DTOs and accept names with or without it.
- `docker logs` only works for the `json-file`, `local` and `journald` log drivers. Others return an API error, which should map to `BadRequest` with the driver named.
## Log
- 2026-10-15: Card created.
+53
View File
@@ -0,0 +1,53 @@
# WS-D — auth + DB
| Field | Value |
|---|---|
| Status | Backlog |
| Owner | — |
| Phase | 1 (parallel) |
| Depends on | P0 |
| Blocks | WS-E, WS-F (to finish), P3 |
| Owned files | `src/db.rs`, `src/auth/mod.rs`, `src/auth/tokens.rs`, `migrations/` |
## Goal
Provide token storage in SQLite, the `slinuxm token` CLI, and one bearer-auth middleware that protects both `/api/v1` and `/mcp`.
## Tasks
Tick these in [`TODO.md`](../../TODO.md).
- **D1** `db::open`:
- Create the parent dir with mode `0700` if it is missing. In production, systemd `StateDirectory=` already creates it.
- Use `SqliteConnectOptions` with `create_if_missing`, `journal_mode(Wal)` and `busy_timeout(5s)`.
- Run `sqlx::migrate!()` from embedded `migrations/`.
- **D2** First migration: `tokens(id INTEGER PK, name TEXT NOT NULL UNIQUE, token_hash TEXT NOT NULL UNIQUE, created_at TEXT NOT NULL, revoked_at TEXT, last_used_at TEXT)`.
- **D3** `tokens.rs` with runtime `sqlx::query` / `query_as` (DEC-3):
- `create(name)` generates 32 bytes from the OS RNG and formats the token as `slinuxm_<base64url>`. Store `hex(sha256(token))` and print the plaintext once with a warning.
- `list()` shows name, created, revoked and last-used, never hashes.
- `revoke(name)` sets `revoked_at`. Revoking twice is a no-op with a message.
- **D4** Implement `auth::tokens::run(TokenCmd, &Config)` behind `slinuxm token create|list|revoke`. The `main.rs` dispatch already exists from P0, so do not edit `main.rs`.
- **D5** Middleware:
- Extract `Authorization: Bearer <t>`, hash it, and look up `WHERE token_hash = ? AND revoked_at IS NULL`.
- On success, optionally update `last_used_at`. Throttle that update to at most once a minute per token, or skip it.
- On failure, return `401` with `WWW-Authenticate: Bearer` and the standard JSON error body. Use the same response for `/mcp` (DEC-4).
- When `auth.enabled = false`, the layer is a pass-through.
- **D6** Expose the middleware so P3 applies it to the protected router only. `/healthz` must stay reachable without a token (review R3).
- **D7** Tests:
- Use an in-memory or tempdir DB.
- Create a token and confirm it is accepted. Revoke it and confirm it is rejected. Garbage and missing headers are rejected.
- Re-assert the P0 startup refusal for auth-off + non-loopback bind.
## Done when
- `slinuxm token create ci` prints a token. `slinuxm token list` shows it. A request with it passes the middleware.
- After `slinuxm token revoke ci`, the same request returns 401 without a server restart.
- D7 passes.
## Notes
- The CLI writes the same DB the server reads, so WAL mode is required. In production, run the CLI as the service user, for example `sudo -u skyg slinuxm token create …`. Document this in the README (I4).
## Log
- 2026-10-15: Card created.
+50
View File
@@ -0,0 +1,50 @@
# WS-E — REST API
| Field | Value |
|---|---|
| Status | Backlog |
| Owner | — |
| Phase | 2 (parallel) |
| Depends on | P0 to start (fakes). A, B, C, D to finish. |
| Blocks | P3 |
| Owned files | `src/api/mod.rs`, `src/api/systemd.rs`, `src/api/journal.rs`, `src/api/docker.rs` |
## Goal
Build thin, read-only GET handlers over `AppState` sources, with SSE for follow endpoints. `api::router(state)` returns a router that P3 nests under `/api/v1`.
## Tasks
Tick these in [`TODO.md`](../../TODO.md).
- **E1** `api::router(AppState) -> Router`. Use axum 0.8 path syntax `/{name}`, because `:name` panics (review R4). GET only.
- **E2** Systemd routes:
- `/systemd/units?state=failed|active|…`
- `/systemd/units/{name}`
- `/systemd/timers`
- **E3** `/journal?unit=&priority=err|3&lines=`. Parse priority by name or number. Clamp `lines` and return 400 on bad input.
- **E4** `/journal/follow?unit=&priority=` returns `Sse` built from the source receiver:
- `KeepAlive::default()`.
- The stream ends on client disconnect, which drops the receiver and stops the blocking thread, or on `state.shutdown` (review G3).
- **E5** Docker routes:
- `/docker/ps`
- `/docker/containers/{name}`
- `/docker/containers/{name}/logs?tail=&since=&follow=`: JSON array when `follow=0`, SSE when `follow=1`. Each SSE event's `event:` is `stdout` or `stderr`.
- **E6** Provide a public `healthz` handler for P3 to mount outside auth. It returns `{"status":"ok","version":…}`.
- **E7** All errors go through `AppError`, giving one JSON shape everywhere.
- **E8** Tests with `fake.rs` sources via `tower::ServiceExt::oneshot`. Cover each route's happy path, 400, 404, and one SSE stream that yields events and then ends.
- **E9** When the fake docker returns `Unavailable`, docker routes return 503 and systemd and journal routes still return 200.
## Done when
- E8 and E9 pass.
- With real sources wired by P3, Verification #3 curl checks pass, including `curl -N` on both SSE endpoints.
## Notes
- Handlers stay thin: parse, call the source, serialize. No business logic.
- `tower-http` `TraceLayer` is applied in P3, not here.
## Log
- 2026-10-15: Card created.
+60
View File
@@ -0,0 +1,60 @@
# WS-F — MCP server
| Field | Value |
|---|---|
| Status | Backlog |
| Owner | — |
| Phase | 2 (parallel) |
| Depends on | P0 to start (fakes, spike findings). A, B, C, D to finish. |
| Blocks | P3 |
| Owned files | `src/mcp/mod.rs`, `src/mcp/prompts.rs`, `src/mcp/resources.rs` |
## Goal
Build an `rmcp` Streamable HTTP server exposing the same data as REST, plus agent-facing context: instructions, rich descriptions, prompts and resources. `mcp::service(state)` returns the Tower service that P3 mounts at `/mcp`.
## Tasks
Tick these in [`TODO.md`](../../TODO.md).
- **F1** `TrackerService` holding `AppState`, with `#[tool_router]`. There are 8 tools (review R5):
- `list_systemd_units { state? }`
- `get_systemd_unit { name }`
- `list_systemd_timers {}`
- `get_journal_logs { unit?, priority?, lines }`
- `tail_journal { unit, seconds }`
- `list_containers {}`
- `get_container { name }`
- `get_container_logs { name, tail }`
- Parameter structs derive `Deserialize` and `JsonSchema`.
- **F2** `tail_journal` clamps `seconds` to `limits.max_follow_secs` and stops early at an entry cap, for example `limits.max_lines`. It returns whatever it collected and never streams indefinitely.
- **F3** `get_container_logs` clamps `tail` to `limits.max_tail`.
- **F4** `ServerHandler::get_info` sets server name/version, capabilities for tools, prompts and resources, and **instructions**. The instructions describe the workflow: start with failed units or unhealthy containers, then pull logs for the specific unit or container, and use timers to explain scheduled jobs.
- **F5** `#[tool(description = …)]` strings explain:
- active/sub states (`failed`, `activating`, `auto-restart`)
- journal priority numbers and names
- Docker health states (`starting`, `healthy`, `unhealthy`, `none`)
- **F6** Prompts in `prompts.rs`:
- `diagnose_unit { name }`: status, then recent err+ logs, then restart count, then a suggested next look.
- `container_health_report {}`: list, then unhealthy and restarting containers, then their logs.
- **F7** Resources in `resources.rs`: `systemd://units/failed` and `docker://containers`. Implement list and read, resolved live on each read, with no subscriptions (DEC-5).
- **F8** `mcp::service(state)` builds `StreamableHttpService::new(|| Ok(TrackerService::new(state.clone())), LocalSessionManager::default().into(), config)`. Apply Origin validation per DEC-8.
- **F9** Tests against fake sources:
- A snapshot of the tool list (names + schemas), so description changes are reviewed deliberately.
- Each tool's happy path.
- `AppError` maps to MCP errors.
- `tail_journal` respects its time cap.
## Done when
- F9 passes.
- With P3 wiring, Verification #4 passes. MCP Inspector at `http://localhost:9109/mcp` with a bearer header lists 8 tools, 2 prompts and 2 resources, and every tool returns data. Without the header the client receives 401.
## Notes
- Tool handlers must not duplicate logic from `api/`. Both call `sources/` directly.
- Keep tool output JSON-shaped (serialized DTOs) so agents can parse it, with a short text summary line where it helps.
## Log
- 2026-10-15: Card created.
+55
View File
@@ -0,0 +1,55 @@
# WS-G — OpenTelemetry export
| Field | Value |
|---|---|
| Status | Backlog |
| Owner | — |
| Phase | 2 (parallel) |
| Depends on | P0 to start (fakes, spike findings). A, B, C to finish. |
| Blocks | P3 |
| Owned files | `src/telemetry/mod.rs`, `src/telemetry/metrics_export.rs`, `src/telemetry/logs_export.rs`, `dev/otelcol.yaml` |
## Goal
Export traces, metrics and logs over a single OTLP exporter config. This is the only push path in the system. Metrics come from a poll loop, and logs fan out from the journal and Docker follow streams.
## Tasks
Tick these in [`TODO.md`](../../TODO.md).
- **G1** `telemetry::init(&Config) -> TelemetryGuard`:
- Build tracer, meter and logger providers from one exporter config: `otel.endpoint`, `otel.protocol` (`grpc` or `http`) and `otel.headers`.
- Resource attributes: `service.name`, `service.version`, `host.name`.
- When `otel.enabled = false`, return a no-op guard.
- **G2** Tracing subscriber: `fmt` to stderr (journald captures it under the unit) plus `tracing-opentelemetry` when enabled. Do **not** route the app's own logs into the OTel logs pipeline. That pipeline is for host logs.
- **G3** Metrics (review G11):
- A background task runs every `metrics_interval_secs`, calls `list_units`, `list_timers` and `ps`, and stores the result in an `ArcSwap` or `RwLock` snapshot.
- Observable gauges read the latest snapshot in their callbacks, so units or containers that disappear stop being reported.
- **G4** Metric set from `plan.md`:
- `systemd.unit.active`, `systemd.unit.failed`, `systemd.unit.restarts`, `systemd.timer.next_elapse_seconds`
- `docker.container.running`, `docker.container.healthy`, `docker.container.uptime_seconds`, `docker.container.restarts`
- Attribute names: `systemd.unit`, `systemd.sub_state`, `systemd.load_state`; `container.name`, `container.id`, `container.image.name` (semconv).
- Scope: apply `otel.metrics.units` / `otel.metrics.containers` globs. With an empty unit list, use the `*.service` + `*.timer` default (DEC-7).
- **G5** Logs (`otel.export_logs = true`, review G4):
- Journal: one `follow()` per configured unit, or one unfiltered follow when `otel.logs.journal_units` is empty. Map priority 0–7 to OTel severity (emerg/alert/crit to FATAL, err to ERROR, warning to WARN, notice/info to INFO, debug to DEBUG).
- Docker: a reconciler subscribes to Docker `events` (`start`, `die`, `destroy`). It starts and stops per-container `logs(follow)` tasks for allowlisted containers and does an initial list at startup. stderr maps to WARN and stdout to INFO.
- Every task restarts with exponential backoff on error. If Docker is unavailable, log once and retry slowly.
- Attributes: `systemd.unit` or `container.name`, plus `host.name` from the resource.
- **G6** On shutdown: cancel all tasks via `state.shutdown`, then `force_flush` and `shutdown` each provider, with a bounded timeout.
- **G7** `dev/otelcol.yaml` with an OTLP receiver (grpc 4317, http 4318) and a `debug` exporter at `verbosity: detailed`. Add a devenv `processes.otelcol` entry, coordinating the `devenv.nix` edit with the P0 owner.
## Done when
- Verification #5 passes. With `[otel] enabled = true`, the collector shows the gauges every interval and log records with correct severity and attributes.
- `logger -p user.err test` appears as an ERROR record.
- Starting a new container shows its logs without restarting the service.
- SIGTERM flushes pending records before exit.
## Notes
- opentelemetry-rust marks metrics and logs Stable and traces Beta. Pin versions together, because the `opentelemetry*` crates must match each other.
- Cardinality: one series per unit or container per attribute set. With the DEC-7 default, expect a few hundred series on a typical host.
## Log
- 2026-10-15: Card created.