added live redoc and slight refactors to tokens
This commit is contained in:
1 parent
435a54a02d
commit
b38c66bcc8
19 files changed
+836
-157
No files matched your search
+63
-56
@@ -16,19 +16,19 @@ User-facing usage is in [`README.md`](README.md). Work tracking is in [`TODO.md`
|
||||
|
||||
## Tech choices
|
||||
|
||||
| Area | Crate | Why |
|
||||
|---|---|---|
|
||||
| HTTP | `axum` 0.8 + `tokio` | rmcp's HTTP transport is a Tower service that mounts onto an axum router |
|
||||
| TLS | `rustls` (ring provider, pinned explicitly) + `tokio-rustls` + `hyper-util` | In-process HTTPS with SIGHUP reload; no openssl |
|
||||
| MCP | `rmcp` 3.5 (official `modelcontextprotocol/rust-sdk`), features `server`, `macros`, `schemars`, `transport-streamable-http-server` | `StreamableHttpService` mounts at `/mcp`; tools via `#[tool]` macros; built-in Host/Origin validation |
|
||||
| systemd | `zbus_systemd` (feature `systemd1`, tokio executor) | Pure-Rust generated bindings for `org.freedesktop.systemd1`. No C dependency |
|
||||
| journald | `systemd` crate, `journal` + `systemd_v245` features (sd-journal FFI) | Only maintained option with live tail (`seek_tail`, `next`, `wait`) and namespaces. Links `libsystemd` (present on any systemd host) |
|
||||
| Docker | `bollard` (unix socket only) | Async Docker/Podman API; `list_containers`, `inspect_container` (health), log streams |
|
||||
| OTel | `opentelemetry`, `opentelemetry_sdk`, `opentelemetry-otlp` 0.33, `tracing-opentelemetry` | One OTLP config (endpoint, protocol, headers) drives all three signals. Metrics and logs are Stable, traces Beta |
|
||||
| Auth DB | `sqlx` (sqlite, bundled) with **runtime** queries | Async, pure-Rust bundled SQLite, no build-time database (DEC-3) |
|
||||
| Peer client | `reqwest` 0.13 (`rustls-no-provider`, ring passed in) + `webpki-roots` | Outbound calls to peers; no openssl, nothing installed process-wide (DEC-14) |
|
||||
| OpenAPI | `utoipa` 6 + `schemars` | Generated REST and MCP description (DEC-15) |
|
||||
| Config | `figment` (TOML + env) + `clap` | Layered config with clear precedence |
|
||||
| Area | Crate | Why |
|
||||
| ----------- | ---------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| HTTP | `axum` 0.8 + `tokio` | rmcp's HTTP transport is a Tower service that mounts onto an axum router |
|
||||
| TLS | `rustls` (ring provider, pinned explicitly) + `tokio-rustls` + `hyper-util` | In-process HTTPS with SIGHUP reload; no openssl |
|
||||
| MCP | `rmcp` 3.5 (official `modelcontextprotocol/rust-sdk`), features `server`, `macros`, `schemars`, `transport-streamable-http-server` | `StreamableHttpService` mounts at `/mcp`; tools via `#[tool]` macros; built-in Host/Origin validation |
|
||||
| systemd | `zbus_systemd` (feature `systemd1`, tokio executor) | Pure-Rust generated bindings for `org.freedesktop.systemd1`. No C dependency |
|
||||
| journald | `systemd` crate, `journal` + `systemd_v245` features (sd-journal FFI) | Only maintained option with live tail (`seek_tail`, `next`, `wait`) and namespaces. Links `libsystemd` (present on any systemd host) |
|
||||
| Docker | `bollard` (unix socket only) | Async Docker/Podman API; `list_containers`, `inspect_container` (health), log streams |
|
||||
| OTel | `opentelemetry`, `opentelemetry_sdk`, `opentelemetry-otlp` 0.33, `tracing-opentelemetry` | One OTLP config (endpoint, protocol, headers) drives all three signals. Metrics and logs are Stable, traces Beta |
|
||||
| Auth DB | `sqlx` (sqlite, bundled) with **runtime** queries | Async, pure-Rust bundled SQLite, no build-time database (DEC-3) |
|
||||
| Peer client | `reqwest` 0.13 (`rustls-no-provider`, ring passed in) + `webpki-roots` | Outbound calls to peers; no openssl, nothing installed process-wide (DEC-14) |
|
||||
| OpenAPI | `utoipa` 6 + `schemars` | Generated REST and MCP description (DEC-15) |
|
||||
| Config | `figment` (TOML + env) + `clap` | Layered config with clear precedence |
|
||||
|
||||
Rust edition 2021; release profile uses LTO and `strip`. Cargo.lock is committed. The only dynamic dependencies are libc, libm, libgcc_s and libsystemd (check with `ldd`).
|
||||
|
||||
@@ -50,6 +50,7 @@ src/
|
||||
lib.rs
|
||||
cli.rs # `slinuxm openapi`
|
||||
openapi.rs # builds the OpenAPI document (route and CLI share it)
|
||||
docs.rs # Redoc viewer at /docs; assets (page, loader, vendored Redoc 2.5.4 and its licenses) in src/docs/
|
||||
config.rs # figment layering + validation
|
||||
error.rs # AppError -> HTTP status + JSON body, and -> MCP ErrorData
|
||||
state.rs # AppState: sources (Arc<dyn ...>), optional DB pool, config, CancellationToken
|
||||
@@ -85,21 +86,22 @@ Precedence: defaults < config file (`--config`, default `/etc/skyg-linux-tracker
|
||||
|
||||
All under `/api/v1`, GET only, behind the auth middleware when enabled. Routes use axum 0.8 `{name}` syntax. Errors are `{"error":{"code","message"}}`.
|
||||
|
||||
| Endpoint | Source |
|
||||
|---|---|
|
||||
| `GET /api/v1/systemd/units?state=failed` | `list_units()` |
|
||||
| `GET /api/v1/systemd/units/{name}` | `unit_status()` |
|
||||
| `GET /api/v1/systemd/timers` | `list_timers()` |
|
||||
| `GET /api/v1/journal?unit=&priority=&lines=` | journal snapshot |
|
||||
| `GET /api/v1/journal/follow?unit=&priority=` (SSE, event `journal`) | journal `follow()` |
|
||||
| `GET /api/v1/docker/ps` | `ps()` |
|
||||
| `GET /api/v1/docker/containers/{name}` | `container_status()` |
|
||||
| `GET /api/v1/docker/containers/{name}/logs?tail=&since=&follow=0\|1` (JSON, or SSE events `stdout`/`stderr`) | `logs()` |
|
||||
| `GET /api/v1/ping` | authenticated liveness, used by a consolidator's API check |
|
||||
| `GET /api/v1/openapi.json` | generated OpenAPI 3.1 document; peer paths and tools only when peers are enabled |
|
||||
| `GET /api/v1/peers`, `GET /api/v1/peers/{peer}` | peer status (only with `peers.enabled`) |
|
||||
| `GET /api/v1/peers/{peer}/{*path}` | allowlisted GET pass-through to the peer; the response carries `x-slinuxm-peer` (only with `peers.enabled`) |
|
||||
| `GET /healthz` | public liveness, `{"status":"ok","version":...}` |
|
||||
| Endpoint | Source |
|
||||
| ------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------- |
|
||||
| `GET /api/v1/systemd/units?state=failed` | `list_units()` |
|
||||
| `GET /api/v1/systemd/units/{name}` | `unit_status()` |
|
||||
| `GET /api/v1/systemd/timers` | `list_timers()` |
|
||||
| `GET /api/v1/journal?unit=&priority=&lines=` | journal snapshot |
|
||||
| `GET /api/v1/journal/follow?unit=&priority=` (SSE, event `journal`) | journal `follow()` |
|
||||
| `GET /api/v1/docker/ps` | `ps()` |
|
||||
| `GET /api/v1/docker/containers/{name}` | `container_status()` |
|
||||
| `GET /api/v1/docker/containers/{name}/logs?tail=&since=&follow=0\|1` (JSON, or SSE events `stdout`/`stderr`) | `logs()` |
|
||||
| `GET /api/v1/ping` | authenticated liveness, used by a consolidator's API check |
|
||||
| `GET /api/v1/openapi.json` | generated OpenAPI 3.1 document; peer paths and tools only when peers are enabled |
|
||||
| `GET /api/v1/peers`, `GET /api/v1/peers/{peer}` | peer status (only with `peers.enabled`) |
|
||||
| `GET /api/v1/peers/{peer}/{*path}` | allowlisted GET pass-through to the peer; the response carries `x-slinuxm-peer` (only with `peers.enabled`) |
|
||||
| `GET /healthz` | public liveness, `{"status":"ok","version":...}` |
|
||||
| `GET /docs`, `GET /docs/redoc.standalone.js` | public static Redoc viewer of the OpenAPI document; the script is embedded in the binary (DEC-16) |
|
||||
|
||||
SSE streams (`axum::response::Sse` with keep-alive) read from the mpsc channels of `sources/` and end on client disconnect or shutdown.
|
||||
|
||||
@@ -122,7 +124,7 @@ let protected = Router::new()
|
||||
.nest("/api/v1", api::router(state))
|
||||
.nest_service("/mcp", mcp::service(state))
|
||||
.layer(auth::layer(state)); // bearer auth for API + MCP only
|
||||
let app = public_router_with_healthz // /healthz stays unauthenticated
|
||||
let app = public_router_with_healthz // /healthz and the static /docs viewer stay unauthenticated
|
||||
.merge(protected)
|
||||
.layer(TraceLayer::new_for_http());
|
||||
```
|
||||
@@ -140,18 +142,19 @@ One OTLP exporter config (grpc or http/protobuf, optional headers) drives three
|
||||
1. **Traces**: a `tracing` subscriber (fmt to stderr) plus the `tracing-opentelemetry` layer when enabled, and `tower-http` `TraceLayer` for spans per HTTP/MCP request.
|
||||
2. **Metrics**: `metrics_export.rs` polls every `metrics_interval_secs` into a snapshot. All instruments are **observable gauges (f64)** reading the latest snapshot, so a vanished unit or container stops being reported instead of keeping a stale value. A failed or timed-out section keeps its previous values.
|
||||
|
||||
| Metric | Unit | Attributes |
|
||||
|---|---|---|
|
||||
| `systemd.unit.active` (0/1) | `1` | `systemd.unit`, `systemd.sub_state`, `systemd.load_state` |
|
||||
| `systemd.unit.failed` (0/1) | `1` | `systemd.unit` |
|
||||
| `systemd.unit.restarts` (services only) | `{restart}` | `systemd.unit` |
|
||||
| `systemd.timer.next_elapse_timestamp_seconds` (absolute unix time; unscheduled timers omitted) | `s` | `systemd.unit` |
|
||||
| `docker.container.running` (0/1) | `1` | `container.name`, `container.image.name` |
|
||||
| `docker.container.healthy` (0/1, only when a healthcheck exists) | `1` | `container.name` |
|
||||
| `docker.container.uptime_seconds` | `s` | `container.name` |
|
||||
| `docker.container.restarts` | `{restart}` | `container.name` |
|
||||
| Metric | Unit | Attributes |
|
||||
| ---------------------------------------------------------------------------------------------- | ----------- | --------------------------------------------------------- |
|
||||
| `systemd.unit.active` (0/1) | `1` | `systemd.unit`, `systemd.sub_state`, `systemd.load_state` |
|
||||
| `systemd.unit.failed` (0/1) | `1` | `systemd.unit` |
|
||||
| `systemd.unit.restarts` (services only) | `{restart}` | `systemd.unit` |
|
||||
| `systemd.timer.next_elapse_timestamp_seconds` (absolute unix time; unscheduled timers omitted) | `s` | `systemd.unit` |
|
||||
| `docker.container.running` (0/1) | `1` | `container.name`, `container.image.name` |
|
||||
| `docker.container.healthy` (0/1, only when a healthcheck exists) | `1` | `container.name` |
|
||||
| `docker.container.uptime_seconds` | `s` | `container.name` |
|
||||
| `docker.container.restarts` | `{restart}` | `container.name` |
|
||||
|
||||
Scope: an empty `otel.metrics.units` exports `*.service` and `*.timer` only (DEC-7); a non-empty list is a glob allowlist. An empty `otel.metrics.containers` exports all containers.
|
||||
|
||||
3. **Peer gauges** (only with `peers.enabled`): `peer.up` (`peer.name`, `peer.check`), `peer.check.duration_seconds`, `peer.last_change_timestamp_seconds` and `peer.api.auth_ok` (`peer.name`). They read the in-memory registry snapshot at each collection, so peers added or removed by a SIGHUP reload appear without re-registering; disabled peers are left out. Outbound peer requests run in a `peer.request` span exported as a client span (`otel.kind = client`) and carry `traceparent`. Metrics need `otel.enabled` and `otel.export_unit_metrics`.
|
||||
4. **Logs**: when `otel.export_logs = true`, `logs_export.rs` runs a journal follow plus per-container docker follow tasks, reconciled against the Docker events stream (start/die/destroy) with backoff, so containers that come and go are tracked. Journal priority maps to OTel severity (emerg/alert/crit to FATAL, err ERROR, warning WARN, notice/info INFO, debug DEBUG); docker stderr is WARN, stdout INFO. Attributes: `systemd.unit`, `log.syslog.identifier`, `process.pid`, `container.name`, `log.iostream`. `otel.logs.journal_units` / `docker_containers` narrow the scope (empty = all).
|
||||
|
||||
@@ -165,6 +168,8 @@ Opt-in (`[peers] enabled = true`): an instance reads other `slinuxm` instances d
|
||||
|
||||
`openapi::document(peers)` is the single builder behind `GET /api/v1/openapi.json` and `slinuxm openapi [--peers]`. `docs/openapi.json` is the `--peers` output. See DEC-15.
|
||||
|
||||
`docs.rs` serves the Redoc viewer (`/docs`, `/docs/redoc.standalone.js`) from assets under `src/docs/` embedded with `include_str!`/`include_bytes!`. See DEC-16.
|
||||
|
||||
## Auth and state (`auth/`, `db.rs`)
|
||||
|
||||
- SQLite at `auth.db_path` (default `/var/lib/skyg-linux-tracker/db.sqlite`). A server-created directory is 0700 and the file 0600; an existing directory is never chmod'd (a group/world-accessible one only logs a warning). WAL mode plus `busy_timeout`, because the token CLI and the server share the DB. Under systemd, `StateDirectory=` creates the directory with the right owner.
|
||||
@@ -183,23 +188,24 @@ Opt-in (`[peers] enabled = true`): an instance reads other `slinuxm` instances d
|
||||
|
||||
All accepted. Rationale is kept; the build history is in [`docs/workstreams/`](docs/workstreams/).
|
||||
|
||||
| ID | Decision | Status |
|
||||
|---|---|---|
|
||||
| DEC-1 | Binary and CLI are `slinuxm` | Decided 2026-10-15 |
|
||||
| DEC-2 | Source traits plus in-memory fakes from the start | Decided 2026-10-15 |
|
||||
| DEC-3 | sqlx runtime queries, no macros | Decided 2026-10-15 |
|
||||
| DEC-4 | MCP auth failure is HTTP 401 | Decided 2026-10-15 |
|
||||
| DEC-5 | MCP resources not subscribable in v1 | Decided 2026-10-15 |
|
||||
| DEC-6 | Docker socket access model | Decided 2026-10-15 |
|
||||
| DEC-7 | Default metrics scope | Decided 2026-10-15 |
|
||||
| DEC-8 | Origin validation for `/mcp` via rmcp | Decided 2026-10-15 |
|
||||
| DEC-9 | Port 1984 and loopback bind by default | Decided 2026-10-16 |
|
||||
| DEC-10 | No CLI wrapper; the app guards the root mistake | Decided 2026-10-16 |
|
||||
| DEC-11 | No declarative or bootstrap tokens in v1 | Decided 2026-10-16 |
|
||||
| DEC-12 | TLS terminated in-process with rustls | Decided 2026-10-16 |
|
||||
| DEC-13 | Outbound peer credentials are external files | Decided 2026-10-19 |
|
||||
| DEC-14 | Consolidator (peers) design | Decided 2026-10-19, implemented 2026-10-20 |
|
||||
| DEC-15 | Generated OpenAPI document | Decided 2026-10-20 |
|
||||
| ID | Decision | Status |
|
||||
| ------ | ------------------------------------------------- | ------------------------------------------ |
|
||||
| DEC-1 | Binary and CLI are `slinuxm` | Decided 2026-10-15 |
|
||||
| DEC-2 | Source traits plus in-memory fakes from the start | Decided 2026-10-15 |
|
||||
| DEC-3 | sqlx runtime queries, no macros | Decided 2026-10-15 |
|
||||
| DEC-4 | MCP auth failure is HTTP 401 | Decided 2026-10-15 |
|
||||
| DEC-5 | MCP resources not subscribable in v1 | Decided 2026-10-15 |
|
||||
| DEC-6 | Docker socket access model | Decided 2026-10-15 |
|
||||
| DEC-7 | Default metrics scope | Decided 2026-10-15 |
|
||||
| DEC-8 | Origin validation for `/mcp` via rmcp | Decided 2026-10-15 |
|
||||
| DEC-9 | Port 1984 and loopback bind by default | Decided 2026-10-16 |
|
||||
| DEC-10 | No CLI wrapper; the app guards the root mistake | Decided 2026-10-16 |
|
||||
| DEC-11 | No declarative or bootstrap tokens in v1 | Decided 2026-10-16 |
|
||||
| DEC-12 | TLS terminated in-process with rustls | Decided 2026-10-16 |
|
||||
| DEC-13 | Outbound peer credentials are external files | Decided 2026-10-19 |
|
||||
| DEC-14 | Consolidator (peers) design | Decided 2026-10-19, implemented 2026-10-20 |
|
||||
| DEC-15 | Generated OpenAPI document | Decided 2026-10-20 |
|
||||
| DEC-16 | Redoc viewer served by the binary | Decided 2026-10-20 |
|
||||
|
||||
- **DEC-1 name.** The plan used three names (repo `linux-system-monitor`, service `skyg-linux-tracker`, CLI `skyg`). The binary, CLI and `[[bin]]` are `slinuxm`; tokens use the `slinuxm_` prefix. Unchanged: product name `skyg-linux-tracker`, unit `skyg-linux-tracker.service`, state dir `/var/lib/skyg-linux-tracker`, config `/etc/skyg-linux-tracker.toml`, env prefix `SKYG_`.
|
||||
- **DEC-2 traits and fakes.** `SystemdSource`, `JournalSource`, `DockerSource` live in `src/sources/mod.rs`, fakes in `src/sources/fake.rs`, `AppState` holds `Arc<dyn ...>` (via `async-trait`). Surfaces can be built and tested without systemd or Docker, at the cost of about 100 extra lines.
|
||||
@@ -219,6 +225,7 @@ All accepted. Rationale is kept; the build history is in [`docs/workstreams/`](d
|
||||
- **HTTP client outcome.** `reqwest` 0.13 with `rustls-no-provider`; the ring provider and webpki roots are passed to the client explicitly, so nothing is installed process-wide and `cargo tree -i openssl` stays empty. Redirects are off and the system proxy is ignored.
|
||||
- **SSRF at connect time.** The host is resolved once, every address is checked against `deny_cidrs` after `to_canonical()` (so `::ffff:169.254.169.254` matches the IPv4 range), one denied address refuses the whole name, and the checked set is pinned with `resolve_to_addrs`, which closes DNS rebinding. The same `deny_networks` / `denied` code runs at config validation and at connect time. The default `deny_cidrs` covers both families (`169.254.0.0/16`, `0.0.0.0/8`, `fe80::/10`, `::/128`, `fd00:ec2::254/128`); private LAN ranges stay allowed.
|
||||
- **DEC-15 generated OpenAPI.** The document is generated so it cannot drift from the code. REST paths are declared on the handlers with `#[utoipa::path]` (utoipa 6, OpenAPI 3.1). DTO schemas are reflected from the existing `schemars::JsonSchema` derives through a bridge in `openapi.rs`, so one derive serves MCP and OpenAPI; only payloads that exist as `serde_json` literals (error body, health) are written by hand and compared with real output in tests. MCP is not REST, so the tools go in the `x-mcp-tools` extension, built from the tool router's own schemas. Peer paths and the two peer tools appear only when peers are enabled. `tests/openapi.rs` is the drift guard: every router route must be documented, `x-mcp-tools` must match `tools/list`, and `docs/openapi.json` must equal `slinuxm openapi --peers` (regenerate with `cargo run -- openapi --peers > docs/openapi.json`). The route is authenticated like the rest of `/api/v1`.
|
||||
- **DEC-16 Redoc viewer.** `GET /docs` renders the generated document with Redoc, served by the binary. The Redoc 2.5.4 standalone bundle is vendored under `src/docs/` (with its licenses) and embedded, so the page makes no CDN or other outbound request. It is stored gzipped (`redoc.standalone.js.gz`, about 320 KiB, under the 512 KiB hygiene limit) and sent as is with `Content-Encoding: gzip`; the refresh recipe is in the `docs.rs` module doc. 2.x was chosen over 3.x because 3.x is an ES module with a telemetry option; the 2.x bundle is a plain script. The viewer routes are public because a browser navigation cannot carry a bearer header, but they are static and hold no host data: the page asks for a token, keeps it in `sessionStorage` and sends it only to `GET /api/v1/openapi.json`, which stays authenticated. The page's one inline script is admitted by a CSP `sha256` computed from the same constant, so the policy cannot drift (`default-src 'none'`, `connect-src 'self'`, `form-action 'none'`, `frame-ancestors 'none'`; `style-src 'unsafe-inline'` and `worker-src blob:` are needed by Redoc itself). Both routes are in the OpenAPI document, so the drift test covers them. Redoc does not render the `x-mcp-tools` extension; the tools are in the JSON only.
|
||||
|
||||
## Design notes and gotchas
|
||||
|
||||
|
||||
Reference in new issue
Block a user