31 KiB
Architecture
skyg-linux-tracker (binary and CLI: slinuxm) is a read-only Linux system monitor in Rust, shipped as a single binary. It exposes systemd state, journald logs and Docker container state/logs over three surfaces that share one core: a REST API, an MCP server (Streamable HTTP) and OpenTelemetry export. Everything is read live from the source on every request. Nothing is collected or buffered, except a small SQLite database holding hashed API tokens.
User-facing usage is in README.md. Work tracking is in TODO.md. Per-workstream design logs from the original build and Phase 5 are archived in docs/workstreams/.
Requirements
- Single Rust binary, read-only against the host.
- MCP over Streamable HTTP with auth, mounted on the same axum server as REST.
- Config from a TOML file,
SKYG_*env vars and CLI flags. Precedence: CLI > env > file > defaults. - OTel: unit and container status as metrics (gauges), journal and docker logs as OTel logs, plus traces for the API itself.
- REST and MCP expose the same data. MCP adds agent-facing context (instructions, tool descriptions, prompts, resources).
- State: SQLite (
/var/lib/skyg-linux-tracker/db.sqlite), access tokens only. - Optional in-process TLS (rustls). No openssl anywhere in the dependency tree.
Tech choices
| Area | Crate | Why |
|---|---|---|
| HTTP | axum 0.8 + tokio |
rmcp's HTTP transport is a Tower service that mounts onto an axum router |
| TLS | rustls (ring provider, pinned explicitly) + tokio-rustls + hyper-util |
In-process HTTPS with SIGHUP reload; no openssl |
| MCP | rmcp 3.5 (official modelcontextprotocol/rust-sdk), features server, macros, schemars, transport-streamable-http-server |
StreamableHttpService mounts at /mcp; tools via #[tool] macros; built-in Host/Origin validation |
| systemd | zbus_systemd (feature systemd1, tokio executor) |
Pure-Rust generated bindings for org.freedesktop.systemd1. No C dependency |
| journald | systemd crate, journal + systemd_v245 features (sd-journal FFI) |
Only maintained option with live tail (seek_tail, next, wait) and namespaces. Links libsystemd (present on any systemd host) |
| Docker | bollard (unix socket only) |
Async Docker/Podman API; list_containers, inspect_container (health), log streams |
| OTel | opentelemetry, opentelemetry_sdk, opentelemetry-otlp 0.33, tracing-opentelemetry |
One OTLP config (endpoint, protocol, headers) drives all three signals. Metrics and logs are Stable, traces Beta |
| Auth DB | sqlx (sqlite, bundled) with runtime queries |
Async, pure-Rust bundled SQLite, no build-time database (DEC-3) |
| Peer client | reqwest 0.13 (rustls-no-provider, ring passed in) + webpki-roots |
Outbound calls to peers; no openssl, nothing installed process-wide (DEC-14) |
| OpenAPI | utoipa 6 + schemars |
Generated REST and MCP description (DEC-15) |
| Config | figment (TOML + env) + clap |
Layered config with clear precedence |
Rust edition 2021; release profile uses LTO and strip. Cargo.lock is committed. The only dynamic dependencies are libc, libm, libgcc_s and libsystemd (check with ldd).
Repo layout
Cargo.toml, Cargo.lock # bin `slinuxm`, lib `skyg_linux_tracker`
devenv.nix # rust toolchain, pkg-config, libsystemd, otelcol; scripts build/run/smoke/token
flake.nix, nix/ # package.nix, module.nix (NixOS), tests/vm.nix (VM test)
config.example.toml # every key with its default and env name
migrations/0001_tokens.sql # embedded sqlx migration
systemd/skyg-linux-tracker.service # hardened example unit
dev/otelcol.yaml # local collector with debug exporter (`devenv up`)
examples/ # systemd_dump, journal_tail, docker_ps: run one source against the live host
docs/openapi.json # generated OpenAPI 3.1 doc (with peers), guarded by a drift test
tests/ # integration.rs, tls.rs, plain.rs, binary_smoke.rs, peers.rs, peers_config.rs, peers_registry.rs, mcp_peers.rs, openapi.rs, common/
src/
main.rs # clap CLI: serve | token create|list|revoke | peer list|check | openapi [--peers], --print-config
lib.rs
cli.rs # `slinuxm openapi`
openapi.rs # builds the OpenAPI document (route and CLI share it)
config.rs # figment layering + validation
error.rs # AppError -> HTTP status + JSON body, and -> MCP ErrorData
state.rs # AppState: sources (Arc<dyn ...>), optional DB pool, config, CancellationToken
server.rs # router assembly, accept loop, graceful shutdown
tls.rs # rustls config, cert/key load and SIGHUP reload
db.rs # sqlx pool, WAL + busy_timeout, migrations, open-or-create (dir 0700, file 0600)
auth/ # bearer middleware (mod.rs), token CRUD (tokens.rs)
sources/ # the core: one trait per domain, live reads only
mod.rs # traits + shared DTOs (UnitSummary, TimerInfo, JournalEntry, ContainerSummary, ...)
systemd.rs, journal.rs, docker.rs # real implementations
fake.rs # in-memory fakes used by handler and tool tests
peers/ # consolidator: config.rs, secrets.rs (token loader), registry.rs, health.rs, client.rs (reqwest transport), proxy.rs, transport.rs (trait), types.rs, error.rs, cli.rs, testing.rs (fakes)
api/ # REST layer: systemd.rs, journal.rs, docker.rs, ping.rs, peers.rs, mod.rs
mcp/ # TrackerService (mod.rs), prompts.rs, resources.rs
telemetry/ # mod.rs (providers, guard, peer request span), metrics_export.rs, logs_export.rs
The key structural decision: sources/ is the only place that talks to D-Bus, journald or Docker. api/, mcp/ and telemetry/ are thin projections over it, which makes "MCP and REST expose the same data" true by construction. peers/ is the one outbound HTTP client; the registry exists only when [peers] enabled = true, and REST and MCP share its proxy functions. Sources are traits held as Arc<dyn ...> in AppState (DEC-2), so every handler and tool test runs against fakes without systemd or Docker. server.rs is the only place the surfaces meet.
Configuration
See config.example.toml; every value shown is the default. Sections: [server] (bind 127.0.0.1, port 1984, allowed_hosts, allowed_origins, [server.tls]), [auth] (enabled, db_path), [otel] (enabled, endpoint, protocol, service_name, export_unit_metrics, export_logs, metrics_interval_secs, [otel.headers], [otel.logs], [otel.metrics] allowlists), [sources] (docker_socket, journal_namespace) and [limits] (max_lines, max_tail, max_follow_secs) and [peers] (off by default; see Consolidator).
Precedence: defaults < config file (--config, default /etc/skyg-linux-tracker.toml) < SKYG_* env (nesting with __, e.g. SKYG_SERVER__PORT) < CLI flags. slinuxm --print-config prints the effective config with secrets redacted. Validation rejects auth.enabled = false with a non-loopback bind, port = 0, and incomplete TLS settings.
Sources (live reads only)
- systemd (
sources/systemd.rs): one shared system-buszbus::Connection.list_units(state)usesListUnits.list_timers()lists*.timerunits (ListUnitsByPatterns) and reads theTimerinterface on each (NextElapseUSecRealtime,LastTriggerUSec,Unit);0oru64::MAXmeans not scheduled.unit_status(name)usesGetUnitonly, neverLoadUnit, so an unloaded unit is NotFound.Unitproperties are read for every unit,Serviceproperties (ExecMainStatus,NRestarts) only for.serviceunits, so those DTO fields areOption. Unit names are validated. No caching. - journal (
sources/journal.rs): all sd-journal access runs insidespawn_blocking(aJournalis neitherSendnorSyncand is opened and kept in the blocking closure).query(filter)seeks backward N entries (unit, priority <= N,linescapped bylimits.max_lines, returned oldest first).follow(filter)is a blockingwait()loop with a short timeout that feeds an mpsc channel and exits when the receiver is dropped or the shutdown token fires. BinaryMESSAGEfields are mapped lossily.sources.journal_namespaceopens a journal namespace. - docker (
sources/docker.rs): lazily constructedbollard::Dockeronsources.docker_socket. A missing or denied socket never fails startup; each call maps it toAppError::Unavailable(HTTP 503 on docker routes only).ps()lists containers and inspects them concurrently (bounded) for health, uptime, restart count and image.logs(name, follow, tail, since)is a stream that stops on drop or shutdown;tailis capped bylimits.max_tail.
REST API
All under /api/v1, GET only, behind the auth middleware when enabled. Routes use axum 0.8 {name} syntax. Errors are {"error":{"code","message"}}.
| Endpoint | Source |
|---|---|
GET /api/v1/systemd/units?state=failed |
list_units() |
GET /api/v1/systemd/units/{name} |
unit_status() |
GET /api/v1/systemd/timers |
list_timers() |
GET /api/v1/journal?unit=&priority=&lines= |
journal snapshot |
GET /api/v1/journal/follow?unit=&priority= (SSE, event journal) |
journal follow() |
GET /api/v1/docker/ps |
ps() |
GET /api/v1/docker/containers/{name} |
container_status() |
GET /api/v1/docker/containers/{name}/logs?tail=&since=&follow=0|1 (JSON, or SSE events stdout/stderr) |
logs() |
GET /api/v1/ping |
authenticated liveness, used by a consolidator's API check |
GET /api/v1/openapi.json |
generated OpenAPI 3.1 document; peer paths and tools only when peers are enabled |
GET /api/v1/peers, GET /api/v1/peers/{peer} |
peer status (only with peers.enabled) |
GET /api/v1/peers/{peer}/{*path} |
allowlisted GET pass-through to the peer; the response carries x-slinuxm-peer (only with peers.enabled) |
GET /healthz |
public liveness, {"status":"ok","version":...} |
SSE streams (axum::response::Sse with keep-alive) read from the mpsc channels of sources/ and end on client disconnect or shutdown.
MCP server
rmcp's StreamableHttpService is a Tower service nested at /mcp on the protected router (mcp::service(state)). One per-session TrackerService shares AppState. Auth failures are HTTP 401 with WWW-Authenticate: Bearer (DEC-4), the same middleware as REST.
Eight tools (ten with peers.enabled, below), each a thin wrapper over sources/ with schemars parameter schemas: list_systemd_units { state? }, get_systemd_unit { name }, list_systemd_timers, get_journal_logs { unit?, priority?, lines }, tail_journal { unit, seconds }, list_containers, get_container { name }, get_container_logs { name, tail }.
- With
peers.enabledthere are ten:list_peersandget_peer { name }are added, each of the eight takes an optionalpeerargument that reads from that peer through this instance, and the instructions gain a peers paragraph. The tool-list snapshot test guards both tables. tail_journalis a bounded follow: it clampssecondstolimits.max_follow_secs, stops at an entry cap and returns what it collected. There are no infinite MCP streams.- Log tools return
{entries, truncated, note}capped at 256 KiB. Source errors come back asCallToolResulterrors with an actionable message. Log content is documented as untrusted data. - Agent context:
ServerHandlerinstructions (start with failed units or unhealthy containers, then pull logs, use timers to explain jobs), rich tool descriptions (systemd states, health states, journal priorities), promptsdiagnose_unit { name }andcontainer_health_report, resourcessystemd://units/failedanddocker://containersplus templatessystemd://unit/{name}anddocker://container/{name}. Resources are list/read, resolved live, not subscribable (DEC-5). - Host/Origin validation uses rmcp's own
with_allowed_hosts/with_allowed_origins(DEC-8). Requests with noOriginpass, so the bearer token is the real gate.
Server wiring (server.rs, main.rs)
let protected = Router::new()
.nest("/api/v1", api::router(state))
.nest_service("/mcp", mcp::service(state))
.layer(auth::layer(state)); // bearer auth for API + MCP only
let app = public_router_with_healthz // /healthz stays unauthenticated
.merge(protected)
.layer(TraceLayer::new_for_http());
Startup order: config, telemetry init, DB open (when auth is on), sources and AppState, exporter tasks, server. On SIGINT/SIGTERM the shared CancellationToken ends SSE streams and background tasks, graceful serve drains (bounded 10 s), exporters are awaited (bounded 10 s), the telemetry guard flushes OTel and the pool closes. The unit uses TimeoutStopSec=30. With TLS and plain HTTP alike the listener is HTTP/1.1, shares one accept loop, caps concurrent connections at 1024 and closes connections that send no request header within 30 s. With auth.enabled = false (loopback only) REST requests get a Host allowlist check (403 otherwise).
TLS
Optional and off by default (DEC-12). With server.tls.enabled the server serves REST and /mcp over HTTPS only, using a PEM chain and key. Both files are validated at startup. SIGHUP re-reads them; a bad or mismatched pair keeps the old certificate and logs an error, established connections are never dropped. The default MCP Origin allowlist switches to https://. A non-loopback bind without TLS logs a startup warning but is not refused. Operator-facing details: docs/tls.md.
OpenTelemetry (telemetry/)
One OTLP exporter config (grpc or http/protobuf, optional headers) drives three signals. The resource carries service.name, service.version and host.name. OTLP is push-based, so this is the one place that polls on a timer (metrics) or fans out continuously (logs); it reuses the same sources/ functions as the other surfaces.
-
Traces: a
tracingsubscriber (fmt to stderr) plus thetracing-opentelemetrylayer when enabled, andtower-httpTraceLayerfor spans per HTTP/MCP request. -
Metrics:
metrics_export.rspolls everymetrics_interval_secsinto a snapshot. All instruments are observable gauges (f64) reading the latest snapshot, so a vanished unit or container stops being reported instead of keeping a stale value. A failed or timed-out section keeps its previous values.Metric Unit Attributes systemd.unit.active(0/1)1systemd.unit,systemd.sub_state,systemd.load_statesystemd.unit.failed(0/1)1systemd.unitsystemd.unit.restarts(services only){restart}systemd.unitsystemd.timer.next_elapse_timestamp_seconds(absolute unix time; unscheduled timers omitted)ssystemd.unitdocker.container.running(0/1)1container.name,container.image.namedocker.container.healthy(0/1, only when a healthcheck exists)1container.namedocker.container.uptime_secondsscontainer.namedocker.container.restarts{restart}container.nameScope: an empty
otel.metrics.unitsexports*.serviceand*.timeronly (DEC-7); a non-empty list is a glob allowlist. An emptyotel.metrics.containersexports all containers. -
Peer gauges (only with
peers.enabled):peer.up(peer.name,peer.check),peer.check.duration_seconds,peer.last_change_timestamp_secondsandpeer.api.auth_ok(peer.name). They read the in-memory registry snapshot at each collection, so peers added or removed by a SIGHUP reload appear without re-registering; disabled peers are left out. Outbound peer requests run in apeer.requestspan exported as a client span (otel.kind = client) and carrytraceparent. Metrics needotel.enabledandotel.export_unit_metrics. -
Logs: when
otel.export_logs = true,logs_export.rsruns a journal follow plus per-container docker follow tasks, reconciled against the Docker events stream (start/die/destroy) with backoff, so containers that come and go are tracked. Journal priority maps to OTel severity (emerg/alert/crit to FATAL, err ERROR, warning WARN, notice/info INFO, debug DEBUG); docker stderr is WARN, stdout INFO. Attributes:systemd.unit,log.syslog.identifier,process.pid,container.name,log.iostream.otel.logs.journal_units/docker_containersnarrow the scope (empty = all).
Shutdown cancels the tasks and flushes all providers.
Consolidator (peers/)
Opt-in ([peers] enabled = true): an instance reads other slinuxm instances declared in config. Without it there is no registry, no background task, no peer route or gauge, and 8 MCP tools. The registry keeps peer status in memory; two checks per peer (TCP connect, authenticated GET /api/v1/ping) run in background loops and SIGHUP reloads the peer list in place. Operator docs: docs/consolidator.md. Decisions: DEC-13, DEC-14.
OpenAPI (openapi.rs)
openapi::document(peers) is the single builder behind GET /api/v1/openapi.json and slinuxm openapi [--peers]. docs/openapi.json is the --peers output. See DEC-15.
Auth and state (auth/, db.rs)
- SQLite at
auth.db_path(default/var/lib/skyg-linux-tracker/db.sqlite). A server-created directory is 0700 and the file 0600; an existing directory is never chmod'd (a group/world-accessible one only logs a warning). WAL mode plusbusy_timeout, because the token CLI and the server share the DB. Under systemd,StateDirectory=creates the directory with the right owner. - Table
tokens(id, name UNIQUE, token_hash UNIQUE, created_at, revoked_at, last_used_at). Tokens are 32 random bytes with theslinuxm_prefix; only the sha256 hash is stored and the plaintext is printed once. No bootstrap or declarative tokens (DEC-11). - Middleware: sha256 of the presented bearer token, indexed lookup of non-revoked tokens (no separate constant-time compare is needed for a high-entropy token's hash). Failure is 401 with
WWW-Authenticate: Bearerand the JSON error body; unmatched paths also return 401 so route existence is not disclosed. Revocation is immediate. - The token CLI refuses to run when the effective uid (root included) does not own the DB file or, if absent, its state directory, and prints the matching
sudo -u <owner> slinuxm token ...command (DEC-10). Token commands do not initialise telemetry.
Build, packaging and deployment
- devenv.nix: Rust toolchain,
pkg-config,systemd(libsystemd headers for the journal FFI),opentelemetry-collector-contrib; scriptsbuild,run,smoke,token;devenv upruns the local collector. - Single binary: pure Rust except libsystemd and bundled SQLite. rustls only;
cargo tree -i opensslmust be empty. The release binary was 19 MB when measured at P3. - systemd unit (
systemd/skyg-linux-tracker.service): staticUser=skyg(notDynamicUser, sosudo -u skyg slinuxm token ...reaches the same DB),StateDirectory=mode 0700,SupplementaryGroups=systemd-journal,dockergroup commented out (DEC-6), plusProtectSystem=strict,NoNewPrivileges, syscall filter and related hardening.systemd-analyze security --offlinescores 1.5 OK. - Nix (
flake.nix,nix/):packages.default(buildRustPackagefromCargo.lock;cmakeis needed foraws-lc-sys, a transitive dependency; the pinned crypto provider for our own TLS is ring) andnixosModules.defaultunderservices.skyg-linux-tracker(user,StateDirectory, journal group, the same hardening as the unit,/etc/skyg-linux-tracker.toml, system-wideslinuxm, firewall options,LoadCredentialTLS, eval assertion that a non-loopback bind needs auth).nix flake checkruns a two-machine VM test (plain and TLS).
Decisions
All accepted. Rationale is kept; the build history is in docs/workstreams/.
| ID | Decision | Status |
|---|---|---|
| DEC-1 | Binary and CLI are slinuxm |
Decided 2026-10-15 |
| DEC-2 | Source traits plus in-memory fakes from the start | Decided 2026-10-15 |
| DEC-3 | sqlx runtime queries, no macros | Decided 2026-10-15 |
| DEC-4 | MCP auth failure is HTTP 401 | Decided 2026-10-15 |
| DEC-5 | MCP resources not subscribable in v1 | Decided 2026-10-15 |
| DEC-6 | Docker socket access model | Decided 2026-10-15 |
| DEC-7 | Default metrics scope | Decided 2026-10-15 |
| DEC-8 | Origin validation for /mcp via rmcp |
Decided 2026-10-15 |
| DEC-9 | Port 1984 and loopback bind by default | Decided 2026-10-16 |
| DEC-10 | No CLI wrapper; the app guards the root mistake | Decided 2026-10-16 |
| DEC-11 | No declarative or bootstrap tokens in v1 | Decided 2026-10-16 |
| DEC-12 | TLS terminated in-process with rustls | Decided 2026-10-16 |
| DEC-13 | Outbound peer credentials are external files | Decided 2026-10-19 |
| DEC-14 | Consolidator (peers) design | Decided 2026-10-19, implemented 2026-10-20 |
| DEC-15 | Generated OpenAPI document | Decided 2026-10-20 |
-
DEC-1 name. The plan used three names (repo
linux-system-monitor, serviceskyg-linux-tracker, CLIskyg). The binary, CLI and[[bin]]areslinuxm; tokens use theslinuxm_prefix. Unchanged: product nameskyg-linux-tracker, unitskyg-linux-tracker.service, state dir/var/lib/skyg-linux-tracker, config/etc/skyg-linux-tracker.toml, env prefixSKYG_. -
DEC-2 traits and fakes.
SystemdSource,JournalSource,DockerSourcelive insrc/sources/mod.rs, fakes insrc/sources/fake.rs,AppStateholdsArc<dyn ...>(viaasync-trait). Surfaces can be built and tested without systemd or Docker, at the cost of about 100 extra lines. -
DEC-3 runtime queries.
sqlx::query!macros needDATABASE_URLat build time or a committed.sqlx/cache. With one table and about five queries, runtimequery/query_asplus tests gives the same safety with no build-time DB. -
DEC-4 HTTP 401. The MCP authorization spec puts auth at the HTTP layer and clients expect 401 with
WWW-Authenticate: Bearer. One middleware gives the same response for/api/v1and/mcp;AppError::Unauthorizedrenders it. -
DEC-5 no subscriptions. Subscriptions need change detection, i.e. background polling, which conflicts with "live reads only". Resources are resolved live on each read.
-
DEC-6 Docker socket.
dockergroup membership is root-equivalent. The unit ships withSupplementaryGroups=dockercommented out; the hardened setup pointssources.docker_socketat a read-only socket proxy (see the README). No code change is needed either way. The NixOS module'sdocker.enableadds the group explicitly. -
DEC-7 metrics scope. "Empty allowlist = all units" would export every loaded unit (often 300+ including devices and mounts), so empty means
*.serviceand*.timer. -
DEC-8 Origin validation. The P0 spike showed rmcp 3.5 validates both headers itself:
with_allowed_hosts(default loopback names) andwith_allowed_origins(default empty means off; requests withoutOriginalways pass). No custom tower layer. Hosts: loopback names,server.bind,server.allowed_hosts. Origins:http(s)://localhost:*,127.0.0.1:*,[::1]:*,<bind>:*plusserver.allowed_origins. -
DEC-9 defaults. Binary and NixOS module both default to port 1984 and
bind = "127.0.0.1"(the binary default was 9109 until 2026-10-16). LAN exposure is an install-time choice: setbind,openFirewallor an interface-scoped rule, andallowedHosts. -
DEC-10 no wrapper. The plain
slinuxmbinary is onPATH; mint tokens withsudo -u skyg-linux-tracker slinuxm token create <name>. Running as root can create a root-owned state dir and WAL files that lock the service out, so the token commands refuse (see Auth and state). -
DEC-11 tokens. Inbound tokens only (outbound: DEC-13). Printed once, never stored in plaintext; no
bootstrapTokensoption and no import of pre-hashed tokens. -
DEC-12 TLS. Bearer tokens over plain HTTP are readable on a LAN, which is the intended deployment, so TLS is native (
server.tls.*, SIGHUP reload so ACME/sops rotation needs no restart). A reverse proxy still works. Off by default; nothing changes when it is off. -
DEC-13 outbound credentials. To call a peer the consolidator must replay a plaintext bearer token, so DEC-11's hash-only storage cannot apply.
slinuxmnever persists a peer token: config holds only atoken_filepath (on NixOS an agenix path delivered viaLoadCredential), read into memory with redactedDebugand re-read on each check so rotation needs no restart. It does not protect against root on the consolidator; mitigate with one read-only token per consolidator. Full text:docs/workstreams/P5-consolidator/decisions.md. There is one token loader,peers::secrets::load_token, used by startup validation,peer checkand every request: it accepts a regular file only, requires owner-only permissions (with an exemption for$CREDENTIALS_DIRECTORY) and rejects CR, LF and NUL so a token cannot split a header. -
DEC-14 consolidator design. Peers are declarative (config / NixOS module, SIGHUP reload, no DB table). Two checks per peer: TCP connect and an authenticated
GET /api/v1/ping(401 reportsapi.up=true, auth=rejected; older peers fall back to/healthz,auth=unknown). TCP 10s/3s, API 30s/5s, two failures to flip down, state in memory. REST: status plus a GET-only allowlisted pass-through under/api/v1/peers/{peer}/.... MCP: optionalpeerargument pluslist_peers/get_peer(10 tools only whenpeers.enabled). No aggregation and no chaining in v1. Peer failures reuseUnavailable; a peer's 401/403 is never surfaced asUnauthorized. Four OTel gauges. SSRF: config-only URLs, no redirects,deny_cidrsat connect time, webpki roots plus optionalca_file, no skip-verify, plain http to non-loopback needsallow_plain_http. Any valid inbound token may read peer data (no scopes).- HTTP client outcome.
reqwest0.13 withrustls-no-provider; the ring provider and webpki roots are passed to the client explicitly, so nothing is installed process-wide andcargo tree -i opensslstays empty. Redirects are off and the system proxy is ignored. - SSRF at connect time. The host is resolved once, every address is checked against
deny_cidrsafterto_canonical()(so::ffff:169.254.169.254matches the IPv4 range), one denied address refuses the whole name, and the checked set is pinned withresolve_to_addrs, which closes DNS rebinding. The samedeny_networks/deniedcode runs at config validation and at connect time. The defaultdeny_cidrscovers both families (169.254.0.0/16,0.0.0.0/8,fe80::/10,::/128,fd00:ec2::254/128); private LAN ranges stay allowed.
- HTTP client outcome.
-
DEC-15 generated OpenAPI. The document is generated so it cannot drift from the code. REST paths are declared on the handlers with
#[utoipa::path](utoipa 6, OpenAPI 3.1). DTO schemas are reflected from the existingschemars::JsonSchemaderives through a bridge inopenapi.rs, so one derive serves MCP and OpenAPI; only payloads that exist asserde_jsonliterals (error body, health) are written by hand and compared with real output in tests. MCP is not REST, so the tools go in thex-mcp-toolsextension, built from the tool router's own schemas. Peer paths and the two peer tools appear only when peers are enabled.tests/openapi.rsis the drift guard: every router route must be documented,x-mcp-toolsmust matchtools/list, anddocs/openapi.jsonmust equalslinuxm openapi --peers(regenerate withcargo run -- openapi --peers > docs/openapi.json). The route is authenticated like the rest of/api/v1.
Design notes and gotchas
Corrections found in the original design review and still worth knowing when changing the code.
- systemd has no
ListTimers(R1). Timers are*.timerunits plus theTimerinterface properties (see Sources). ExecMainStatusandNRestartsareServiceproperties (R2), absent for timers, sockets and mounts, henceOptionDTO fields./healthzmust stay public (R3). The auth layer sits on the protected router only, merged with a public router.- axum 0.8 routes use
/{name}(R4). The old:namesyntax panics at startup. - There are 8 MCP tools, or 10 with
peers.enabled(R5). Changing the count is a deliberate change; the tool-list snapshot test guards both tables. - Metric allowlists and
otel.headersare config keys (R6) and documented inconfig.example.toml. - No openssl (R7). Gate:
cargo tree -i opensslprints nothing. - Runtime sqlx queries (R8, DEC-3).
- Blocking sd-journal work, snapshot or follow, always runs in
spawn_blockingand never holds aJournalacross.await(G1). - Follow loops use
wait()with a timeout and exit when the receiver closes or shutdown fires, otherwise threads leak when an SSE client disconnects (G2). Every stream and background task selects on the oneCancellationToken, otherwise graceful shutdown waits forever on open SSE (G3). - Docker containers come and go, so log fan-out reconciles from the events stream (G4). A missing Docker socket is a per-request 503, never a startup failure (G9).
- Request bounds are config-driven (
limits.*) and unit/container names are validated (G7). Observable gauges fed from a snapshot avoid stale series (G11). unit_statususesGetUnitrather thanLoadUnitso a read-only tool has no side effects.rmcpaccepts requests without anOriginheader, so the bearer token is the real gate. Withbind = 0.0.0.0, LAN clients need their Host names inserver.allowed_hosts(a startup warning is logged otherwise).- The Docker group is root-equivalent even though our code only reads; prefer a read-only socket proxy.
Verification
A release is verified by hand on a real host with these six checks (last full run: minipc1, 2026-10-19; #5 is still open as I6-5 in TODO.md). With peers enabled, also run two instances against each other, cross-check a real peer, and confirm the peer gauges in the collector; the last two are still open (M6 in TODO.md).
cargo buildinsidedevenv shell; unit tests for config precedence and DTO mapping.- Run locally:
cargo run -- serve; create a token withcargo run -- token create <name>. - REST:
curl -H "Authorization: Bearer ..." localhost:1984/api/v1/systemd/units?state=failed; SSE follow endpoints viacurl -N. - MCP: point a client at
http://localhost:1984/mcp(for examplenpx @modelcontextprotocol/inspector), verify the tool list, call each tool, check rejection without the header. - OTel: run
otelcolwith a debug exporter (devenv up), enable[otel], confirm unit/container gauges appear on the poll interval and journal/docker log records flow with correct severity and attributes. - Negative tests: no token gives 401;
auth.enabled = falsewithbind = 0.0.0.0makes startup refuse; a missing docker socket gives a clean 503 on docker endpoints with the rest of the API unaffected; SIGTERM with an open SSE client exits promptly. Automated intests/integration.rs.
For NixOS, the gate is nix build and nix flake check including the VM test.