2026-10-20 15:54:13 -04:00
2026-10-19 14:14:29 -04:00
2026-10-15 17:07:03 -04:00
2026-10-15 17:07:03 -04:00
2026-10-15 16:33:07 -04:00
2026-10-20 15:54:03 -04:00

skyg-linux-tracker

A small, read-only monitor for a single Linux host. One binary, slinuxm, serves:

  • a REST API (with Server-Sent Events for follow endpoints),
  • an MCP server (Streamable HTTP) so agents can inspect the host,
  • optional OpenTelemetry export (metrics and logs over OTLP).

Data sources are systemd (D-Bus), the journal (libsystemd) and the Docker Engine API (unix socket). Nothing here can start, stop or modify anything.

Quick start

Try it on your own machine in a minute. It needs no root and no install, and it keeps its token database in a temp directory.

devenv shell && cargo build --release          # or just: cargo build --release with the prerequisites below
export SKYG_AUTH__DB_PATH=/tmp/slinuxm-demo/db.sqlite
BIN=target/release/slinuxm

TOKEN=$($BIN token create demo)                # printed once, never stored in plaintext
$BIN serve &                                   # http://127.0.0.1:1984, loopback only

curl -s http://127.0.0.1:1984/healthz          # public
curl -s -H "Authorization: Bearer $TOKEN" 'http://127.0.0.1:1984/api/v1/systemd/units?state=failed'
curl -s -H "Authorization: Bearer $TOKEN" 'http://127.0.0.1:1984/api/v1/journal?priority=err&lines=20'
curl -s -H "Authorization: Bearer $TOKEN" http://127.0.0.1:1984/api/v1/docker/ps   # 503 if Docker is not running

Point an MCP client at http://127.0.0.1:1984/mcp with the same bearer header (see MCP). Stop the server with kill %1.

The service listens on loopback by default. To reach it from another machine set server.bind, keep auth on, and use TLS. For a permanent install as a systemd service, see Install and Tokens.

Build

devenv shell                 # toolchain, pkg-config, libsystemd, otelcol-contrib
cargo build --release        # target/release/slinuxm

Without devenv you need a Rust toolchain, pkg-config and the libsystemd development files.

Install

sudo install -m 0755 target/release/slinuxm /usr/local/bin/slinuxm
sudo useradd --system --no-create-home --shell /usr/sbin/nologin skyg
sudo install -m 0644 config.example.toml /etc/skyg-linux-tracker.toml   # then edit
sudo install -m 0644 systemd/skyg-linux-tracker.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now skyg-linux-tracker

The unit runs as the skyg user with SupplementaryGroups=systemd-journal and a private StateDirectory (/var/lib/skyg-linux-tracker, mode 0700) holding the token database. Its sandbox options are listed in the unit file.

Config

See config.example.toml; every value shown is the default. Precedence, lowest to highest: defaults, config file, SKYG_* environment variables (nesting with __, for example SKYG_SERVER__PORT), CLI flags. slinuxm --print-config prints the effective configuration with secrets redacted.

Startup refuses auth.enabled = false on a non-loopback server.bind.

Tokens

All /api/v1/* and /mcp requests need Authorization: Bearer <token>. /healthz is public. Tokens are stored hashed (sha256) in SQLite, and the plaintext is shown once.

Run the token CLI as the service user (the unit's User=skyg), i.e. sudo -u <service user> slinuxm token create <name>. The CLI refuses to run (with the matching sudo -u <owner> ... hint) when the effective uid, root included, does not own the database file or, if it does not exist yet, its existing state directory, so it cannot create or modify state owned by another user:

sudo -u skyg slinuxm token create my-laptop --config /etc/skyg-linux-tracker.toml
sudo -u skyg slinuxm token list    --config /etc/skyg-linux-tracker.toml
sudo -u skyg slinuxm token revoke my-laptop --config /etc/skyg-linux-tracker.toml

Revocation takes effect immediately; no restart is needed.

Notes:

  • auth.db_path must stay under the unit's StateDirectory (/var/lib/skyg-linux-tracker). The unit uses ProtectSystem=strict, so any other location needs a matching ReadWritePaths=.
  • token create needs the state directory to exist, so start the service once first (systemd creates it with the right owner), then run the CLI as skyg.
  • A state directory the server creates itself is made 0700 and the database file 0600 from creation. An existing directory is never chmod'd; a group/world-accessible one only logs a warning.

REST

Base: http://127.0.0.1:1984, all routes are GET and read-only. GET /healthz is public; everything under /api/v1 needs the bearer token. Routes: systemd/units, systemd/units/{name}, systemd/timers, journal, journal/follow (SSE), docker/ps, docker/containers/{name}, docker/containers/{name}/logs (JSON or SSE). Docker routes return 503 when Docker is unavailable. Errors are {"error":{"code","message"}}.

curl -H "Authorization: Bearer $TOKEN" 'http://127.0.0.1:1984/api/v1/systemd/units?state=failed'
curl -N -H "Authorization: Bearer $TOKEN" 'http://127.0.0.1:1984/api/v1/journal/follow?priority=err'

Full route table with parameters: docs/api.md.

The whole surface is also a generated OpenAPI 3.1 document, docs/openapi.json: GET /api/v1/openapi.json serves it (same bearer token), slinuxm openapi [--peers] prints it offline without reading any config or token, and x-mcp-tools in it lists every MCP tool with its JSON Schema. Regenerate with cargo run -- openapi --peers > docs/openapi.json; a test fails if the file is stale.

MCP

Endpoint: http://127.0.0.1:1984/mcp (Streamable HTTP, same bearer token). Eight tools (list_systemd_units, get_systemd_unit, list_systemd_timers, get_journal_logs, tail_journal, list_containers, get_container, get_container_logs), two prompts, two resources and two resource templates. A consolidator adds list_peers and get_peer and a peer argument on every tool, for ten tools. Log content is untrusted data and is labelled as such.

Claude Code: claude mcp add --transport http linux-tracker http://127.0.0.1:1984/mcp --header "Authorization: Bearer $TOKEN".

/mcp validates Origin and Host against DNS rebinding. If you bind to 0.0.0.0 or serve behind a name, add the names clients use to server.allowed_hosts (for example ["monitor.lan:1984"]) and browser origins to server.allowed_origins. Generic client config, Inspector and curl handshake, and the full Origin/Host rules: docs/api.md.

TLS

TLS is optional and off by default; a non-loopback bind without TLS logs a startup warning. Enable it with a PEM chain and key:

[server.tls]
enabled = true
cert_file = "/etc/skyg/tls/cert.pem"   # PEM chain, leaf first
key_file = "/etc/skyg/tls/key.pem"     # PEM, PKCS#8, PKCS#1 or SEC1

REST and /mcp are then served over HTTPS only. Replace the files and send SIGHUP to rotate without dropping connections (a bad pair keeps the old certificate). rustls only; no openssl. Self-signed example, rotation caveats, ACME and sops paths: docs/tls.md.

Peers (consolidator)

One slinuxm can watch other instances: declare them as peers and it health-checks each one and passes read-only requests through to it, so one MCP endpoint or one REST base URL covers the fleet. Off by default; a peer-less instance keeps exactly the same routes, eight MCP tools, metrics and background tasks as before.

[peers]
enabled = true                  # other intervals, timeouts and caps: config.example.toml

[[peers.peer]]
name = "minipc1"                # ^[a-z0-9][a-z0-9_-]{0,31}$, at most 64 peers
url = "https://minipc1:1984"    # http:// to a non-loopback host also needs allow_plain_http = true
token_file = "/run/credentials/skyg-linux-tracker.service/peer-minipc1"
ca_file = "/etc/skyg/private-ca.pem"   # optional extra trust anchor

[[peers.peer]]
name = "nas"
url = "http://nas:1984"
token_file = "/etc/skyg/peer-nas.token"   # 0600, owned by the service user
allow_plain_http = true

token_file holds a path, never the token. A consolidator has to replay the peer's plaintext bearer token, so the token lives in a file managed outside slinuxm (agenix on NixOS), is read into memory on every check and request, and never appears in the config, --print-config, the database or the logs. Mint one token per consolidator on every peer and use it only there (sudo -u <service user> slinuxm token create consolidator-<host>), so a single token revoke on a peer cuts that consolidator off and nothing else. Replacing the file is enough to rotate: the next check reads the new value, and no restart is needed. On NixOS also run systemctl reload skyg-linux-tracker so systemd re-copies the credential the unit holds (RefreshOnReload=credentials).

Two checks per peer, kept in memory and reset on restart:

  • server: a TCP connect to the peer's port.
  • API: an authenticated GET /api/v1/ping. A 401 means the API runs and our token does not: api.up = true with api.auth = "rejected". That is reported to you as a 503 naming the peer, never as a 401 that would suggest re-authenticating with this host. A peer without the ping route falls back to public /healthz and reports api.auth = "unknown".
curl -H "Authorization: Bearer $TOKEN" http://127.0.0.1:1984/api/v1/peers
curl -H "Authorization: Bearer $TOKEN" 'http://127.0.0.1:1984/api/v1/peers/minipc1/systemd/units?state=failed'
sudo -u skyg slinuxm peer list --config /etc/skyg-linux-tracker.toml
sudo -u skyg slinuxm peer check minipc1 --config /etc/skyg-linux-tracker.toml

GET /api/v1/peers lists every peer, GET /api/v1/peers/{peer} one of them. GET /api/v1/peers/{peer}/... is a GET-only, allowlisted pass-through (systemd/..., journal, journal/follow, docker/...): the peer's answer comes back unchanged with x-slinuxm-peer: <peer>, a peer that is down, rejects our token or times out is a 503 naming it, x-slinuxm-hop from a caller is a 400, and unknown peers are a 404. With peers enabled MCP grows list_peers and get_peer, and every other tool accepts an optional peer argument (ten tools instead of eight). There is no aggregation across peers in this version: ask one peer at a time. OTel gains peer.up, peer.api.auth_ok, peer.check.duration_seconds and peer.last_change_timestamp_seconds.

On NixOS the module declares peers and delivers each token through systemd LoadCredential, so the service user never needs to read the agenix output:

services.skyg-linux-tracker.peers = {
  enable = true;
  peers.minipc1 = {
    url = "https://minipc1:1984";
    tokenFile = config.age.secrets.peer-minipc1.path;   # a string path, never a path literal
  };
};

Worked agenix setup, the threat model, the firewall note (every peer must accept the consolidator's address on its port), reload and shutdown semantics: docs/consolidator.md. The peers tests are tests/peers.rs (in-process, no VM) and the three-machine NixOS VM test: devenv shell -- dev-test integration peers, and for the VM test devenv shell -- dev-test vm peers, which is nix build .#checks.x86_64-linux.vm-peers -L.

NixOS

The flake exports packages.<system>.default and nixosModules.default.

# flake.nix
inputs.skyg-linux-tracker.url = "github:<owner>/linux-system-monitor";

# in your nixosSystem modules
modules = [
  skyg-linux-tracker.nixosModules.default
  { services.skyg-linux-tracker.enable = true; }
];

That runs the service as the skyg-linux-tracker user on 127.0.0.1:1984 with the same sandbox as systemd/skyg-linux-tracker.service, writes /etc/skyg-linux-tracker.toml (the path the CLI reads by default) and installs slinuxm system-wide. Mint the first token as the service user:

sudo -u skyg-linux-tracker slinuxm token create my-laptop

Options (under services.skyg-linux-tracker): port (1984), bind ("127.0.0.1"), tls.{enable,certFile,keyFile} (passed through LoadCredential, so the service user never reads the original key; systemctl reload skyg-linux-tracker re-reads them; for ACME add security.acme.certs.<name>.reloadServices = [ "skyg-linux-tracker" ]), openFirewall (all interfaces), firewallInterfaces (recommended for a LAN), allowedHosts, docker.enable (adds the root-equivalent docker group), settings (freeform TOML merged over the module defaults) and environmentFile (secrets such as SKYG_OTEL__HEADERS__AUTHORIZATION, kept out of the store and out of the generated TOML).

Exposure: keep the default loopback bind and reach it over Tailscale or a TLS reverse proxy. Bearer tokens over plain HTTP are readable on a LAN. A non-loopback bind with auth.enabled = false fails at evaluation time. For bind = "0.0.0.0" add the names clients use to allowedHosts (networking.hostName is added for you), and prefer firewallInterfaces = [ "enp3s0" ] over openFirewall.

services.skyg-linux-tracker = {
  enable = true;
  bind = "0.0.0.0";
  firewallInterfaces = [ "enp3s0" ];
  allowedHosts = [ "monitor.lan:1984" ];
  tls = { enable = true; certFile = "/var/lib/acme/monitor.lan/fullchain.pem"; keyFile = "/var/lib/acme/monitor.lan/key.pem"; };
};

nix flake check runs a two-machine VM test (plain and TLS) covering healthz, 401, token mint and revoke through the CLI, /mcp and the Origin check.

Docker access

Reading the Docker socket directly requires group docker, which is root-equivalent. The unit file has this commented out. Prefer a read-only socket proxy that allows only GET /containers/*, /events and /version, for example tecnativa/docker-socket-proxy with CONTAINERS=1 EVENTS=1 POST=0, exposed as a unix socket (or loopback TCP through a socat bridge), then:

[sources]
docker_socket = "/run/docker-proxy/docker.sock"

A stock install does not add the docker group, so out of the box the Docker routes return 503 until you either grant access or configure a proxy. If the socket is missing or unreadable, Docker routes return 503 and everything else keeps working.

OpenTelemetry

[otel]
enabled = true
endpoint = "http://localhost:4317"   # gRPC; or http://localhost:4318 with protocol = "http/protobuf"
export_logs = true

Metrics are polled every metrics_interval_secs (systemd unit active/failed/restarts and timer next elapse; container running/healthy/uptime/restarts). Logs fan out from the journal and from Docker containers. For a local collector with a debug exporter: devenv up (runs otelcol-contrib --config dev/otelcol.yaml). On SIGTERM pending records are flushed before exit.

Verifying a release binary

ldd /usr/local/bin/slinuxm    # libc, libm, libgcc_s, libsystemd and its dependencies only

Testing

Checks run through one runner, dev/dev-test.nu. devenv shell -- dev-test … runs it from any shell, including an interactive one. devenv shell -- dev-test --help lists every subcommand and the cargo or nix command behind it.

devenv shell -- dev-test unit                               # cargo test --lib --bins
devenv shell -- dev-test integration [filter] [--nocapture] # cargo test --test '*' (integration targets only)
devenv shell -- dev-test lint                               # cargo fmt --check, clippy -D warnings, no openssl
devenv shell -- dev-test vm [vm|peers]                      # NixOS VM test (slow, needs /dev/kvm)
devenv shell -- dev-test all                                # lint, unit, integration; stops at the first failure
devenv shell -- dev-test ci                                 # all plus the vm test
devenv shell -- smoke                                       # same as: cargo test --test binary_smoke -- --nocapture

The smoke test runs the real built slinuxm binary against a throwaway SQLite database in a temp directory: it creates a token with the CLI, starts serve on a free loopback port, exercises /healthz, REST and MCP with and without the token, revokes the token, and checks a clean SIGTERM exit and a 0600 database. It needs no root and no network. Checks that depend on host data (systemd over D-Bus, the journal) are skipped with a printed reason when the source is unavailable; Docker is never required.

Development

devenv shell -- dev-test all                                # fmt, clippy, unit and integration tests

devenv shell -- dev-test --help shows the underlying commands (cargo fmt --check, cargo clippy --all-targets -- -D warnings, cargo test --lib --bins, cargo test --test '*'). Nushell: use dev/dev-test.nu * gives dev-test unit, dev-test vm peers, ... with tab completion. zsh users add source <(nu dev/dev-test.nu completions zsh) to .zshrc after compinit. An interactive devenv shell installs bash completion. The integration tests in tests/ start the real router on an ephemeral port with fake sources. Design notes live in ARCHITECTURE.md; work tracking in TODO.md; extended docs in docs/.

Contributors: enter devenv shell to install the git hooks; details in docs/hooks.md.

S
Description
No description provided
Readme
1.7 MiB
0 Stars 1 Watchers 0 Forks
Languages
Rust 86.7%
Nix 6.8%
Nushell 4.5%
Shell 1.4%
JavaScript 0.3%
Other 0.3%