6.8 KiB
Operations
Day-to-day tasks for a running install. The service user below is skyg-linux-tracker (NixOS) or skyg (the manual example unit); substitute yours.
Tokens
sudo -u skyg-linux-tracker slinuxm token create my-laptop # prints the token once
sudo -u skyg-linux-tracker slinuxm token list # name, created, revoked, last used
sudo -u skyg-linux-tracker slinuxm token rotate my-laptop # new secret, printed once; the old one stops working
sudo -u skyg-linux-tracker slinuxm token revoke my-laptop
- Tokens look like
slinuxm_.... Only a sha256 hash is stored, so a lost token cannot be recovered: rotate it. - Every change takes effect on the next request. No restart, no reload.
rotatealso re-enables a revoked token under the same name.- Use one token per client (laptop, agent, consolidator), so you can revoke one without touching the others.
- Run the commands as the service user. Run as anyone else, root included, they refuse and print the right
sudo -ucommand. The firsttoken createneeds the state directory, which the service creates on its first start. - If the config is not at
/etc/skyg-linux-tracker.toml, add--config <file>.
Use the API
TOKEN=slinuxm_...
BASE=http://127.0.0.1:1984
curl -s $BASE/healthz # public
curl -s -H "Authorization: Bearer $TOKEN" "$BASE/api/v1/systemd/units?state=failed"
curl -s -H "Authorization: Bearer $TOKEN" "$BASE/api/v1/systemd/timers"
curl -s -H "Authorization: Bearer $TOKEN" "$BASE/api/v1/journal?unit=nginx.service&priority=err&lines=50"
curl -N -H "Authorization: Bearer $TOKEN" "$BASE/api/v1/journal/follow?priority=warning" # live stream (SSE)
curl -s -H "Authorization: Bearer $TOKEN" "$BASE/api/v1/docker/ps"
Every route, parameter and error: api.md. The same reference is browsable at $BASE/docs (paste a token when asked), and slinuxm openapi prints it as OpenAPI 3.1.
Connect an MCP client
Endpoint http://127.0.0.1:1984/mcp, Streamable HTTP, with the bearer header.
Claude Code:
claude mcp add --transport http linux-tracker http://127.0.0.1:1984/mcp \
--header "Authorization: Bearer $TOKEN"
Any client that takes a JSON config with headers:
{
"mcpServers": {
"linux-tracker": {
"url": "http://127.0.0.1:1984/mcp",
"headers": { "Authorization": "Bearer slinuxm_..." }
}
}
}
The server offers eight tools (list_systemd_units, get_systemd_unit, list_systemd_timers, get_journal_logs, tail_journal, list_containers, get_container, get_container_logs), two prompts, two resources and two resource templates. Details and a raw curl handshake: api.md.
Expose it beyond localhost
The default is loopback only. The safest way to reach it from elsewhere is to leave it there and use an SSH tunnel, Tailscale or a TLS reverse proxy. To listen on the network directly:
- Set
server.bind("0.0.0.0"or"::"). Auth must stay on. - Turn on TLS (tls.md), unless the network is trusted: bearer tokens are readable in plain HTTP.
- Add the names clients use to
server.allowed_hosts(e.g."monitor.lan:1984"), or/mcprejects them. - Open the port in the firewall, ideally on the LAN interface only.
On NixOS that is bind, tls.*, allowedHosts and firewallInterfaces; see install.md.
Docker access
Reading the Docker socket needs either:
-
A read-only socket proxy (recommended). For example tecnativa/docker-socket-proxy with
CONTAINERS=1 EVENTS=1 POST=0, exposed as a unix socket. Then:[sources] docker_socket = "/run/docker-proxy/docker.sock" -
The
dockergroup. Simple, but membership is root-equivalent: anyone who controls the process could start a privileged container. Manual unit: uncommentSupplementaryGroups=docker. NixOS:docker.enable = true.
Out of the box neither is set up, so the Docker routes and tools return 503 while the rest works.
OpenTelemetry
[otel]
enabled = true
endpoint = "http://collector:4317" # or :4318 with protocol = "http/protobuf"
export_logs = true # also forward logs, not only metrics
Metrics (every metrics_interval_secs, gauges):
| Metric | Meaning |
|---|---|
systemd.unit.active, systemd.unit.failed |
0/1 per unit |
systemd.unit.restarts |
restart count (services) |
systemd.timer.next_elapse_timestamp_seconds |
next run of a timer, unix time |
docker.container.running, docker.container.healthy |
0/1 per container |
docker.container.uptime_seconds, docker.container.restarts |
per container |
peer.* |
peer health, with peers enabled (consolidator.md) |
By default only *.service and *.timer units are exported; set [otel.metrics] units to a glob list to change that.
Logs (export_logs = true): journal entries and container stdout/stderr, with severity mapped from the journal priority (stderr is WARN, stdout INFO). Narrow them with [otel.logs].
Traces: one span per HTTP and MCP request.
For a local test collector that prints everything it receives: devenv up in a checkout.
Reload, restart, stop
systemctl reload skyg-linux-tracker # SIGHUP: re-read the TLS pair and [peers]; connections stay up
systemctl restart skyg-linux-tracker # anything else in the config
systemctl stop skyg-linux-tracker # graceful: open streams end, requests drain, OTel flushes
Troubleshooting
The server logs to stderr; under systemd: journalctl -u skyg-linux-tracker.
| Symptom | Likely cause |
|---|---|
401 on every request |
Missing Authorization: Bearer header, or the token was revoked or rotated. /healthz works without a token. |
503 on /api/v1/docker/* only |
No access to the Docker socket, or Docker is not running. See Docker access. |
403 on REST with auth off |
With auth.enabled = false, REST accepts only loopback Host names plus allowed_hosts. |
| MCP client fails to connect from another host | Its Host (or browser Origin) is not in server.allowed_hosts / allowed_origins. |
token create refuses with a sudo -u hint |
You ran it as the wrong user, root included. Use the command it prints. |
token create cannot find the database |
The service has never started (no state directory yet), or --config points at a different file. |
| Journal results are short or empty | The service user is not in systemd-journal, or sources.journal_namespace names the wrong namespace. |
| Server refuses to start | Read the first error line: it names the config key, the TLS file or the peer token file. slinuxm --print-config shows the effective config. |
| Port changed in config, nothing happened | Port, bind and auth need a restart, not a reload. |