8.8 KiB
8.8 KiB
Phase 5 decisions
Part of Phase 5: Consolidator. Record decisions here and nowhere else; workstream files link to row IDs. Interface details that follow from a decision live in contracts.md.
DEC-13 (accepted 2026-10-19): outbound peer credentials are external files
- DEC-11 today (inbound): a token that clients present to this service is generated by
slinuxm token create, shown once, and only its sha256 hash is stored. A stolen database yields nothing usable, because the service never needs the plaintext again. - Why that cannot apply to peers (outbound): to call a peer, the consolidator must send the peer's plaintext bearer token. A hash is useless for that, so the plaintext has to exist on the consolidator somewhere. DEC-11's "never stored" cannot hold literally.
- DEC-13 rule:
slinuxmitself never persists a peer token. It owns no copy: not in the DB, not inconfig.toml, not in--print-configor logs, not in the Nix store. The config holds only atoken_filepath. The secret lives in a file managed by something else (agenix on NixOS, decrypted at activation to/run/agenix/...with root-only permissions and delivered to the service throughLoadCredential).slinuxmreads it into memory (redactedDebug) and re-reads it on each check, so rotation needs no restart. - What this does and does not protect: an offline copy of the DB, a backup, the Nix store or a support bundle contains no peer secret, and the encrypted secret in git is decryptable only by the host key. It does not protect against root on the consolidator host, who can read the decrypted file; that is inherent to any scheme that must replay a bearer token. Mitigation: each consolidator uses its own read-only token on every peer (
consolidator-<host>), so one peer-sidetoken revokecuts that consolidator off alone. - Scope: DEC-11 is amended to "inbound tokens are stored only as hashes"; DEC-13 covers outbound. Both go into ARCHITECTURE.md Decisions after acceptance.
Open decisions
| ID | Question | Options | Recommendation | Decision |
|---|---|---|---|---|
| D1 | How are peers defined? | (a) CLI plus peers table, server polls the table; (b) config file plus SIGHUP reload; (c) both |
Decided by the user (2026-10-19): declarative. Peers are defined in config (on NixOS, in the module); no DB table, no peer add. Reload via SIGHUP, shared with TLS. |
Declarative |
| D2 | Where do outbound peer tokens live (DEC-11 forbids plaintext tokens)? | (a) plaintext column in the DB; (b) AES-GCM in the DB; (c) per-peer files written by a CLI; (d) config references a token_file, contents come from an external secret manager |
Decided by the user (2026-10-19): agenix. (d): config holds only a path; on NixOS config.age.secrets.<x>.path passed through LoadCredential. The secret is never in the Nix store, the config, the DB or --print-config. New DEC-13: DEC-11 covers inbound tokens only; outbound tokens are externally managed files. Blast radius: use a read-only token per consolidator (consolidator-<host>) so a peer can revoke it alone. |
agenix token_file |
| D3 | Must the DB exist when auth.enabled = false? |
n/a | Dropped: peers no longer use the DB. | Dropped |
| D4 | What is "API running"? | (a) public /healthz 200; (b) new authenticated GET /api/v1/ping; (c) existing cheap endpoint such as systemd/timers |
(b) returning {status, version}; /healthz stays public. A 401 is reported as api.up = true, auth = rejected (the API is running, our credential is not), so the operator sees the real fault. Fallback for older peers without /ping: /healthz plus auth = unknown. Adds one route to every instance (small, authed, harmless). |
Accepted 2026-10-19 (recommendation) |
| D5 | Intervals, timeouts, state? | Proposed: TCP 10 s / 3 s, API 30 s / 5 s, 2 consecutive failures to flip down, immediate recovery; state in memory only | Accept the proposal; last_change resets on restart (documented). Persisting history conflicts with "live reads only" (DEC-5 reasoning); history belongs to the OTel backend. |
Accepted 2026-10-19 (recommendation) |
| D6 | REST proxy shape | (a) /api/v1/peers/{peer}/... pass-through of an allowlist; (b) typed endpoints per peer resource; (c) status only |
(a). Zero duplicated DTOs, peers evolve independently, allowlist keeps it from becoming an open proxy. | Accepted 2026-10-19 (recommendation) |
| D7 | MCP shape | (a) optional peer arg on existing 8 tools plus list_peers/get_peer (10 tools); (b) separate peer_* tool copies (16+); (c) status tools only |
(a). Tool count becomes 10 only when peers.enabled; snapshot test covers both. Keeps context small for agents. |
Accepted 2026-10-19 (recommendation) |
| D8 | Aggregation (fan-out, e.g. "failed units on all peers") | (a) none in this phase; (b) peer = "*" on the list tools plus GET /api/v1/fleet/failed; (c) both with per-peer partial-failure reporting |
(a) for this phase, (b) as the first follow-up: partial failures need a result shape decision ({peer, ok, data|error}) that is easier to get right once pass-through is proven. |
Accepted 2026-10-19 (recommendation) |
| D9 | Peer error mapping | Peer 404 -> NotFound; 400/422 -> BadRequest; peer 401/403 (our token rejected) -> Unavailable "peer rejected credentials" (never Unauthorized, which would tell the caller to re-auth with us); timeout, refused, DNS, TLS, 5xx, oversize -> Unavailable with peer name and class; unknown {peer} -> NotFound; malformed peer JSON -> Internal. Optionally a new BadGateway variant (502/504). |
Reuse Unavailable (503) and put the class in the message and an extra peer field; no new variant, so the error contract and MCP mapping are unchanged. Revisit if clients need to distinguish "consolidator down" from "peer down". |
Accepted 2026-10-19 (recommendation) |
| D10 | OTel metrics | peer.up (0/1; attrs peer.name, peer.check = server|api); peer.api.auth_ok (0/1; peer.name; confirmed by the user as a first-class metric: server up + api up + auth_ok = 0 is the "token rejected" state, kept separate from "API down" so alerts can distinguish a revoked or rotated credential from an outage); peer.check.duration_seconds (s; peer.name, peer.check); peer.last_change_timestamp_seconds (s; peer.name, peer.check). All observable f64 gauges from the registry snapshot; no URL or version in attributes (cardinality, and URLs may embed hostnames the operator treats as sensitive). |
Accept. | Accepted 2026-10-19 (recommendation) |
| D11 | SSRF and TLS to peers | Peers come only from configuration (root- or Nix-managed); no request parameter ever supplies a URL. http/https only, no userinfo, query or fragment; redirects never followed; deny_cidrs checked against the resolved IP at connect time (LAN/private ranges must stay allowed, so default denies only link-local/metadata and unspecified). HTTPS verified against the webpki roots plus an optional per-peer ca_file; no skip-verify flag. Plain http to a non-loopback host requires allow_plain_http = true on that peer and logs a warning (the token travels in clear, same stance as DEC-12). |
Accept. Alternative to weigh: system roots (needs rustls-native-certs, one more dep) instead of webpki roots; recommend webpki plus custom CA since LAN peers will use a private CA anyway. |
Accepted 2026-10-19 (recommendation) |
| D12 | May a peer be a consolidator (chaining)? | (a) no, hop header blocks it; (b) allow with depth limit | (a). Prevents loops and amplification; revisit with D8. | Accepted 2026-10-19 (recommendation) |
| D13 | Declarative peers (NixOS list, agenix token files)? | (a) not in v1; (b) module renders config entries | Decided by the user: this is the primary and only mechanism (see D1, D2). | Declarative |
| D14 | Who may read peer status or proxy? | (a) any valid inbound token; (b) new scoped tokens | (a). Tokens have no scopes today; scoping is a separate phase. Note in docs that a consolidator token reaches every peer's data. | Accepted 2026-10-19 (recommendation) |
| D15 | HTTP client crate | (a) reqwest direct dep (already in the tree via OTel), rustls with ring passed explicitly; (b) hyper-util client plus tokio-rustls |
(a) if cargo tree -i openssl stays empty and no second crypto provider is pulled in; otherwise (b). Decide in B1, record the outcome. |
Accepted 2026-10-19 (recommendation) |
Where each decision lands
| Decision | Implemented in |
|---|---|
D1, D2, D13 (declarative, agenix token_file) |
ws-a-config.md (A1 to A3), ws-f-nixos-docs.md (F1) |
D4 (/api/v1/ping), D5 (intervals), D11, D15 |
ws-b-registry.md |
| D6, D9, D12, D14 | ws-c-rest.md, error table in contracts.md section 4 |
| D7, D8 | ws-d-mcp.md |
| D10 | ws-e-telemetry.md, contracts.md section 8 |