Files
skyg-linux-tracker/plan/phase-5-consolidator/ws-b-registry.md
T

9.1 KiB

WS-B: Client, health, registry

Part of Phase 5: Consolidator. Contracts: contracts.md sections 1 to 4 and 8.

Field Value
Status Complete
Owner goose
Depends on A0 (stubs and seams) and A1 to A3 for real config/secrets; can start on B1 as soon as A0 lands
Blocks integration of C, D, E (they are developed against the fake registry and finish for real once B lands); F
Can start when A0 is merged (design and client code can begin earlier in a branch against contracts.md)
Owned files src/peers/client.rs, src/peers/health.rs, src/peers/registry.rs (bodies; signatures per contracts), src/api/ping.rs (new authenticated GET /api/v1/ping, see D4), Cargo.toml/Cargo.lock, src/main.rs (serve() only), src/tls.rs (SIGHUP fan-out line only, if the handler stays there), tests/peers_registry.rs, tests/common/mod.rs (append new helpers only)
Do not touch src/peers/types.rs, transport.rs, error.rs, testing.rs (change via contracts.md only); src/peers/config.rs, secrets.rs, cli.rs, src/config.rs (WS-A); src/peers/proxy.rs, src/api/peers.rs, src/api/mod.rs, src/error.rs (WS-C / A0); src/mcp/** (D); src/telemetry/** (E); nix/**, docs/**, README.md (F, T)

Shared-file seams:

  • src/state.rs: do not edit; A0 added AppState.peers. B assigns it in serve() in src/main.rs (state is built there by AppState::live, then state.peers = peers::start(&state) before server::build_router). One edit region, serve(); WS-A edits the Command enum elsewhere in the file.
  • SIGHUP: serve() installs the handler and tls::spawn_sighup_reload consumes it. B adds the fan-out so the same signal also calls PeerRegistry::reload (and the plain-TLS-off branch no longer just logs when peers are enabled). Keep the TLS reload behaviour unchanged.
  • src/api/mod.rs: untouched; ping.rs is already merged by A0. B only fills ping::routes().
  • Cargo.toml/Cargo.lock: B owns dependency edits (reqwest direct dep per D15, features, any CIDR crate not already added by A). Regenerate Cargo.lock, never hand-merge.
  • tests/common/mod.rs: append-only helpers; WS-F appends too.

Goal

A PeerRegistry built from config that keeps live status; a bounded HTTP client.

Tasks

  • B1 client: connect/request timeouts, redirects disabled, GET only, response byte cap enforced while streaming (not just Content-Length), per-peer Semaphore plus a global one, custom CA per peer, rustls with the ring provider passed explicitly; cargo tree -i openssl stays empty. Implements PeerTransport for real (ReqwestTransport) and records the D15 outcome in the Log.
  • B2 health loops: one task per peer with its own child CancellationToken and start jitter; TCP check = TcpStream::connect(host:port) under timeout; API check = authenticated GET /api/v1/ping; classification ok | unauthorized | http_error | timeout | unreachable | tls_error. Provides health::check_once (shared with slinuxm peer check, A4) and the /api/v1/ping handler in ping.rs returning {status, version}; fallback for peers without /ping is /healthz plus auth = unknown (D4).
  • B3 status model (in memory only): {server: {up, since, last_checked, last_error}, api: {up, since, last_checked, last_error, latency_ms, auth: ok|rejected|unknown, peer_version}}; flips need fail_threshold consecutive failures, recovery is immediate; since is the last change time and resets on restart.
  • B4 reload: on startup build the registry; on SIGHUP (shared handler with TLS) re-read only the [peers] block and token files, start loops for new peers, cancel removed ones, restart changed ones; a bad reload keeps the old registry (same rule as TLS T3). Token files are re-read on every API check, so rotating an agenix secret needs no restart. Shutdown awaits all loops (bounded).
  • B5 PeerTransport trait and fake; SSRF guard (D11) applied at config load and again at connect time on the resolved address. The trait and fake already exist from A0 (contracts section 2); B5 is now the SSRF guard on the real transport plus any fake extensions the loops need (changed through contracts.md).

Inputs

  • contracts.md sections 1 to 4 (types, transport, registry API, error mapping), 6 (config), 10 (the test's timings and rotation flow).
  • Fake: FakeTransport and registry_with for loop/classification unit tests with paused time; real in-process servers for the integration test.

Done when

Integration test starts a second in-process server (fake sources) as a peer and observes up, down (port closed), api-down-but-server-up (listener that accepts and hangs), unauthorized (wrong token) and recovery; one hung peer does not delay another peer's check beyond its own timeout; clippy, fmt, cargo tree -i openssl clean.

Handoff

  • Real PeerRegistry with the contract API (snapshot, status, get, reload, shutdown) assigned to AppState.peers when peers.enabled.
  • GET /api/v1/ping; ReqwestTransport; health::check_once for A4.
  • Post in C, D, E, F Logs when the registry is wired, so they swap their fakes for real end-to-end checks.

Log

  • 2026-10-19: Split from the single phase plan 2026-10-19. The trait and fake moved to A0 so C/D/E are not blocked on this stream; B keeps the implementations.

  • 2026-10-19 B1 (D15): chose reqwest (option a). Direct dependency reqwest 0.13.5 with default-features = false and rustls-no-provider only (no native-tls, no stream — that feature cannot resolve wasm-streams — and no rustls feature that selects aws-lc). Response bodies are capped by polling chunk(). Each client is a rustls ClientConfig built with builder_with_provider(ring) and webpki roots, plus an optional per-peer CA; install_default is not called. cargo tree -i openssl is empty. aws-lc-rs remains in the tree only through opentelemetry-otlp's own reqwest and is not used by this client. src/peers/mod.rs is not owned here, so client.rs is mounted as peers::registry::client (ReqwestTransport is re-exported from peers::registry).

  • 2026-10-19 B2: one task per peer (both probes in one tokio::select!) with a deterministic name-hash jitter capped at a quarter of the shorter interval. Probes are health::probe_server / health::probe_api; check_once joins them for A4. A 401/403 is api.up = true, auth = rejected; a 404 falls back to public /healthz with auth = unknown (older peers, D4). The loop reuses FakeTransport for the classification/threshold tests and a second in-process slinuxm (fake sources) for up/down/recovery, api-down-but-server-up (listener accepts, never answers) and rejected-token cases in tests/peers_registry.rs.

  • 2026-10-19 B3: ServerTracker/ApiTracker keep since and consecutive failures in loop-local state, so a surviving peer keeps its timestamps across a reload and a restart resets them. fail_threshold consecutive failures flip a check down; a success recovers in one check. last_error is always the fixed class string (never peer text or a URL). The first check records since whether it is up or down.

  • 2026-10-19 B4: PeerRegistry::start builds the real transport and loops; src/main.rs::serve() assigns state.peers. SIGHUP now reloads TLS (when enabled) and peers from one handler (tls::spawn_sighup_reload); a config error is logged and the old registry stays. reload keeps statuses of surviving peers, cancels removed/disabled/changed loops and starts new ones. Token files are read on every API check, so agenix rotation needs no restart. peers concurrency and size caps are fixed for the process lifetime; only the peer set, intervals, timeouts and threshold follow a reload. PeerRegistry::shutdown cancels and joins loops with a 5 s bound. A reload with [peers] enabled = false empties the snapshot (routes stay mounted until restart). main() passes the config path and CLI overrides into serve() because a SIGHUP reload must re-read the same file; that is the only edit outside serve().

  • 2026-10-19 B5: connect-time SSRF guard resolves the peer host, rejects any address in deny_cidrs (parsed with ipnet at ReqwestTransport::new), and pins the allowed address into the HTTP client; plain HTTP to a non-loopback host requires allow_plain_http. tcp_check uses the same resolution and guard. DNS failures map to dns, all-denied to unreachable.

Handoff (2026-10-19)

Complete; B1 to B5 and the authenticated GET /api/v1/ping route are in. D15 outcome is recorded above (reqwest, rustls-no-provider with ring passed explicitly, body cap enforced while streaming; cargo tree -i openssl is empty). The registry API matches contracts section 3 and is already used by the A0 seams (snapshot, status, get, reload, shutdown); C/D/E can now switch their fake registries to PeerRegistry::start.

Verification run: cargo fmt --check, cargo clippy --all-targets -- -D warnings, cargo test (unit plus the in-process two-server cases in tests/peers_registry.rs), cargo tree -i openssl (empty). No Nix files were touched, so no VM test is involved; when the Nix gate is free, WS-T covers the module/VM path. Nothing in this workstream mints tokens or writes outside the state dir.