Files
skyg-linux-tracker/plan/phase-5-consolidator/ws-b-registry.md
T

65 lines
8.9 KiB
Markdown

# WS-B: Client, health, registry
Part of [Phase 5: Consolidator](README.md). Contracts: [contracts.md](contracts.md) sections 1 to 4 and 8.
| Field | Value |
|---|---|
| Status | Complete |
| Owner | goose |
| Depends on | A0 (stubs and seams) and A1 to A3 for real config/secrets; can start on B1 as soon as A0 lands |
| Blocks | integration of C, D, E (they are developed against the fake registry and finish for real once B lands); F |
| Can start when | A0 is merged (design and client code can begin earlier in a branch against contracts.md) |
| Owned files | `src/peers/client.rs`, `src/peers/health.rs`, `src/peers/registry.rs` (bodies; signatures per contracts), `src/api/ping.rs` (new authenticated `GET /api/v1/ping`, see D4), `Cargo.toml`/`Cargo.lock`, `src/main.rs` (`serve()` only), `src/tls.rs` (SIGHUP fan-out line only, if the handler stays there), `tests/peers_registry.rs`, `tests/common/mod.rs` (append new helpers only) |
| Do not touch | `src/peers/types.rs`, `transport.rs`, `error.rs`, `testing.rs` (change via contracts.md only); `src/peers/config.rs`, `secrets.rs`, `cli.rs`, `src/config.rs` (WS-A); `src/peers/proxy.rs`, `src/api/peers.rs`, `src/api/mod.rs`, `src/error.rs` (WS-C / A0); `src/mcp/**` (D); `src/telemetry/**` (E); `nix/**`, `docs/**`, `README.md` (F, T) |
Shared-file seams:
- `src/state.rs`: do not edit; A0 added `AppState.peers`. B assigns it in `serve()` in `src/main.rs` (state is built there by `AppState::live`, then `state.peers = peers::start(&state)` before `server::build_router`). One edit region, `serve()`; WS-A edits the `Command` enum elsewhere in the file.
- SIGHUP: `serve()` installs the handler and `tls::spawn_sighup_reload` consumes it. B adds the fan-out so the same signal also calls `PeerRegistry::reload` (and the plain-TLS-off branch no longer just logs when peers are enabled). Keep the TLS reload behaviour unchanged.
- `src/api/mod.rs`: untouched; `ping.rs` is already merged by A0. B only fills `ping::routes()`.
- `Cargo.toml`/`Cargo.lock`: B owns dependency edits (reqwest direct dep per D15, features, any CIDR crate not already added by A). Regenerate `Cargo.lock`, never hand-merge.
- `tests/common/mod.rs`: append-only helpers; WS-F appends too.
## Goal
A `PeerRegistry` built from config that keeps live status; a bounded HTTP client.
## Tasks
- **B1** client: connect/request timeouts, redirects disabled, GET only, response byte cap enforced while streaming (not just `Content-Length`), per-peer `Semaphore` plus a global one, custom CA per peer, rustls with the ring provider passed explicitly; `cargo tree -i openssl` stays empty. Implements `PeerTransport` for real (`ReqwestTransport`) and records the D15 outcome in the Log.
- **B2** health loops: one task per peer with its own child `CancellationToken` and start jitter; TCP check = `TcpStream::connect(host:port)` under timeout; API check = authenticated `GET /api/v1/ping`; classification `ok | unauthorized | http_error | timeout | unreachable | tls_error`. Provides `health::check_once` (shared with `slinuxm peer check`, A4) and the `/api/v1/ping` handler in `ping.rs` returning `{status, version}`; fallback for peers without `/ping` is `/healthz` plus `auth = unknown` (D4).
- **B3** status model (in memory only): `{server: {up, since, last_checked, last_error}, api: {up, since, last_checked, last_error, latency_ms, auth: ok|rejected|unknown, peer_version}}`; flips need `fail_threshold` consecutive failures, recovery is immediate; `since` is the last change time and resets on restart.
- **B4** reload: on startup build the registry; on SIGHUP (shared handler with TLS) re-read only the `[peers]` block and token files, start loops for new peers, cancel removed ones, restart changed ones; a bad reload keeps the old registry (same rule as TLS T3). Token files are re-read on every API check, so rotating an agenix secret needs no restart. Shutdown awaits all loops (bounded).
- **B5** `PeerTransport` trait and fake; SSRF guard (D11) applied at config load and again at connect time on the resolved address. The trait and fake already exist from A0 (contracts section 2); B5 is now the SSRF guard on the real transport plus any fake extensions the loops need (changed through contracts.md).
## Inputs
- [contracts.md](contracts.md) sections 1 to 4 (types, transport, registry API, error mapping), 6 (config), 10 (the test's timings and rotation flow).
- Fake: `FakeTransport` and `registry_with` for loop/classification unit tests with paused time; real in-process servers for the integration test.
## Done when
Integration test starts a second in-process server (fake sources) as a peer and observes up, down (port closed), api-down-but-server-up (listener that accepts and hangs), unauthorized (wrong token) and recovery; one hung peer does not delay another peer's check beyond its own timeout; clippy, fmt, `cargo tree -i openssl` clean.
## Handoff
- Real `PeerRegistry` with the contract API (`snapshot`, `status`, `get`, `reload`, `shutdown`) assigned to `AppState.peers` when `peers.enabled`.
- `GET /api/v1/ping`; `ReqwestTransport`; `health::check_once` for A4.
- Post in C, D, E, F Logs when the registry is wired, so they swap their fakes for real end-to-end checks.
## Log
- 2026-10-19: Split from the single phase plan 2026-10-19. The trait and fake moved to A0 so C/D/E are not blocked on this stream; B keeps the implementations.
- 2026-10-19 B1 (D15): chose reqwest (option a). Direct dependency `reqwest` 0.13.5 with `default-features = false` and `rustls-no-provider` only (no `native-tls`, no `stream` — that feature cannot resolve `wasm-streams` — and no `rustls` feature that selects aws-lc). Response bodies are capped by polling `chunk()`. Each client is a rustls `ClientConfig` built with `builder_with_provider(ring)` and webpki roots, plus an optional per-peer CA; `install_default` is not called. `cargo tree -i openssl` is empty. `aws-lc-rs` remains in the tree only through `opentelemetry-otlp`'s own reqwest and is not used by this client. `src/peers/mod.rs` is not owned here, so `client.rs` is mounted as `peers::registry::client` (`ReqwestTransport` is re-exported from `peers::registry`).
- 2026-10-19 B2: one task per peer (both probes in one `tokio::select!`) with a deterministic name-hash jitter capped at a quarter of the shorter interval. Probes are `health::probe_server` / `health::probe_api`; `check_once` joins them for A4. A 401/403 is `api.up = true, auth = rejected`; a 404 falls back to public `/healthz` with `auth = unknown` (older peers, D4). The loop reuses `FakeTransport` for the classification/threshold tests and a second in-process `slinuxm` (fake sources) for up/down/recovery, api-down-but-server-up (listener accepts, never answers) and rejected-token cases in `tests/peers_registry.rs`.
- 2026-10-19 B3: `ServerTracker`/`ApiTracker` keep `since` and consecutive failures in loop-local state, so a surviving peer keeps its timestamps across a reload and a restart resets them. `fail_threshold` consecutive failures flip a check down; a success recovers in one check. `last_error` is always the fixed class string (never peer text or a URL). The first check records `since` whether it is up or down.
- 2026-10-19 B4: `PeerRegistry::start` builds the real transport and loops; `src/main.rs::serve()` assigns `state.peers`. SIGHUP now reloads TLS (when enabled) and peers from one handler (`tls::spawn_sighup_reload`); a config error is logged and the old registry stays. `reload` keeps statuses of surviving peers, cancels removed/disabled/changed loops and starts new ones. Token files are read on every API check, so agenix rotation needs no restart. `peers` concurrency and size caps are fixed for the process lifetime; only the peer set, intervals, timeouts and threshold follow a reload. `PeerRegistry::shutdown` cancels and joins loops with a 5 s bound.
- 2026-10-19 B5: connect-time SSRF guard resolves the peer host, rejects any address in `deny_cidrs` (parsed with `ipnet` at `ReqwestTransport::new`), and pins the allowed address into the HTTP client; plain HTTP to a non-loopback host requires `allow_plain_http`. `tcp_check` uses the same resolution and guard. DNS failures map to `dns`, all-denied to `unreachable`.
## Handoff (2026-10-19)
Complete; B1 to B5 and the authenticated `GET /api/v1/ping` route are in. D15 outcome is recorded above (reqwest, rustls-no-provider with ring passed explicitly, body cap enforced while streaming; `cargo tree -i openssl` is empty). The registry API matches contracts section 3 and is already used by the A0 seams (`snapshot`, `status`, `get`, `reload`, `shutdown`); C/D/E can now switch their fake registries to `PeerRegistry::start`.
Verification run: `cargo fmt --check`, `cargo clippy --all-targets -- -D warnings`, `cargo test` (unit plus the in-process two-server cases in `tests/peers_registry.rs`), `cargo tree -i openssl` (empty). No Nix files were touched, so no VM test is involved; when the Nix gate is free, WS-T covers the module/VM path. Nothing in this workstream mints tokens or writes outside the state dir.