8.9 KiB
WS-B: Client, health, registry
Part of Phase 5: Consolidator. Contracts: contracts.md sections 1 to 4 and 8.
| Field | Value |
|---|---|
| Status | Complete |
| Owner | goose |
| Depends on | A0 (stubs and seams) and A1 to A3 for real config/secrets; can start on B1 as soon as A0 lands |
| Blocks | integration of C, D, E (they are developed against the fake registry and finish for real once B lands); F |
| Can start when | A0 is merged (design and client code can begin earlier in a branch against contracts.md) |
| Owned files | src/peers/client.rs, src/peers/health.rs, src/peers/registry.rs (bodies; signatures per contracts), src/api/ping.rs (new authenticated GET /api/v1/ping, see D4), Cargo.toml/Cargo.lock, src/main.rs (serve() only), src/tls.rs (SIGHUP fan-out line only, if the handler stays there), tests/peers_registry.rs, tests/common/mod.rs (append new helpers only) |
| Do not touch | src/peers/types.rs, transport.rs, error.rs, testing.rs (change via contracts.md only); src/peers/config.rs, secrets.rs, cli.rs, src/config.rs (WS-A); src/peers/proxy.rs, src/api/peers.rs, src/api/mod.rs, src/error.rs (WS-C / A0); src/mcp/** (D); src/telemetry/** (E); nix/**, docs/**, README.md (F, T) |
Shared-file seams:
src/state.rs: do not edit; A0 addedAppState.peers. B assigns it inserve()insrc/main.rs(state is built there byAppState::live, thenstate.peers = peers::start(&state)beforeserver::build_router). One edit region,serve(); WS-A edits theCommandenum elsewhere in the file.- SIGHUP:
serve()installs the handler andtls::spawn_sighup_reloadconsumes it. B adds the fan-out so the same signal also callsPeerRegistry::reload(and the plain-TLS-off branch no longer just logs when peers are enabled). Keep the TLS reload behaviour unchanged. src/api/mod.rs: untouched;ping.rsis already merged by A0. B only fillsping::routes().Cargo.toml/Cargo.lock: B owns dependency edits (reqwest direct dep per D15, features, any CIDR crate not already added by A). RegenerateCargo.lock, never hand-merge.tests/common/mod.rs: append-only helpers; WS-F appends too.
Goal
A PeerRegistry built from config that keeps live status; a bounded HTTP client.
Tasks
- B1 client: connect/request timeouts, redirects disabled, GET only, response byte cap enforced while streaming (not just
Content-Length), per-peerSemaphoreplus a global one, custom CA per peer, rustls with the ring provider passed explicitly;cargo tree -i opensslstays empty. ImplementsPeerTransportfor real (ReqwestTransport) and records the D15 outcome in the Log. - B2 health loops: one task per peer with its own child
CancellationTokenand start jitter; TCP check =TcpStream::connect(host:port)under timeout; API check = authenticatedGET /api/v1/ping; classificationok | unauthorized | http_error | timeout | unreachable | tls_error. Provideshealth::check_once(shared withslinuxm peer check, A4) and the/api/v1/pinghandler inping.rsreturning{status, version}; fallback for peers without/pingis/healthzplusauth = unknown(D4). - B3 status model (in memory only):
{server: {up, since, last_checked, last_error}, api: {up, since, last_checked, last_error, latency_ms, auth: ok|rejected|unknown, peer_version}}; flips needfail_thresholdconsecutive failures, recovery is immediate;sinceis the last change time and resets on restart. - B4 reload: on startup build the registry; on SIGHUP (shared handler with TLS) re-read only the
[peers]block and token files, start loops for new peers, cancel removed ones, restart changed ones; a bad reload keeps the old registry (same rule as TLS T3). Token files are re-read on every API check, so rotating an agenix secret needs no restart. Shutdown awaits all loops (bounded). - B5
PeerTransporttrait and fake; SSRF guard (D11) applied at config load and again at connect time on the resolved address. The trait and fake already exist from A0 (contracts section 2); B5 is now the SSRF guard on the real transport plus any fake extensions the loops need (changed through contracts.md).
Inputs
- contracts.md sections 1 to 4 (types, transport, registry API, error mapping), 6 (config), 10 (the test's timings and rotation flow).
- Fake:
FakeTransportandregistry_withfor loop/classification unit tests with paused time; real in-process servers for the integration test.
Done when
Integration test starts a second in-process server (fake sources) as a peer and observes up, down (port closed), api-down-but-server-up (listener that accepts and hangs), unauthorized (wrong token) and recovery; one hung peer does not delay another peer's check beyond its own timeout; clippy, fmt, cargo tree -i openssl clean.
Handoff
- Real
PeerRegistrywith the contract API (snapshot,status,get,reload,shutdown) assigned toAppState.peerswhenpeers.enabled. GET /api/v1/ping;ReqwestTransport;health::check_oncefor A4.- Post in C, D, E, F Logs when the registry is wired, so they swap their fakes for real end-to-end checks.
Log
-
2026-10-19: Split from the single phase plan 2026-10-19. The trait and fake moved to A0 so C/D/E are not blocked on this stream; B keeps the implementations.
-
2026-10-19 B1 (D15): chose reqwest (option a). Direct dependency
reqwest0.13.5 withdefault-features = falseandrustls-no-provideronly (nonative-tls, nostream— that feature cannot resolvewasm-streams— and norustlsfeature that selects aws-lc). Response bodies are capped by pollingchunk(). Each client is a rustlsClientConfigbuilt withbuilder_with_provider(ring)and webpki roots, plus an optional per-peer CA;install_defaultis not called.cargo tree -i opensslis empty.aws-lc-rsremains in the tree only throughopentelemetry-otlp's own reqwest and is not used by this client.src/peers/mod.rsis not owned here, soclient.rsis mounted aspeers::registry::client(ReqwestTransportis re-exported frompeers::registry). -
2026-10-19 B2: one task per peer (both probes in one
tokio::select!) with a deterministic name-hash jitter capped at a quarter of the shorter interval. Probes arehealth::probe_server/health::probe_api;check_oncejoins them for A4. A 401/403 isapi.up = true, auth = rejected; a 404 falls back to public/healthzwithauth = unknown(older peers, D4). The loop reusesFakeTransportfor the classification/threshold tests and a second in-processslinuxm(fake sources) for up/down/recovery, api-down-but-server-up (listener accepts, never answers) and rejected-token cases intests/peers_registry.rs. -
2026-10-19 B3:
ServerTracker/ApiTrackerkeepsinceand consecutive failures in loop-local state, so a surviving peer keeps its timestamps across a reload and a restart resets them.fail_thresholdconsecutive failures flip a check down; a success recovers in one check.last_erroris always the fixed class string (never peer text or a URL). The first check recordssincewhether it is up or down. -
2026-10-19 B4:
PeerRegistry::startbuilds the real transport and loops;src/main.rs::serve()assignsstate.peers. SIGHUP now reloads TLS (when enabled) and peers from one handler (tls::spawn_sighup_reload); a config error is logged and the old registry stays.reloadkeeps statuses of surviving peers, cancels removed/disabled/changed loops and starts new ones. Token files are read on every API check, so agenix rotation needs no restart.peersconcurrency and size caps are fixed for the process lifetime; only the peer set, intervals, timeouts and threshold follow a reload.PeerRegistry::shutdowncancels and joins loops with a 5 s bound. -
2026-10-19 B5: connect-time SSRF guard resolves the peer host, rejects any address in
deny_cidrs(parsed withipnetatReqwestTransport::new), and pins the allowed address into the HTTP client; plain HTTP to a non-loopback host requiresallow_plain_http.tcp_checkuses the same resolution and guard. DNS failures map todns, all-denied tounreachable.
Handoff (2026-10-19)
Complete; B1 to B5 and the authenticated GET /api/v1/ping route are in. D15 outcome is recorded above (reqwest, rustls-no-provider with ring passed explicitly, body cap enforced while streaming; cargo tree -i openssl is empty). The registry API matches contracts section 3 and is already used by the A0 seams (snapshot, status, get, reload, shutdown); C/D/E can now switch their fake registries to PeerRegistry::start.
Verification run: cargo fmt --check, cargo clippy --all-targets -- -D warnings, cargo test (unit plus the in-process two-server cases in tests/peers_registry.rs), cargo tree -i openssl (empty). No Nix files were touched, so no VM test is involved; when the Nix gate is free, WS-T covers the module/VM path. Nothing in this workstream mints tokens or writes outside the state dir.