diff --git a/README.md b/README.md index aaba528..9aad6db 100644 --- a/README.md +++ b/README.md @@ -131,7 +131,7 @@ token_file = "/etc/skyg/peer-nas.token" # 0600, owned by the service user allow_plain_http = true ``` -`token_file` holds a path, never the token. A consolidator has to replay the peer's plaintext bearer token, so the token lives in a file managed outside `slinuxm` (agenix on NixOS), is read into memory on every check and request, and never appears in the config, `--print-config`, the database or the logs. Mint **one token per consolidator on every peer** and use it only there (`sudo -u slinuxm token create consolidator-`), so a single `token revoke` on a peer cuts that consolidator off and nothing else. Replacing the file is enough to rotate: no restart. +`token_file` holds a path, never the token. A consolidator has to replay the peer's plaintext bearer token, so the token lives in a file managed outside `slinuxm` (agenix on NixOS), is read into memory on every check and request, and never appears in the config, `--print-config`, the database or the logs. Mint **one token per consolidator on every peer** and use it only there (`sudo -u slinuxm token create consolidator-`), so a single `token revoke` on a peer cuts that consolidator off and nothing else. Replacing the file is enough to rotate: the next check reads the new value, and no restart is needed. On NixOS also run `systemctl reload skyg-linux-tracker` so systemd re-copies the credential the unit holds (`RefreshOnReload=credentials`). Two checks per peer, kept in memory and reset on restart: diff --git a/docs/consolidator.md b/docs/consolidator.md index c099946..3fb72f5 100644 --- a/docs/consolidator.md +++ b/docs/consolidator.md @@ -109,7 +109,7 @@ The module renders the peers and the credentials; you only declare them: ```nix services.skyg-linux-tracker = { enable = true; - bind = "0.0.0.0"; + bind = "0.0.0.0"; # inbound, only if callers reach this host from other machines openFirewall = true; # or firewallInterfaces = [ "enp3s0" ]; peers = { enable = true; @@ -168,7 +168,7 @@ agenix -e secrets/peer-nas.age # paste the token printed on nas services.skyg-linux-tracker = { enable = true; - bind = "0.0.0.0"; + bind = "0.0.0.0"; # inbound for remote callers; peers need no inbound port openFirewall = true; peers = { enable = true; @@ -202,6 +202,10 @@ The module turns that into one `[[peers.peer]]` entry per enabled peer, with `to Outside NixOS the same split applies by hand: SIGHUP re-reads the TLS pair and `[peers]` (peer entries, intervals, timeouts, `fail_threshold`); everything else needs a restart. +### Shutdown + +SIGTERM stops the listener, cancels the shared token and then drains. The health loops end on that token, and a proxied follow stream is closed rather than held until the peer decides to stop sending: shutdown ends the SSE stream to your caller even when the peer keeps its socket open (`tests/peers.rs` asserts exactly that against a listener that never closes). A buffered proxied request is bounded by `request_timeout_secs` and a follow stream by the peer's own cap, so no peer connection outlives the process by more than one timeout. + ## Firewall and networking Every **peer** must accept the consolidator's address on its port; the consolidator only needs outbound access. On a peer's NixOS host that is `openFirewall = true` or `firewallInterfaces = [ "enp3s0" ]`; a peer bound to `127.0.0.1` is not reachable from another machine, so a consolidator needs the peer to bind a LAN address (with auth on, which the module enforces) and either TLS or a trusted network. @@ -223,13 +227,17 @@ DNS rebinding is still not a risk: the set is resolved once per request and pinn ## Testing -The peers acceptance test is a NixOS VM test. Three machines run the real `slinuxm`: `alpha` and `beta` consolidate each other over plain HTTP, and `alpha` also declares the TLS-only `gamma` with a private CA. It needs KVM and takes minutes. +Two layers, neither of which needs a VM for the first: + +- `tests/peers.rs` drives the whole path in-process: a consolidator served by the real router and registry with the real outbound client, against peers that are either another in-process `slinuxm` (fake sources, real auth with a minted token) or a hand-written listener for a peer that misbehaves. It covers a plain peer and a TLS peer with its own CA, a peer signed by the wrong CA, a redirect, an oversize (declared and chunked) body, a silent peer, a slow-loris peer, the 64-peer cap, shutdown ending a proxied follow, and a peer declared by name that listens on one address family only. +- The NixOS VM test boots three machines running the real `slinuxm`: `alpha` and `beta` consolidate each other over plain HTTP, and `alpha` also declares the TLS-only `gamma` with a private CA (bound IPv4 only, so the address-family fallback stays guarded). It asserts status, fan-in, proxied data against a direct call, MCP parity, token install/rotation/revocation, peer removal on SIGHUP and an outage, all with `MainPID` unchanged where a reload is promised. `nix flake check` also runs `checks.module-peers`, an evaluation-only case list for the module: every rendering and assertion fails the check without booting anything. ```sh -devenv shell -- dev-test vm peers # nix build .#tests.x86_64-linux.peers -L +devenv shell -- dev-test integration peers # cargo test --test peers +devenv shell -- dev-test vm peers # nix build .#checks.x86_64-linux.vm-peers -L ``` -Against a tree without the consolidator the test stops at the first peers assertion; that is expected, the test is written ahead of the feature. `dev/dev-test.nu` is the repository's test runner. `devenv shell -- dev-test --help` lists its subcommands and the command behind each. In an interactive `devenv shell`, `dev-test …` is that same script. +`dev/dev-test.nu` is the repository's test runner. `devenv shell -- dev-test --help` lists its subcommands and the command behind each. In an interactive `devenv shell`, `dev-test …` is that same script. In Nushell the runner is a module. Load it once, for example in `config.nu`, and `dev-test unit`, `dev-test vm peers`, ... complete natively. Integration-test name completion is cached in `target/skyg-test-filters.cache` at the repo root (not relative to the current directory, so completion works from any subdirectory), keyed on the newest mtime of the Rust sources and the Cargo manifests. A failing check raises a catchable error instead of exiting, so a red `dev-test unit` does not close the Nushell session that loaded the module; script mode still exits non-zero. diff --git a/nix/tests/peers.nix b/nix/tests/peers.nix index 15e221c..b124219 100644 --- a/nix/tests/peers.nix +++ b/nix/tests/peers.nix @@ -338,7 +338,7 @@ pkgs.testers.runNixOSTest { assert http(alpha, base, A, "-X POST")[0] == 405 assert http(alpha, base, A, "-X DELETE")[0] == 405 status, _ = http(alpha, base, A, "-H 'x-slinuxm-hop: 1'") - assert 400 <= status < 500, status + assert status == 400, status assert http(alpha, f"http://127.0.0.1:{PORT}/api/v1/peers/nope", A)[0] == 404 with subtest("token rejected: revoke on beta, then restore with a new token file"):