F: correct the peers docs against the finished behaviour

docs/consolidator.md: the Testing section described the pre-feature test
(no tests/peers.rs, the old tests.<system>.peers attribute, a note about
assertions failing by design) and is rewritten for both layers, the
module's eval check included. A shutdown subsection backs the README's
promise, the agenix examples say why a consolidator would bind 0.0.0.0
(remote callers; peers need no inbound port), and the VM test asserts
the chained-request status is exactly 400 as documented. README: token
rotation on NixOS also needs a reload for the LoadCredential copy.
This commit is contained in:
ari committed 2026-10-20 15:04:09 -04:00
1 parent 0a83386de5
commit 04992e202d
3 files changed
+15 -7

No files matched your search

+1 -1
View File
@@ -131,7 +131,7 @@ token_file = "/etc/skyg/peer-nas.token" # 0600, owned by the service user
allow_plain_http = true
```
`token_file` holds a path, never the token. A consolidator has to replay the peer's plaintext bearer token, so the token lives in a file managed outside `slinuxm` (agenix on NixOS), is read into memory on every check and request, and never appears in the config, `--print-config`, the database or the logs. Mint **one token per consolidator on every peer** and use it only there (`sudo -u <service user> slinuxm token create consolidator-<host>`), so a single `token revoke` on a peer cuts that consolidator off and nothing else. Replacing the file is enough to rotate: no restart.
`token_file` holds a path, never the token. A consolidator has to replay the peer's plaintext bearer token, so the token lives in a file managed outside `slinuxm` (agenix on NixOS), is read into memory on every check and request, and never appears in the config, `--print-config`, the database or the logs. Mint **one token per consolidator on every peer** and use it only there (`sudo -u <service user> slinuxm token create consolidator-<host>`), so a single `token revoke` on a peer cuts that consolidator off and nothing else. Replacing the file is enough to rotate: the next check reads the new value, and no restart is needed. On NixOS also run `systemctl reload skyg-linux-tracker` so systemd re-copies the credential the unit holds (`RefreshOnReload=credentials`).
Two checks per peer, kept in memory and reset on restart:
+13 -5
View File
@@ -109,7 +109,7 @@ The module renders the peers and the credentials; you only declare them:
```nix
services.skyg-linux-tracker = {
enable = true;
bind = "0.0.0.0";
bind = "0.0.0.0"; # inbound, only if callers reach this host from other machines
openFirewall = true; # or firewallInterfaces = [ "enp3s0" ];
peers = {
enable = true;
@@ -168,7 +168,7 @@ agenix -e secrets/peer-nas.age # paste the token printed on nas
services.skyg-linux-tracker = {
enable = true;
bind = "0.0.0.0";
bind = "0.0.0.0"; # inbound for remote callers; peers need no inbound port
openFirewall = true;
peers = {
enable = true;
@@ -202,6 +202,10 @@ The module turns that into one `[[peers.peer]]` entry per enabled peer, with `to
Outside NixOS the same split applies by hand: SIGHUP re-reads the TLS pair and `[peers]` (peer entries, intervals, timeouts, `fail_threshold`); everything else needs a restart.
### Shutdown
SIGTERM stops the listener, cancels the shared token and then drains. The health loops end on that token, and a proxied follow stream is closed rather than held until the peer decides to stop sending: shutdown ends the SSE stream to your caller even when the peer keeps its socket open (`tests/peers.rs` asserts exactly that against a listener that never closes). A buffered proxied request is bounded by `request_timeout_secs` and a follow stream by the peer's own cap, so no peer connection outlives the process by more than one timeout.
## Firewall and networking
Every **peer** must accept the consolidator's address on its port; the consolidator only needs outbound access. On a peer's NixOS host that is `openFirewall = true` or `firewallInterfaces = [ "enp3s0" ]`; a peer bound to `127.0.0.1` is not reachable from another machine, so a consolidator needs the peer to bind a LAN address (with auth on, which the module enforces) and either TLS or a trusted network.
@@ -223,13 +227,17 @@ DNS rebinding is still not a risk: the set is resolved once per request and pinn
## Testing
The peers acceptance test is a NixOS VM test. Three machines run the real `slinuxm`: `alpha` and `beta` consolidate each other over plain HTTP, and `alpha` also declares the TLS-only `gamma` with a private CA. It needs KVM and takes minutes.
Two layers, neither of which needs a VM for the first:
- `tests/peers.rs` drives the whole path in-process: a consolidator served by the real router and registry with the real outbound client, against peers that are either another in-process `slinuxm` (fake sources, real auth with a minted token) or a hand-written listener for a peer that misbehaves. It covers a plain peer and a TLS peer with its own CA, a peer signed by the wrong CA, a redirect, an oversize (declared and chunked) body, a silent peer, a slow-loris peer, the 64-peer cap, shutdown ending a proxied follow, and a peer declared by name that listens on one address family only.
- The NixOS VM test boots three machines running the real `slinuxm`: `alpha` and `beta` consolidate each other over plain HTTP, and `alpha` also declares the TLS-only `gamma` with a private CA (bound IPv4 only, so the address-family fallback stays guarded). It asserts status, fan-in, proxied data against a direct call, MCP parity, token install/rotation/revocation, peer removal on SIGHUP and an outage, all with `MainPID` unchanged where a reload is promised. `nix flake check` also runs `checks.module-peers`, an evaluation-only case list for the module: every rendering and assertion fails the check without booting anything.
```sh
devenv shell -- dev-test vm peers # nix build .#tests.x86_64-linux.peers -L
devenv shell -- dev-test integration peers # cargo test --test peers
devenv shell -- dev-test vm peers # nix build .#checks.x86_64-linux.vm-peers -L
```
Against a tree without the consolidator the test stops at the first peers assertion; that is expected, the test is written ahead of the feature. `dev/dev-test.nu` is the repository's test runner. `devenv shell -- dev-test --help` lists its subcommands and the command behind each. In an interactive `devenv shell`, `dev-test …` is that same script.
`dev/dev-test.nu` is the repository's test runner. `devenv shell -- dev-test --help` lists its subcommands and the command behind each. In an interactive `devenv shell`, `dev-test …` is that same script.
In Nushell the runner is a module. Load it once, for example in `config.nu`, and `dev-test unit`, `dev-test vm peers`, ... complete natively. Integration-test name completion is cached in `target/skyg-test-filters.cache` at the repo root (not relative to the current directory, so completion works from any subdirectory), keyed on the newest mtime of the Rust sources and the Cargo manifests. A failing check raises a catchable error instead of exiting, so a red `dev-test unit` does not close the Nushell session that loaded the module; script mode still exits non-zero.
+1 -1
View File
@@ -338,7 +338,7 @@ pkgs.testers.runNixOSTest {
assert http(alpha, base, A, "-X POST")[0] == 405
assert http(alpha, base, A, "-X DELETE")[0] == 405
status, _ = http(alpha, base, A, "-H 'x-slinuxm-hop: 1'")
assert 400 <= status < 500, status
assert status == 400, status
assert http(alpha, f"http://127.0.0.1:{PORT}/api/v1/peers/nope", A)[0] == 404
with subtest("token rejected: revoke on beta, then restore with a new token file"):