F: correct the peers docs against the finished behaviour
docs/consolidator.md: the Testing section described the pre-feature test (no tests/peers.rs, the old tests.<system>.peers attribute, a note about assertions failing by design) and is rewritten for both layers, the module's eval check included. A shutdown subsection backs the README's promise, the agenix examples say why a consolidator would bind 0.0.0.0 (remote callers; peers need no inbound port), and the VM test asserts the chained-request status is exactly 400 as documented. README: token rotation on NixOS also needs a reload for the LoadCredential copy.
This commit is contained in:
1 parent
0a83386de5
commit
04992e202d
3 files changed
+15
-7
No files matched your search
@@ -131,7 +131,7 @@ token_file = "/etc/skyg/peer-nas.token" # 0600, owned by the service user
|
||||
allow_plain_http = true
|
||||
```
|
||||
|
||||
`token_file` holds a path, never the token. A consolidator has to replay the peer's plaintext bearer token, so the token lives in a file managed outside `slinuxm` (agenix on NixOS), is read into memory on every check and request, and never appears in the config, `--print-config`, the database or the logs. Mint **one token per consolidator on every peer** and use it only there (`sudo -u <service user> slinuxm token create consolidator-<host>`), so a single `token revoke` on a peer cuts that consolidator off and nothing else. Replacing the file is enough to rotate: no restart.
|
||||
`token_file` holds a path, never the token. A consolidator has to replay the peer's plaintext bearer token, so the token lives in a file managed outside `slinuxm` (agenix on NixOS), is read into memory on every check and request, and never appears in the config, `--print-config`, the database or the logs. Mint **one token per consolidator on every peer** and use it only there (`sudo -u <service user> slinuxm token create consolidator-<host>`), so a single `token revoke` on a peer cuts that consolidator off and nothing else. Replacing the file is enough to rotate: the next check reads the new value, and no restart is needed. On NixOS also run `systemctl reload skyg-linux-tracker` so systemd re-copies the credential the unit holds (`RefreshOnReload=credentials`).
|
||||
|
||||
Two checks per peer, kept in memory and reset on restart:
|
||||
|
||||
|
||||
+13
-5
@@ -109,7 +109,7 @@ The module renders the peers and the credentials; you only declare them:
|
||||
```nix
|
||||
services.skyg-linux-tracker = {
|
||||
enable = true;
|
||||
bind = "0.0.0.0";
|
||||
bind = "0.0.0.0"; # inbound, only if callers reach this host from other machines
|
||||
openFirewall = true; # or firewallInterfaces = [ "enp3s0" ];
|
||||
peers = {
|
||||
enable = true;
|
||||
@@ -168,7 +168,7 @@ agenix -e secrets/peer-nas.age # paste the token printed on nas
|
||||
|
||||
services.skyg-linux-tracker = {
|
||||
enable = true;
|
||||
bind = "0.0.0.0";
|
||||
bind = "0.0.0.0"; # inbound for remote callers; peers need no inbound port
|
||||
openFirewall = true;
|
||||
peers = {
|
||||
enable = true;
|
||||
@@ -202,6 +202,10 @@ The module turns that into one `[[peers.peer]]` entry per enabled peer, with `to
|
||||
|
||||
Outside NixOS the same split applies by hand: SIGHUP re-reads the TLS pair and `[peers]` (peer entries, intervals, timeouts, `fail_threshold`); everything else needs a restart.
|
||||
|
||||
### Shutdown
|
||||
|
||||
SIGTERM stops the listener, cancels the shared token and then drains. The health loops end on that token, and a proxied follow stream is closed rather than held until the peer decides to stop sending: shutdown ends the SSE stream to your caller even when the peer keeps its socket open (`tests/peers.rs` asserts exactly that against a listener that never closes). A buffered proxied request is bounded by `request_timeout_secs` and a follow stream by the peer's own cap, so no peer connection outlives the process by more than one timeout.
|
||||
|
||||
## Firewall and networking
|
||||
|
||||
Every **peer** must accept the consolidator's address on its port; the consolidator only needs outbound access. On a peer's NixOS host that is `openFirewall = true` or `firewallInterfaces = [ "enp3s0" ]`; a peer bound to `127.0.0.1` is not reachable from another machine, so a consolidator needs the peer to bind a LAN address (with auth on, which the module enforces) and either TLS or a trusted network.
|
||||
@@ -223,13 +227,17 @@ DNS rebinding is still not a risk: the set is resolved once per request and pinn
|
||||
|
||||
## Testing
|
||||
|
||||
The peers acceptance test is a NixOS VM test. Three machines run the real `slinuxm`: `alpha` and `beta` consolidate each other over plain HTTP, and `alpha` also declares the TLS-only `gamma` with a private CA. It needs KVM and takes minutes.
|
||||
Two layers, neither of which needs a VM for the first:
|
||||
|
||||
- `tests/peers.rs` drives the whole path in-process: a consolidator served by the real router and registry with the real outbound client, against peers that are either another in-process `slinuxm` (fake sources, real auth with a minted token) or a hand-written listener for a peer that misbehaves. It covers a plain peer and a TLS peer with its own CA, a peer signed by the wrong CA, a redirect, an oversize (declared and chunked) body, a silent peer, a slow-loris peer, the 64-peer cap, shutdown ending a proxied follow, and a peer declared by name that listens on one address family only.
|
||||
- The NixOS VM test boots three machines running the real `slinuxm`: `alpha` and `beta` consolidate each other over plain HTTP, and `alpha` also declares the TLS-only `gamma` with a private CA (bound IPv4 only, so the address-family fallback stays guarded). It asserts status, fan-in, proxied data against a direct call, MCP parity, token install/rotation/revocation, peer removal on SIGHUP and an outage, all with `MainPID` unchanged where a reload is promised. `nix flake check` also runs `checks.module-peers`, an evaluation-only case list for the module: every rendering and assertion fails the check without booting anything.
|
||||
|
||||
```sh
|
||||
devenv shell -- dev-test vm peers # nix build .#tests.x86_64-linux.peers -L
|
||||
devenv shell -- dev-test integration peers # cargo test --test peers
|
||||
devenv shell -- dev-test vm peers # nix build .#checks.x86_64-linux.vm-peers -L
|
||||
```
|
||||
|
||||
Against a tree without the consolidator the test stops at the first peers assertion; that is expected, the test is written ahead of the feature. `dev/dev-test.nu` is the repository's test runner. `devenv shell -- dev-test --help` lists its subcommands and the command behind each. In an interactive `devenv shell`, `dev-test …` is that same script.
|
||||
`dev/dev-test.nu` is the repository's test runner. `devenv shell -- dev-test --help` lists its subcommands and the command behind each. In an interactive `devenv shell`, `dev-test …` is that same script.
|
||||
|
||||
In Nushell the runner is a module. Load it once, for example in `config.nu`, and `dev-test unit`, `dev-test vm peers`, ... complete natively. Integration-test name completion is cached in `target/skyg-test-filters.cache` at the repo root (not relative to the current directory, so completion works from any subdirectory), keyed on the newest mtime of the Rust sources and the Cargo manifests. A failing check raises a catchable error instead of exiting, so a red `dev-test unit` does not close the Nushell session that loaded the module; script mode still exits non-zero.
|
||||
|
||||
|
||||
+1
-1
@@ -338,7 +338,7 @@ pkgs.testers.runNixOSTest {
|
||||
assert http(alpha, base, A, "-X POST")[0] == 405
|
||||
assert http(alpha, base, A, "-X DELETE")[0] == 405
|
||||
status, _ = http(alpha, base, A, "-H 'x-slinuxm-hop: 1'")
|
||||
assert 400 <= status < 500, status
|
||||
assert status == 400, status
|
||||
assert http(alpha, f"http://127.0.0.1:{PORT}/api/v1/peers/nope", A)[0] == 404
|
||||
|
||||
with subtest("token rejected: revoke on beta, then restore with a new token file"):
|
||||
|
||||
Reference in new issue
Block a user