Files
skyg-linux-tracker/plan/phase-5-consolidator/ws-t-tests.md
T
2026-10-19 16:43:35 -04:00

8.9 KiB

WS-T: Integration test suite (written first, user request)

Part of Phase 5: Consolidator. Contracts: contracts.md.

Field Value
Status In progress: nix/tests/peers.nix is written; T7 runner and docs open
Owner
Depends on none (uses the module's freeform settings for [peers] and a systemd override for LoadCredential, so it needs no module changes). F1 later swaps this for the first-class module options
Blocks nothing technically; it is the acceptance gate: goes green at the end of WS-E (with A, B, C, D landed), then F2 promotes it to checks
Can start when day one
Owned files nix/tests/peers.nix (done), flake.nix (the tests.<system>.peers output only), dev/test.nu, devenv.nix (nushell package, test script, completion hook), AGENTS.md and README.md (command lists only, see seams), docs/consolidator.md (creates the file with only a ## Testing section, see seams)
Do not touch src/** (all streams), nix/module.nix, nix/tests/vm.nix, systemd/skyg-linux-tracker.service (F), config.example.toml (A/F), tests/peers.rs and tests/common/mod.rs (F; in-process variants by A/B live in their own test files, see ws-a, ws-b), docs/api.md (C)

Shared-file seams:

  • flake.nix: T owns it until F2. F2 later moves exactly one attribute, tests.${system}.peers, into checks (rename checks.vm-peers). Nobody else edits flake.nix; it already carries the uncommitted peers output.
  • devenv.nix: T only.
  • README.md: T edits only the command/test list near the existing commands; F4 owns the new peers section. Different headings, so no overlap. AGENTS.md: T replaces the command list; F4 adds only the MCP tool-count note.
  • docs/consolidator.md: T creates it with a single ## Testing section (the one-line use dev/test.nu * and the completion setup). F4 adds every other section and must not edit ## Testing.

Goal

Executable acceptance criteria before the implementation. Two NixOS VMs each run the real slinuxm binary and consolidate each other: VM alpha declares beta as a peer, VM beta declares alpha, and each proxies the other's data. Expected to fail until WS-E lands, which is the point.

Note on the written test: nix/tests/peers.nix also contains a third node gamma (TLS-only, no peers) that alpha declares with a private CA via ca_file, covering the TLS variant. It is already written and, by design, fails at the first peers assertion (phase 2, "declared but token is still a placeholder -> auth rejected") against the current binary; phase 1 (boot, healthz, token minting, marker units, token-file placeholders) already passes.

Tasks

  • T1 two machines alpha and beta with distinct hostnames, each with a distinct fake workload (a marker systemd unit marker-alpha.service / marker-beta.service, and a deliberately failing unit on one side) so proxied data is distinguishable from local data. Done in nix/tests/peers.nix (failing unit is broken-beta.service; gamma with marker-gamma.service added).
  • T2 each mints a read-only token via sudo -u skyg-linux-tracker slinuxm token create consolidator-<other>, writes the other's token to a root-only 0400 file (stand-in for an agenix path), and declares the other as a peer with that token_file. Done (files under /var/lib/peer-tokens, delivered via LoadCredential).
  • T3 assertions on both directions: /api/v1/peers shows the peer with server up, api up and auth ok; GET /api/v1/peers/<other>/systemd/units/marker-<other>.service returns the other machine's unit and not a local one; MCP list_peers and a list_systemd_units call with peer return the same data; response header x-slinuxm-peer. Done in the test script.
  • T4 token rejected: revoke the token on beta; alpha reports server up, api up, auth = rejected, peer.api.auth_ok = 0, and proxy calls fail with 503 naming the peer, without becoming a 401 for the caller; restore with a new token file and no restart, status recovers. Done in the script except the peer.api.auth_ok = 0 metric check (needs WS-E; add it to the script, or leave to F2 which may query the collector/metrics endpoint, note in Log).
  • T5 outage: stop beta's service: alpha flips server and api after fail_threshold, alpha's own endpoints stay healthy, and recovery is detected; stop beta entirely (VM down) and check that alpha's requests are bounded by timeouts. Done in the script.
  • T6 no chaining and read-only: a request carrying x-slinuxm-hop is rejected; non-GET to a proxy path is 405; a TLS variant with a test CA via ca_file. Done (405 for POST/DELETE, hop header 4xx, gamma over HTTPS).
  • T7 a simple test runner CLI written in Nushell: dev/test.nu (devenv adds pkgs.nushell; also exposed as the devenv script test, so devenv shell -- test unit works from any shell). One entry point so contributors and agents do not memorise cargo/nix incantations. Structure it as a Nushell script with a main and main <sub> subcommands (def "main unit", ...) so --help is generated and completion comes from the typed signatures:
    • test unit runs cargo test --lib --bins (no sockets, no host access).
    • test integration [filter] runs cargo test --tests (plain, tls, integration, binary_smoke, peers once it exists). filter is passed to cargo as a name filter; --nocapture flag supported.
    • test lint runs cargo fmt --check, cargo clippy --all-targets -- -D warnings and the cargo tree -i openssl emptiness check.
    • test vm [name] runs a NixOS VM test. name is one of vm (from checks) or peers (nix build .#tests.x86_64-linux.peers -L; moves to checks when green). Refuses without /dev/kvm and says why; prints that it is slow.
    • test all = lint, unit, integration, stopping at the first failure; test ci = all plus vm. No args prints help.
    • Each step prints a header and a final table (step, result, seconds); exit code non-zero on any failure (use error make/exit 1, never swallow cargo's exit status). Read-only against the host: no /etc, systemd or Docker changes (live-host checks stay manual, see Verification plan in the README).
    • Autocomplete (must work, and is a done-criterion):
      • Nushell: subcommands and flags complete natively once the script is loaded as a module. Dynamic values use custom completers (def "nu-complete test vm" [] { ["vm" "peers"] } and name: string@"nu-complete test vm"; integration filters complete from cargo test --tests -- --list test names, cached for the session). The devenv enterShell prints/loads use dev/test.nu * for Nushell users, and docs/consolidator.md shows the one-line use for a permanent setup.
      • Other shells (the repo owner's login shell is zsh): test completions zsh|bash prints a small static completion script (subcommands, vm names, flags) generated from the same command table in the script, so zsh/bash tab completion works through devenv shell too (source <(test completions zsh) in enterShell).
      • Verified by a self-test: test --help and test vm --help list every subcommand/flag, and a check in dev/test.nu (test self-check) fails if the completion table and the defined subcommands diverge.
    • AGENTS.md and README replace their scattered command lists with test ..., keeping the underlying commands visible in --help.

Inputs

  • contracts.md section 10 (test config contract) and sections 5 and 7 (REST/MCP shapes the script asserts). No fakes: the test runs the real binary.
  • nix/tests/peers.nix is the source of truth for section 10; if the test must change, change it and update contracts.md in the same PR, then log in the affected streams.

Done when

The test builds under nix build .#tests.x86_64-linux.peers (or equivalent) and evaluates, fails cleanly at the first peers assertion against the current binary, and is documented in docs/consolidator.md. test unit|integration|lint|vm|all each run and report correctly on the current tree, and tab completion works in Nushell, zsh and bash (T7). It must pass at the end of WS-E, then move to checks.vm-peers (F2 does the move).

Handoff

  • To everyone: the test runner and its subcommand names (referenced by AGENTS.md/README and by every stream's "Done when").
  • To WS-F: a passing peers.nix (ready to promote) and any changes made to its assumptions, recorded in contracts.md section 10.
  • To WS-A/B: the test integration step picks up any tests/peers*.rs automatically.

Log

  • 2026-10-19: Split from the single phase plan 2026-10-19. nix/tests/peers.nix already written (T1 to T6) and failing at the first peers assertion by design; flake.nix exposes it as tests.<system>.peers. T7 not started. Ownership change from the single plan: tests/peers.rs moved to WS-F; WS-A/B in-process tests use their own files to avoid collisions.