67 lines
8.9 KiB
Markdown
67 lines
8.9 KiB
Markdown
# WS-T: Integration test suite (written first, user request)
|
|
|
|
Part of [Phase 5: Consolidator](README.md). Contracts: [contracts.md](contracts.md).
|
|
|
|
| Field | Value |
|
|
|---|---|
|
|
| Status | In progress: `nix/tests/peers.nix` is written; T7 runner and docs open |
|
|
| Owner | |
|
|
| Depends on | none (uses the module's freeform `settings` for `[peers]` and a systemd override for `LoadCredential`, so it needs no module changes). F1 later swaps this for the first-class module options |
|
|
| Blocks | nothing technically; it is the acceptance gate: goes green at the end of [WS-E](ws-e-telemetry.md) (with A, B, C, D landed), then F2 promotes it to `checks` |
|
|
| Can start when | day one |
|
|
| Owned files | `nix/tests/peers.nix` (done), `flake.nix` (the `tests.<system>.peers` output only), `dev/test.nu`, `devenv.nix` (nushell package, `test` script, completion hook), `AGENTS.md` and `README.md` (command lists only, see seams), `docs/consolidator.md` (creates the file with only a `## Testing` section, see seams) |
|
|
| Do not touch | `src/**` (all streams), `nix/module.nix`, `nix/tests/vm.nix`, `systemd/skyg-linux-tracker.service` (F), `config.example.toml` (A/F), `tests/peers.rs` and `tests/common/mod.rs` (F; in-process variants by A/B live in their own test files, see [ws-a](ws-a-config.md), [ws-b](ws-b-registry.md)), `docs/api.md` (C) |
|
|
|
|
Shared-file seams:
|
|
|
|
- `flake.nix`: T owns it until F2. F2 later moves exactly one attribute, `tests.${system}.peers`, into `checks` (rename `checks.vm-peers`). Nobody else edits `flake.nix`; it already carries the uncommitted peers output.
|
|
- `devenv.nix`: T only.
|
|
- `README.md`: T edits only the command/test list near the existing commands; F4 owns the new peers section. Different headings, so no overlap. `AGENTS.md`: T replaces the command list; F4 adds only the MCP tool-count note.
|
|
- `docs/consolidator.md`: T creates it with a single `## Testing` section (the one-line `use dev/test.nu *` and the completion setup). F4 adds every other section and must not edit `## Testing`.
|
|
|
|
## Goal
|
|
|
|
Executable acceptance criteria before the implementation. Two NixOS VMs each run the real `slinuxm` binary and consolidate each other: VM `alpha` declares `beta` as a peer, VM `beta` declares `alpha`, and each proxies the other's data. Expected to fail until WS-E lands, which is the point.
|
|
|
|
Note on the written test: `nix/tests/peers.nix` also contains a third node `gamma` (TLS-only, no peers) that `alpha` declares with a private CA via `ca_file`, covering the TLS variant. It is already written and, by design, fails at the first peers assertion (phase 2, "declared but token is still a placeholder -> auth rejected") against the current binary; phase 1 (boot, healthz, token minting, marker units, token-file placeholders) already passes.
|
|
|
|
## Tasks
|
|
|
|
- **T1** two machines `alpha` and `beta` with distinct hostnames, each with a distinct fake workload (a marker systemd unit `marker-alpha.service` / `marker-beta.service`, and a deliberately failing unit on one side) so proxied data is distinguishable from local data. **Done** in `nix/tests/peers.nix` (failing unit is `broken-beta.service`; `gamma` with `marker-gamma.service` added).
|
|
- **T2** each mints a read-only token via `sudo -u skyg-linux-tracker slinuxm token create consolidator-<other>`, writes the other's token to a root-only 0400 file (stand-in for an agenix path), and declares the other as a peer with that `token_file`. **Done** (files under `/var/lib/peer-tokens`, delivered via `LoadCredential`).
|
|
- **T3** assertions on both directions: `/api/v1/peers` shows the peer with server up, api up and auth ok; `GET /api/v1/peers/<other>/systemd/units/marker-<other>.service` returns the other machine's unit and not a local one; MCP `list_peers` and a `list_systemd_units` call with `peer` return the same data; response header `x-slinuxm-peer`. **Done** in the test script.
|
|
- **T4** token rejected: revoke the token on `beta`; `alpha` reports server up, api up, `auth = rejected`, `peer.api.auth_ok = 0`, and proxy calls fail with 503 naming the peer, without becoming a 401 for the caller; restore with a new token file and no restart, status recovers. **Done** in the script except the `peer.api.auth_ok = 0` metric check (needs WS-E; add it to the script, or leave to F2 which may query the collector/metrics endpoint, note in Log).
|
|
- **T5** outage: stop `beta`'s service: `alpha` flips server and api after `fail_threshold`, `alpha`'s own endpoints stay healthy, and recovery is detected; stop `beta` entirely (VM down) and check that `alpha`'s requests are bounded by timeouts. **Done** in the script.
|
|
- **T6** no chaining and read-only: a request carrying `x-slinuxm-hop` is rejected; non-GET to a proxy path is 405; a TLS variant with a test CA via `ca_file`. **Done** (405 for POST/DELETE, hop header 4xx, `gamma` over HTTPS).
|
|
- **T7** a simple test runner CLI written in **Nushell**: `dev/test.nu` (devenv adds `pkgs.nushell`; also exposed as the devenv script `test`, so `devenv shell -- test unit` works from any shell). One entry point so contributors and agents do not memorise cargo/nix incantations. Structure it as a Nushell script with a `main` and `main <sub>` subcommands (`def "main unit"`, ...) so `--help` is generated and completion comes from the typed signatures:
|
|
- `test unit` runs `cargo test --lib --bins` (no sockets, no host access).
|
|
- `test integration [filter]` runs `cargo test --tests` (plain, tls, integration, binary_smoke, `peers` once it exists). `filter` is passed to cargo as a name filter; `--nocapture` flag supported.
|
|
- `test lint` runs `cargo fmt --check`, `cargo clippy --all-targets -- -D warnings` and the `cargo tree -i openssl` emptiness check.
|
|
- `test vm [name]` runs a NixOS VM test. `name` is one of `vm` (from `checks`) or `peers` (`nix build .#tests.x86_64-linux.peers -L`; moves to `checks` when green). Refuses without `/dev/kvm` and says why; prints that it is slow.
|
|
- `test all` = lint, unit, integration, stopping at the first failure; `test ci` = `all` plus `vm`. No args prints help.
|
|
- Each step prints a header and a final table (step, result, seconds); exit code non-zero on any failure (use `error make`/`exit 1`, never swallow cargo's exit status). Read-only against the host: no `/etc`, systemd or Docker changes (live-host checks stay manual, see Verification plan in the [README](README.md#verification-plan)).
|
|
- **Autocomplete (must work, and is a done-criterion):**
|
|
- Nushell: subcommands and flags complete natively once the script is loaded as a module. Dynamic values use custom completers (`def "nu-complete test vm" [] { ["vm" "peers"] }` and `name: string@"nu-complete test vm"`; integration filters complete from `cargo test --tests -- --list` test names, cached for the session). The devenv `enterShell` prints/loads `use dev/test.nu *` for Nushell users, and `docs/consolidator.md` shows the one-line `use` for a permanent setup.
|
|
- Other shells (the repo owner's login shell is zsh): `test completions zsh|bash` prints a small static completion script (subcommands, `vm` names, flags) generated from the same command table in the script, so zsh/bash tab completion works through `devenv shell` too (`source <(test completions zsh)` in `enterShell`).
|
|
- Verified by a self-test: `test --help` and `test vm --help` list every subcommand/flag, and a check in `dev/test.nu` (`test self-check`) fails if the completion table and the defined subcommands diverge.
|
|
- AGENTS.md and README replace their scattered command lists with `test ...`, keeping the underlying commands visible in `--help`.
|
|
|
|
## Inputs
|
|
|
|
- [contracts.md](contracts.md) section 10 (test config contract) and sections 5 and 7 (REST/MCP shapes the script asserts). No fakes: the test runs the real binary.
|
|
- `nix/tests/peers.nix` is the source of truth for section 10; if the test must change, change it and update contracts.md in the same PR, then log in the affected streams.
|
|
|
|
## Done when
|
|
|
|
The test builds under `nix build .#tests.x86_64-linux.peers` (or equivalent) and evaluates, fails cleanly at the first peers assertion against the current binary, and is documented in `docs/consolidator.md`. `test unit|integration|lint|vm|all` each run and report correctly on the current tree, and tab completion works in Nushell, zsh and bash (T7). It must pass at the end of WS-E, then move to `checks.vm-peers` (F2 does the move).
|
|
|
|
## Handoff
|
|
|
|
- To everyone: the `test` runner and its subcommand names (referenced by AGENTS.md/README and by every stream's "Done when").
|
|
- To WS-F: a passing `peers.nix` (ready to promote) and any changes made to its assumptions, recorded in contracts.md section 10.
|
|
- To WS-A/B: the `test integration` step picks up any `tests/peers*.rs` automatically.
|
|
|
|
## Log
|
|
|
|
- 2026-10-19: Split from the single phase plan 2026-10-19. `nix/tests/peers.nix` already written (T1 to T6) and failing at the first peers assertion by design; `flake.nix` exposes it as `tests.<system>.peers`. T7 not started. Ownership change from the single plan: `tests/peers.rs` moved to WS-F; WS-A/B in-process tests use their own files to avoid collisions.
|