T: rename runner to dev-test
This commit is contained in:
1 parent
6dade6239d
commit
b476ac2b0c
6 files changed
+331
-219
No files matched your search
@@ -4,12 +4,12 @@ Part of [Phase 5: Consolidator](README.md). Contracts: [contracts.md](contracts.
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Status | T1 to T7 done; review round 1 addressed. `peers.nix` fails at the first peers assertion until WS-E lands (by design) |
|
||||
| Status | T1 to T7 done; review round 1 addressed; runner renamed to `dev-test` (`dev/dev-test.nu`). `peers.nix` fails at the first peers assertion until WS-E lands (by design) |
|
||||
| Owner | |
|
||||
| Depends on | none (uses the module's freeform `settings` for `[peers]` and a systemd override for `LoadCredential`, so it needs no module changes). F1 later swaps this for the first-class module options |
|
||||
| Blocks | nothing technically; it is the acceptance gate: goes green at the end of [WS-E](ws-e-telemetry.md) (with A, B, C, D landed), then F2 promotes it to `checks` |
|
||||
| Can start when | day one |
|
||||
| Owned files | `nix/tests/peers.nix` (done), `flake.nix` (the `tests.<system>.peers` output only), `dev/test.nu`, `devenv.nix` (nushell package, `test` script, completion hook), `AGENTS.md` and `README.md` (command lists only, see seams), `docs/consolidator.md` (creates the file with only a `## Testing` section, see seams) |
|
||||
| Owned files | `nix/tests/peers.nix` (done), `flake.nix` (the `tests.<system>.peers` output only), `dev/dev-test.nu`, `devenv.nix` (nushell package, `dev-test` script, completion hook), `AGENTS.md` and `README.md` (command lists only, see seams), `docs/consolidator.md` (creates the file with only a `## Testing` section, see seams) |
|
||||
| Do not touch | `src/**` (all streams), `nix/module.nix`, `nix/tests/vm.nix`, `systemd/skyg-linux-tracker.service` (F), `config.example.toml` (A/F), `tests/peers.rs` and `tests/common/mod.rs` (F; in-process variants by A/B live in their own test files, see [ws-a](ws-a-config.md), [ws-b](ws-b-registry.md)), `docs/api.md` (C) |
|
||||
|
||||
Shared-file seams:
|
||||
@@ -17,7 +17,7 @@ Shared-file seams:
|
||||
- `flake.nix`: T owns it until F2. F2 later moves exactly one attribute, `tests.${system}.peers`, into `checks` (rename `checks.vm-peers`). Nobody else edits `flake.nix`; it already carries the uncommitted peers output.
|
||||
- `devenv.nix`: T only.
|
||||
- `README.md`: T edits only the command/test list near the existing commands; F4 owns the new peers section. Different headings, so no overlap. `AGENTS.md`: T replaces the command list; F4 adds only the MCP tool-count note.
|
||||
- `docs/consolidator.md`: T creates it with a single `## Testing` section (the one-line `use dev/test.nu *` and the completion setup). F4 adds every other section and must not edit `## Testing`.
|
||||
- `docs/consolidator.md`: T creates it with a single `## Testing` section (the one-line `use dev/dev-test.nu *` and the completion setup). F4 adds every other section and must not edit `## Testing`.
|
||||
|
||||
## Goal
|
||||
|
||||
@@ -33,18 +33,18 @@ Note on the written test: `nix/tests/peers.nix` also contains a third node `gamm
|
||||
- **T4** token rejected: revoke the token on `beta`; `alpha` reports server up, api up, `auth = rejected`, `peer.api.auth_ok = 0`, and proxy calls fail with 503 naming the peer, without becoming a 401 for the caller; restore with a new token file and no restart, status recovers. **Done** in the script except the `peer.api.auth_ok = 0` metric check (needs WS-E; add it to the script, or leave to F2 which may query the collector/metrics endpoint, note in Log).
|
||||
- **T5** outage: stop `beta`'s service: `alpha` flips server and api after `fail_threshold`, `alpha`'s own endpoints stay healthy, and recovery is detected; stop `beta` entirely (VM down) and check that `alpha`'s requests are bounded by timeouts. **Done** in the script.
|
||||
- **T6** no chaining and read-only: a request carrying `x-slinuxm-hop` is rejected; non-GET to a proxy path is 405; a TLS variant with a test CA via `ca_file`. **Done** (405 for POST/DELETE, hop header 4xx, `gamma` over HTTPS).
|
||||
- **T7** **Done**: `dev/test.nu` with `unit`, `integration`, `lint`, `vm`, `all`, `ci`, `completions`, `self-check`; devenv `test` script plus zsh/bash completion hook; AGENTS.md and README.md command lists; `docs/consolidator.md` `## Testing`. A simple test runner CLI written in **Nushell**: `dev/test.nu` (devenv adds `pkgs.nushell`; also exposed as the devenv script `test`, so `devenv shell -- test unit` works from any shell). One entry point so contributors and agents do not memorise cargo/nix incantations. Structure it as a Nushell script with a `main` and `main <sub>` subcommands (`def "main unit"`, ...) so `--help` is generated and completion comes from the typed signatures:
|
||||
- `test unit` runs `cargo test --lib --bins` (no sockets, no host access).
|
||||
- `test integration [filter]` runs `cargo test --test '*'` (integration targets only: plain, tls, integration, binary_smoke, `peers` once it exists), so `test all` does not run the unit tests twice. `filter` is passed to cargo as a name filter. `--nocapture` is a libtest flag and is always placed after `--`.
|
||||
- `test lint` runs `cargo fmt --check`, `cargo clippy --all-targets -- -D warnings` and the `cargo tree -i openssl` emptiness check.
|
||||
- `test vm [name]` runs a NixOS VM test. `name` is one of `vm` (from `checks`) or `peers` (`nix build .#tests.x86_64-linux.peers -L`; moves to `checks` when green). A missing `/dev/kvm` fails that step and says why, after earlier `ci` steps, and the summary table still prints. The step prints that the build is slow.
|
||||
- `test all` = lint, unit, integration, stopping at the first failure; `test ci` = `all` plus `vm`. No args prints help.
|
||||
- **T7** **Done**: `dev/dev-test.nu` with `unit`, `integration`, `lint`, `vm`, `all`, `ci`, `completions`, `self-check`; devenv `dev-test` script plus zsh/bash completion hook; AGENTS.md and README.md command lists; `docs/consolidator.md` `## Testing`. A simple test runner CLI written in **Nushell**: `dev/dev-test.nu` (devenv adds `pkgs.nushell`; also exposed as the devenv script `dev-test`, so `devenv shell -- dev-test unit` works from any shell, including an interactive one). One entry point so contributors and agents do not memorise cargo/nix incantations. The command was renamed from `test` to `dev-test` (user decision, see Log) because `test` is a shell builtin. Structure it as a Nushell script with a `main` and subcommands literally named `dev-test <sub>` (`export def "dev-test unit"`, ...) so `--help` is generated and completion comes from the typed signatures. Nushell binds `main` to the file stem, so only commands named `dev-test ...` complete as `dev-test unit`:
|
||||
- `dev-test unit` runs `cargo test --lib --bins` (no sockets, no host access).
|
||||
- `dev-test integration [filter]` runs `cargo test --test '*'` (integration targets only: plain, tls, integration, binary_smoke, `peers` once it exists), so `dev-test all` does not run the unit tests twice. `filter` is passed to cargo as a name filter. `--nocapture` is a libtest flag and is always placed after `--`.
|
||||
- `dev-test lint` runs `cargo fmt --check`, `cargo clippy --all-targets -- -D warnings` and the `cargo tree -i openssl` emptiness check.
|
||||
- `dev-test vm [name]` runs a NixOS VM test. `name` is one of `vm` (from `checks`) or `peers` (`nix build .#tests.x86_64-linux.peers -L`; moves to `checks` when green). A missing `/dev/kvm` fails that step and says why, after earlier `ci` steps, and the summary table still prints. The step prints that the build is slow.
|
||||
- `dev-test all` = lint, unit, integration, stopping at the first failure; `dev-test ci` = `all` plus `vm`. No args prints help.
|
||||
- Each step prints a header and a final table (step, result, seconds); exit code non-zero on any failure (use `error make`/`exit 1`, never swallow cargo's exit status). Read-only against the host: no `/etc`, systemd or Docker changes (live-host checks stay manual, see Verification plan in the [README](README.md#verification-plan)).
|
||||
- **Autocomplete (must work, and is a done-criterion):**
|
||||
- Nushell: subcommands and flags complete natively once the script is loaded as a module. Dynamic values use custom completers (`def "nu-complete test vm" [] { ["vm" "peers"] }` and `name: string@"nu-complete test vm"`; integration filters complete from `cargo test --test '*' -- --list`, cached in `target/skyg-test-filters.cache` and keyed on the newest mtime of the Rust sources and Cargo manifests). A completer cannot update the caller's environment, so an env var does not survive Tab. `docs/consolidator.md` shows the one-line `use dev/test.nu *` for a permanent setup.
|
||||
- Other shells (the repo owner's login shell is zsh): `nu dev/test.nu completions zsh|bash` prints a completion script generated from the same command table. `test` is a shell builtin, so in an interactive bash or zsh `test unit` exits 0 without running the suite. The script defines a `test` function that sends known subcommands to the runner and otherwise calls `builtin test`, which keeps `test -f` working. `devenv shell -- test …` does not need the function: it finds the devenv script on PATH. Interactive `devenv shell` sources the bash script and prints a one-line note on stderr. zsh is not started by that hook; after `compinit`, zsh users add `source <(nu dev/test.nu completions zsh)` to `.zshrc` themselves. direnv does not carry the function.
|
||||
- Verified by a self-test: `test --help` and `test vm --help` list every subcommand/flag, and a check in `dev/test.nu` (`test self-check`) fails if the completion table and the defined subcommands diverge.
|
||||
- AGENTS.md and README replace their scattered command lists with `test ...`, keeping the underlying commands visible in `--help`.
|
||||
- Nushell: subcommands and flags complete natively once the script is loaded as a module. Dynamic values use custom completers (`def "nu-complete dev-test vm" [] { ["vm" "peers"] }` and `name: string@"nu-complete dev-test vm"`; integration filters complete from `cargo test --test '*' -- --list`, cached in `target/skyg-test-filters.cache` and keyed on the newest mtime of the Rust sources and Cargo manifests). A completer cannot update the caller's environment, so an env var does not survive Tab. `docs/consolidator.md` shows the one-line `use dev/dev-test.nu *` for a permanent setup.
|
||||
- Other shells (the repo owner's login shell is zsh): `nu dev/dev-test.nu completions zsh|bash` prints a completion script generated from the same command table. `dev-test` is not a builtin, so the script only registers completion (`compadd` in zsh, `compgen` in bash) and does not define a function. Interactive `dev-test …` runs the devenv script on PATH. `self-check` invokes those completion functions and fails if the candidate list is empty. Interactive `devenv shell` sources the bash script and prints nothing, so `devenv shell -- dev-test completions zsh` stays pipe-clean. zsh is not started by that hook; after `compinit`, zsh users add `source <(nu dev/dev-test.nu completions zsh)` to `.zshrc` themselves. direnv does not source it for them.
|
||||
- Verified by a self-test: `dev-test --help` and `dev-test vm --help` list every subcommand/flag, and a check in `dev/dev-test.nu` (`dev-test self-check`) fails if the completion table and the defined subcommands diverge.
|
||||
- AGENTS.md and README replace their scattered command lists with `dev-test ...`, keeping the underlying commands visible in `--help`.
|
||||
|
||||
## Inputs
|
||||
|
||||
@@ -53,13 +53,13 @@ Note on the written test: `nix/tests/peers.nix` also contains a third node `gamm
|
||||
|
||||
## Done when
|
||||
|
||||
The test builds under `nix build .#tests.x86_64-linux.peers` (or equivalent) and evaluates, fails cleanly at the first peers assertion against the current binary, and is documented in `docs/consolidator.md`. `test unit|integration|lint|vm|all` each run and report correctly on the current tree, and tab completion works in Nushell, zsh and bash (T7). It must pass at the end of WS-E, then move to `checks.vm-peers` (F2 does the move).
|
||||
The test builds under `nix build .#tests.x86_64-linux.peers` (or equivalent) and evaluates, fails cleanly at the first peers assertion against the current binary, and is documented in `docs/consolidator.md`. `dev-test unit|integration|lint|vm|all` each run and report correctly on the current tree, and tab completion works in Nushell, zsh and bash (T7). It must pass at the end of WS-E, then move to `checks.vm-peers` (F2 does the move).
|
||||
|
||||
## Handoff
|
||||
|
||||
- To everyone: the `test` runner and its subcommand names (referenced by AGENTS.md/README and by every stream's "Done when").
|
||||
- To everyone: the `dev-test` runner and its subcommand names (referenced by AGENTS.md/README and by every stream's "Done when").
|
||||
- To WS-F: a passing `peers.nix` (ready to promote) and any changes made to its assumptions, recorded in contracts.md section 10.
|
||||
- To WS-A/B: the `test integration` step picks up any `tests/peers*.rs` automatically.
|
||||
- To WS-A/B: the `dev-test integration` step picks up any `tests/peers*.rs` automatically.
|
||||
|
||||
## Log
|
||||
|
||||
@@ -78,3 +78,10 @@ The test builds under `nix build .#tests.x86_64-linux.peers` (or equivalent) and
|
||||
9. Script mode is `--wrapped`, so flags reach our parser. Unknown flags and extra positionals are errors. `--nocapture` is valid only on `integration`. `self-check` covers the parser and the CLI forms `unit --nocapture` and `integration a b`.
|
||||
10. `TODO.md:28` T7 remains unticked on purpose.
|
||||
- Handoff 2026-10-19 (WS-T review round 1 -> all): same subcommand names. Invoke as `devenv shell -- test <subcommand>` or `nu dev/test.nu <subcommand>`. Interactive `devenv shell` installs the bash wrapper; zsh needs the `.zshrc` line after `compinit`. `nu dev/test.nu self-check` is the runner's test. Not run here (no nix builds in this worktree): `nu dev/test.nu vm peers`, which on a KVM host is `nix build .#tests.x86_64-linux.peers -L`. `nix/tests/peers.nix` is unchanged.
|
||||
- 2026-10-19 (WS-T, user decision): Renamed the runner from `test` to `dev-test` and the module from `dev/test.nu` to `dev/dev-test.nu`. Supersedes the review-round-1 note that kept the name `test`. `test` is a shell builtin, so an interactive `test unit` never reached the runner and the completion script had to define a `test` function that fell through to `builtin test`. `dev-test` is an external command: `devenv shell -- dev-test unit|integration|lint|vm|all|ci` and an interactive `dev-test …` both run the script. Completion scripts only register completion (`compadd` / `compgen`); they do not define a function. `_describe` emitted no candidates outside a completion widget, so zsh now uses `compadd` and `self-check` fails if the bash or zsh candidate list is empty. Nushell commands are named `dev-test <sub>` because the file stem is `dev-test`. argv/`--nocapture` and the filter cache are unchanged. `nix/tests/peers.nix` is unchanged. Also fixed a latent help bug: the `<(...)` example sat in a double-quoted string, so Nushell ran the completions command and `--help` inlined the whole zsh script; the line is concatenated now and `self-check` rejects an embedded script. Earlier dated Log entries keep the old name because they record what happened then; the Status line, T7 text and Handoff use the current name.
|
||||
Stale `dev/test.nu` names left for the merger (other streams' files, not edited here):
|
||||
- `plan/phase-5-consolidator/README.md` workstream index
|
||||
- `plan/phase-5-consolidator/ws-a-config.md` Do not touch
|
||||
- `plan/phase-5-consolidator/ws-f-nixos-docs.md` Do not touch
|
||||
- `TODO.md` T7 line (unticked on purpose; the merger ticks it)
|
||||
- Handoff 2026-10-19 (WS-T rename -> all): invoke the runner as `devenv shell -- dev-test <subcommand>` or `nu dev/dev-test.nu <subcommand>`. Interactive bash completion is installed by `devenv shell`; zsh, after `compinit`: `source <(nu dev/dev-test.nu completions zsh)`. Nushell: `use dev/dev-test.nu *`. `nu dev/dev-test.nu self-check` is the runner's test. Verified here: `nu dev/dev-test.nu self-check` and `nu dev/dev-test.nu all` (fmt, clippy, openssl, unit, integration) green, plus `devenv shell -- dev-test --help`. Not run here (no nix builds in this worktree): `nu dev/dev-test.nu vm peers`, which on a KVM host is `nix build .#tests.x86_64-linux.peers -L`. `nix/tests/peers.nix` is unchanged.
|
||||
Reference in new issue
Block a user