T: rename runner to dev-test

This commit is contained in:
ari committed 2026-10-19 22:37:08 -04:00
1 parent 6dade6239d
commit b476ac2b0c
6 files changed
+331 -219

No files matched your search

+11 -11
View File
@@ -18,21 +18,21 @@ Canonical guidance for AI coding agents in this repo (no CLAUDE.md; this file is
Enter the toolchain with `devenv shell` (Rust, pkg-config, libsystemd, nushell, otelcol). Without it you need a Rust toolchain, `pkg-config` and the libsystemd dev files.
All checks go through one runner, `dev/test.nu`. `devenv shell -- test …` runs the devenv script on PATH. In an interactive bash or zsh, `test` is the shell builtin, so `test unit` exits 0 without running the suite; use `devenv shell -- test unit` or `nu dev/test.nu unit`. An interactive `devenv shell` installs a bash function that runs known subcommands and leaves `test -f` to the builtin. zsh users add `source <(nu dev/test.nu completions zsh)` to `.zshrc` after `compinit`. `nu dev/test.nu --help` lists the underlying cargo and nix commands.
All checks go through one runner, `dev/dev-test.nu`. `devenv shell -- dev-test …` runs the devenv script on PATH, and the same command works in an interactive shell. zsh users add `source <(nu dev/dev-test.nu completions zsh)` to `.zshrc` after `compinit`. An interactive `devenv shell` installs bash completion. `nu dev/dev-test.nu --help` lists the underlying cargo and nix commands.
```sh
devenv shell -- test lint # cargo fmt --check, clippy -D warnings, cargo tree -i openssl (empty)
devenv shell -- test unit # cargo test --lib --bins
devenv shell -- test integration [filter] [--nocapture] # cargo test --test '*' (plain, tls, integration, binary_smoke, peers)
devenv shell -- test vm [vm|peers] # NixOS VM test (slow, needs /dev/kvm)
devenv shell -- test all # lint, unit, integration; stops at the first failure
devenv shell -- test ci # all plus the vm test
devenv shell -- smoke # real binary smoke test (cargo test --test binary_smoke -- --nocapture)
cargo build --release # target/release/slinuxm
devenv shell -- dev-test lint # cargo fmt --check, clippy -D warnings, cargo tree -i openssl (empty)
devenv shell -- dev-test unit # cargo test --lib --bins
devenv shell -- dev-test integration [filter] [--nocapture] # cargo test --test '*' (plain, tls, integration, binary_smoke, peers)
devenv shell -- dev-test vm [vm|peers] # NixOS VM test (slow, needs /dev/kvm)
devenv shell -- dev-test all # lint, unit, integration; stops at the first failure
devenv shell -- dev-test ci # all plus the vm test
devenv shell -- smoke # real binary smoke test (cargo test --test binary_smoke -- --nocapture)
cargo build --release # target/release/slinuxm
nix build && nix flake check # package, module, VM test (slow; Nix changes only)
```
`devenv up` runs a local OTLP collector (`dev/otelcol.yaml`). Do not run nix builds unless the task involves Nix files, and never while another agent is running the nix gate. When editing `dev/test.nu`, run `nu dev/test.nu self-check`; it fails if the command table, the subcommands, the help text, the cargo argv or the shell wrapper drift apart.
`devenv up` runs a local OTLP collector (`dev/otelcol.yaml`). Do not run nix builds unless the task involves Nix files, and never while another agent is running the nix gate. When editing `dev/dev-test.nu`, run `nu dev/dev-test.nu self-check`; it fails if the command table, the subcommands, the help text, the cargo argv or the shell completion drift apart.
## Conventions
@@ -63,7 +63,7 @@ nix build && nix flake check # package, module, VM te
## Definition of done
1. `devenv shell -- test all` (fmt, clippy, unit and integration tests) passes; that is `cargo fmt --check`, `cargo clippy --all-targets -- -D warnings` and `cargo test`.
1. `devenv shell -- dev-test all` (fmt, clippy, unit and integration tests) passes; that is `cargo fmt --check`, `cargo clippy --all-targets -- -D warnings` and `cargo test`.
2. New behaviour has tests (fakes for handlers and tools; `tests/` for end to end).
3. Docs updated: `README.md`, `config.example.toml`, `ARCHITECTURE.md` as relevant.
4. `TODO.md` updated; any deferred work is in the Backlog with a reason.
+10 -10
View File
@@ -178,16 +178,16 @@ ldd /usr/local/bin/slinuxm # libc, libm, libgcc_s, libsystemd and its depende
## Testing
Checks run through one runner, `dev/test.nu`. `devenv shell -- test …` runs it from any shell. In an interactive bash or zsh, `test` is the shell builtin, so `test unit` exits 0 without running the suite. `devenv shell -- test --help` lists every subcommand and the cargo or nix command behind it.
Checks run through one runner, `dev/dev-test.nu`. `devenv shell -- dev-test …` runs it from any shell, including an interactive one. `devenv shell -- dev-test --help` lists every subcommand and the cargo or nix command behind it.
```sh
devenv shell -- test unit # cargo test --lib --bins
devenv shell -- test integration [filter] [--nocapture] # cargo test --test '*' (integration targets only)
devenv shell -- test lint # cargo fmt --check, clippy -D warnings, no openssl
devenv shell -- test vm [vm|peers] # NixOS VM test (slow, needs /dev/kvm)
devenv shell -- test all # lint, unit, integration; stops at the first failure
devenv shell -- test ci # all plus the vm test
devenv shell -- smoke # same as: cargo test --test binary_smoke -- --nocapture
devenv shell -- dev-test unit # cargo test --lib --bins
devenv shell -- dev-test integration [filter] [--nocapture] # cargo test --test '*' (integration targets only)
devenv shell -- dev-test lint # cargo fmt --check, clippy -D warnings, no openssl
devenv shell -- dev-test vm [vm|peers] # NixOS VM test (slow, needs /dev/kvm)
devenv shell -- dev-test all # lint, unit, integration; stops at the first failure
devenv shell -- dev-test ci # all plus the vm test
devenv shell -- smoke # same as: cargo test --test binary_smoke -- --nocapture
```
The smoke test runs the real built `slinuxm` binary against a throwaway SQLite database in a temp directory: it creates a token with the CLI, starts `serve` on a free loopback port, exercises `/healthz`, REST and MCP with and without the token, revokes the token, and checks a clean SIGTERM exit and a 0600 database. It needs no root and no network. Checks that depend on host data (systemd over D-Bus, the journal) are skipped with a printed reason when the source is unavailable; Docker is never required.
@@ -195,7 +195,7 @@ The smoke test runs the real built `slinuxm` binary against a throwaway SQLite d
## Development
```sh
devenv shell -- test all # fmt, clippy, unit and integration tests
devenv shell -- dev-test all # fmt, clippy, unit and integration tests
```
`devenv shell -- test --help` shows the underlying commands (`cargo fmt --check`, `cargo clippy --all-targets -- -D warnings`, `cargo test --lib --bins`, `cargo test --test '*'`). Nushell: `use dev/test.nu *` gives `test unit`, `test vm peers`, ... with tab completion. zsh users add `source <(nu dev/test.nu completions zsh)` to `.zshrc` after `compinit`. An interactive `devenv shell` installs the bash function; `test -f` stays the builtin. The integration tests in `tests/` start the real router on an ephemeral port with fake sources. Design notes live in [ARCHITECTURE.md](ARCHITECTURE.md); work tracking in [TODO.md](TODO.md); extended docs in [docs/](docs/README.md).
`devenv shell -- dev-test --help` shows the underlying commands (`cargo fmt --check`, `cargo clippy --all-targets -- -D warnings`, `cargo test --lib --bins`, `cargo test --test '*'`). Nushell: `use dev/dev-test.nu *` gives `dev-test unit`, `dev-test vm peers`, ... with tab completion. zsh users add `source <(nu dev/dev-test.nu completions zsh)` to `.zshrc` after `compinit`. An interactive `devenv shell` installs bash completion. The integration tests in `tests/` start the real router on an ephemeral port with fake sources. Design notes live in [ARCHITECTURE.md](ARCHITECTURE.md); work tracking in [TODO.md](TODO.md); extended docs in [docs/](docs/README.md).
+270 -165
View File
@@ -14,8 +14,16 @@ const command_table = [
{ name: "self-check", usage: "check that the table, commands and help agree", flags: [] }
]
# Baked into the generated shell function so completion works from any directory.
# Baked into dry-run spawns. Docs and help use the repo-relative path.
const SCRIPT_PATH = path self
const SCRIPT_REL = "dev/dev-test.nu"
# File stem. Nushell binds `main` to this name on `use`, so every subcommand
# is `dev-test <sub>` or tab completion of `dev-test` does not list them.
const RUNNER = "dev-test"
# Same name in both generated scripts, so the self-check can call it.
const COMP_FN = "_dev_test"
# Completers cannot update the caller's environment, so the filter list is a file.
const FILTER_CACHE = "target/skyg-test-filters.cache"
@@ -32,101 +40,102 @@ const FILTER_CACHE = "target/skyg-test-filters.cache"
# completions print a zsh or bash completion script
# self-check fail when the command table, the commands and the help text diverge
#
# Loaded as a module (`use dev/test.nu *`) this file provides `test` plus one
# command per subcommand, so Nushell completes names and flags natively.
# Loaded as a module (`use dev/dev-test.nu *`) the file stem is `dev-test`, so
# the subcommands have to be named `dev-test <sub>` or Nushell will not
# complete them.
#
# The checks are read-only against the host: no /etc, systemd or Docker changes.
#
# Script mode (`nu dev/test.nu`) is `--wrapped` so flags reach the parser below.
# Script mode (`nu dev/dev-test.nu`) is `--wrapped` so flags reach the parser below.
# Nushell would otherwise accept `--nocapture` on every subcommand and reject
# the rest before `main` can explain them.
export def --wrapped main [...args: string] {
let parsed = (parse-script-args $args)
match $parsed.cmd {
"help" => { print-help }
"unit" => { test unit }
"unit" => { dev-test unit }
"integration" => {
if ($parsed.positional | is-empty) {
test integration --nocapture=$parsed.nocapture
dev-test integration --nocapture=$parsed.nocapture
} else {
test integration $parsed.positional --nocapture=$parsed.nocapture
dev-test integration $parsed.positional --nocapture=$parsed.nocapture
}
}
"lint" => { test lint }
"lint" => { dev-test lint }
"vm" => {
if ($parsed.positional | is-empty) {
test vm
dev-test vm
} else {
test vm $parsed.positional
dev-test vm $parsed.positional
}
}
"all" => { test all }
"ci" => { test ci }
"all" => { dev-test all }
"ci" => { dev-test ci }
"completions" => {
if ($parsed.positional | is-empty) {
test completions
dev-test completions
} else {
test completions $parsed.positional
dev-test completions $parsed.positional
}
}
"self-check" => { test self-check }
"self-check" => { dev-test self-check }
_ => { error make { msg: $"subcommand '($parsed.cmd)' has no dispatch arm" } }
}
}
# Run the unit tests: no sockets, no host access.
export def "test unit" [] {
export def "dev-test unit" [] {
run-steps (unit-steps)
}
# Run the integration tests (plain, tls, integration, binary_smoke, peers).
export def "test integration" [
filter?: string@"nu-complete test filters" # cargo name filter, e.g. a test name or file
--nocapture # do not capture test output
export def "dev-test integration" [
filter?: string@"nu-complete dev-test filters" # cargo name filter, e.g. a test name or file
--nocapture # do not capture test output
] {
run-steps (integration-steps $filter $nocapture)
}
# Formatting, lints and the dependency check.
export def "test lint" [] {
export def "dev-test lint" [] {
run-steps (lint-steps)
}
# Run a NixOS VM test. Slow: it builds and boots VMs.
export def "test vm" [
name?: string@"nu-complete test vm" # vm (checks) or peers (tests.<system>.peers)
export def "dev-test vm" [
name?: string@"nu-complete dev-test vm" # vm (checks) or peers (tests.<system>.peers)
] {
let name = ($name | default "vm")
if $name not-in (nu-complete test vm) {
error make { msg: $"unknown VM test '($name)'; expected one of: (nu-complete test vm | str join ', ')" }
if $name not-in (nu-complete dev-test vm) {
error make { msg: $"unknown VM test '($name)'; expected one of: (nu-complete dev-test vm | str join ', ')" }
}
run-steps (vm-steps $name)
}
# lint, unit and integration; stops at the first failure.
export def "test all" [] {
export def "dev-test all" [] {
run-steps ((lint-steps) | append (unit-steps) | append (integration-steps null false))
}
# `test all` plus the VM test.
export def "test ci" [] {
# `dev-test all` plus the VM test.
export def "dev-test ci" [] {
run-steps ((lint-steps) | append (unit-steps) | append (integration-steps null false) | append (vm-steps "vm"))
}
# Print a completion script for a shell that cannot load this module (zsh, bash).
export def "test completions" [
shell?: string@"nu-complete test shells" # zsh or bash
export def "dev-test completions" [
shell?: string@"nu-complete dev-test shells" # zsh or bash
] {
let shell = ($shell | default "zsh")
match $shell {
"zsh" => { print (completion-script zsh) }
"bash" => { print (completion-script bash) }
_ => { error make { msg: $"unknown shell '($shell)'; expected one of: (nu-complete test shells | str join ', ')" } }
_ => { error make { msg: $"unknown shell '($shell)'; expected one of: (nu-complete dev-test shells | str join ', ')" } }
}
}
# Fail if the command table, the defined commands or the help text diverge.
export def "test self-check" [] {
export def "dev-test self-check" [] {
let problems = (
(check-consistency)
++ (check-integration-args)
@@ -134,27 +143,28 @@ export def "test self-check" [] {
++ (check-filter-cache)
++ (check-openssl-verdict)
++ (check-kvm-step)
++ (check-shell-wrapper)
++ (check-shell-completion)
++ (check-nu-completions)
++ (check-docs)
)
if not ($problems | is-empty) {
print -e "test self-check failed:"
print -e "dev-test self-check failed:"
$problems | each {|p| print -e $" - ($p)" } | ignore
exit 1
}
print "test self-check: ok"
print "dev-test self-check: ok"
}
def "nu-complete test vm" [] {
def "nu-complete dev-test vm" [] {
["vm" "peers"]
}
def "nu-complete test shells" [] {
def "nu-complete dev-test shells" [] {
["zsh" "bash"]
}
# `cargo test --test '*'` matches integration targets only, same as the integration step.
def "nu-complete test filters" [] {
def "nu-complete dev-test filters" [] {
let key = (try { source-mtime-key } catch { "unknown" })
try {
cached-lines (filter-cache-path) $key {|| list-test-names }
@@ -260,7 +270,7 @@ def kvm-refusal [] {
}
def vm-steps [name: string] {
# The KVM check runs in the step body. Raising it here would abort `test ci`
# The KVM check runs in the step body. Raising it here would abort `dev-test ci`
# before lint, unit and integration, and the summary table would not print.
let attr = (match $name {
"peers" => "tests"
@@ -389,19 +399,19 @@ def run-checked [cmd: string, args: list<string>] {
def help-footer [] {
[
"In an interactive bash or zsh, `test` is the shell builtin, so `test unit` exits 0 without running the suite."
"Run `devenv shell -- test <subcommand>` or `nu dev/test.nu <subcommand>`."
"An interactive `devenv shell` installs a bash function for known subcommands; `test -f` stays the builtin."
"zsh: after compinit, add `source <(nu dev/test.nu completions zsh)` to .zshrc. direnv cannot carry that function."
"Nushell: `use dev/test.nu *`."
$"Run `devenv shell -- ($RUNNER) <subcommand>` or `nu ($SCRIPT_REL) <subcommand>`."
"An interactive `devenv shell` installs bash completion for this command."
# Single quotes: `(...)` inside a double-quoted string runs the command.
('zsh: after compinit, add `source <(nu ' + $SCRIPT_REL + ' completions zsh)` to .zshrc. direnv will not source it.')
$"Nushell: `use ($SCRIPT_REL) *`."
] | str join "\n"
}
def print-help [] {
print "Run the repository's checks."
print ""
print "Usage: devenv shell -- test <subcommand> [args]"
print " nu dev/test.nu <subcommand> [args]"
print $"Usage: devenv shell -- ($RUNNER) <subcommand> [args]"
print $" nu ($SCRIPT_REL) <subcommand> [args]"
print ""
$command_table | each {|c| {subcommand: $c.name, flags: ($c.flags | str join " "), what: $c.usage} } | table -e | print
print ""
@@ -410,15 +420,14 @@ def print-help [] {
def completion-script [shell: string] {
let subcommands = ($command_table | get name | str join " ")
let pattern = (($command_table | get name) | append ["--help" "-h"] | str join "|")
let flags = ($command_table | get flags | flatten | uniq | str join " ")
let replacements = {
"__SUBCOMMANDS__": $subcommands
"__PATTERN__": $pattern
"__FLAGS__": $flags
"__VM__": (nu-complete test vm | str join " ")
"__SHELLS__": (nu-complete test shells | str join " ")
"__SCRIPT__": $SCRIPT_PATH
"__VM__": (nu-complete dev-test vm | str join " ")
"__SHELLS__": (nu-complete dev-test shells | str join " ")
"__FN__": $COMP_FN
"__RUNNER__": $RUNNER
}
let template = (match $shell {
"zsh" => (zsh-template)
@@ -428,20 +437,21 @@ def completion-script [shell: string] {
}
def zsh-template [] {
# `_describe` emits nothing unless a completion widget is already active, so
# the candidate list was empty. `compadd` writes the words itself.
# A raw string cannot start with `#` in Nushell, so the template starts on the next line.
r#'
#compdef test
# zsh completion for the repository test runner.
# `test` is a shell builtin: `test unit` exits 0 and runs nothing. This function
# sends known subcommands to the runner and leaves every other call to `builtin test`.
# Install after compinit (this hook cannot see zsh from devenv's bash, and direnv
# will not carry the function):
# source <(nu dev/test.nu completions zsh)
_test_runner() {
#compdef __RUNNER__
# zsh completion for the dev-test runner. The command is an external program,
# so this file only registers completion; it does not define a dev-test function.
# Install after compinit (devenv's hook runs in bash, and direnv will not source
# it for you):
# source <(nu dev/dev-test.nu completions zsh)
__FN__() {
local -a subcommands
subcommands=(__SUBCOMMANDS__)
if (( CURRENT == 2 )); then
_describe 'subcommand' subcommands
compadd -- "${subcommands[@]}"
return
fi
case "${words[2]}" in
@@ -450,33 +460,18 @@ _test_runner() {
completions) compadd -- __SHELLS__ ;;
esac
}
test() {
if [[ $# -eq 0 ]]; then
command nu "__SCRIPT__"
return
fi
case "$1" in
__PATTERN__)
command nu "__SCRIPT__" "$@"
;;
*)
builtin test "$@"
;;
esac
}
if (( $+functions[compdef] )); then
compdef _test_runner test
compdef __FN__ __RUNNER__
fi
'# | str trim --left
}
def bash-template [] {
r#'
# bash completion for the repository test runner.
# `test` is a shell builtin: `test unit` exits 0 and runs nothing. This function
# sends known subcommands to the runner and leaves every other call to `builtin test`.
# source <(nu dev/test.nu completions bash)
_test_runner() {
# bash completion for the dev-test runner. The command is an external program,
# so this file only registers completion; it does not define a dev-test function.
# source <(nu dev/dev-test.nu completions bash)
__FN__() {
local cur sub i
cur="${COMP_WORDS[COMP_CWORD]}"
sub=""
@@ -498,33 +493,19 @@ _test_runner() {
esac
return 0
}
test() {
if [ $# -eq 0 ]; then
command nu "__SCRIPT__"
return
fi
case "$1" in
__PATTERN__)
command nu "__SCRIPT__" "$@"
;;
*)
builtin test "$@"
;;
esac
}
complete -F _test_runner test
complete -F __FN__ __RUNNER__
'# | str trim --left
}
# Script-mode argv is one list. Swallowing a flag or a spare word made
# `test unit --nocapture` and `test integration a b` look successful.
# `dev-test unit --nocapture` and `dev-test integration a b` look successful.
def parse-script-args [args: list<string>] {
if ($args | is-empty) or ($args | any {|a| ($a == "--help") or ($a == "-h") }) {
return {cmd: "help", positional: null, nocapture: false}
}
let sub = ($args | first)
if $sub not-in ($command_table | get name) {
error make { msg: $"unknown subcommand '($sub)'; run 'test --help' for the list" }
error make { msg: $"unknown subcommand '($sub)'; run '($RUNNER) --help' for the list" }
}
let rest = ($args | skip 1)
let allowed = ($command_table | where name == $sub | get 0.flags)
@@ -533,7 +514,7 @@ def parse-script-args [args: list<string>] {
for a in $rest {
if ($a | str starts-with "-") {
if $a not-in $allowed {
error make { msg: $"unknown flag '($a)' for 'test ($sub)'" }
error make { msg: $"unknown flag '($a)' for '($RUNNER) ($sub)'" }
}
if $a == "--nocapture" {
$nocapture = true
@@ -545,7 +526,7 @@ def parse-script-args [args: list<string>] {
let max_pos = (if $sub in ["integration" "vm" "completions"] { 1 } else { 0 })
if ($positionals | length) > $max_pos {
let what = (if $max_pos == 0 { "no arguments" } else { "at most one argument" })
error make { msg: $"test ($sub) takes ($what); got '($positionals | str join ' ')'" }
error make { msg: $"($RUNNER) ($sub) takes ($what); got '($positionals | str join ' ')'" }
}
{cmd: $sub, positional: ($positionals | get 0?), nocapture: $nocapture}
}
@@ -553,51 +534,49 @@ def parse-script-args [args: list<string>] {
# Compare the table with the commands Nushell actually parsed from this file and
# with the help block above; every entry must agree in all three places.
def check-consistency [] {
let defined = (scope commands | where {|c| $c.name == "test" or ($c.name | str starts-with "test ")} | get name
| where {|n| $n != "test" } | each {|n| $n | str substring 5.. })
let prefix = $"($RUNNER) "
let defined = (
scope commands
| where {|c| $c.name == $RUNNER or ($c.name | str starts-with $prefix)}
| get name
| where {|n| $n != $RUNNER}
| each {|n| $n | str substring ($prefix | str length).. }
)
let table = ($command_table | get name)
# The entry point is `test` once the file is used as a module and `main` when
# it runs as a script; both modes must agree with the table.
let entry = (scope commands | where {|c| $c.name in ["test" "main"]} | get 0?)
# The entry point is `dev-test` once the file is used as a module and `main`
# when it runs as a script; both modes must agree with the table.
let entry = (scope commands | where {|c| $c.name in [$RUNNER "main"]} | get 0?)
let usage = (if $entry == null { "" } else { $entry.extra_description | default "" })
mut problems = []
for name in $table {
if $name not-in $defined {
$problems = ($problems | append $"table lists '($name)' but there is no 'test ($name)' command")
$problems = ($problems | append $"table lists '($name)' but there is no '($RUNNER) ($name)' command")
}
if not ($usage =~ ('(?m)^\s+' + $name + '\s')) {
$problems = ($problems | append $"help text does not list '($name)'")
}
let cmd = (scope commands | where name == $"test ($name)" | get 0?)
let cmd = (scope commands | where name == $"($RUNNER) ($name)" | get 0?)
let flags = (if $cmd == null { [] } else { $cmd.signatures | get any | where {|p| $p.parameter_type in ["switch" "named"]} | each {|p| $"--($p.parameter_name)" } | sort })
let expected = (($command_table | where name == $name | get 0.flags) | sort)
if $flags != $expected {
$problems = ($problems | append $"flags of 'test ($name)' are ($flags | to json -r), table says ($expected | to json -r)")
$problems = ($problems | append $"flags of '($RUNNER) ($name)' are ($flags | to json -r), table says ($expected | to json -r)")
}
}
for name in $defined {
if $name not-in $table {
$problems = ($problems | append $"command 'test ($name)' is not in the command table")
}
}
# The generated shells must at least parse; only checked where the shell exists.
for shell in (nu-complete test shells) {
if not ((which $shell | is-empty)) {
let file = ($nu.temp-dir | path join $"test-completions-($nu.pid).($shell)")
completion-script $shell | save -f $file
let syntax = (run-external $shell "-n" $file | complete)
rm -f $file
if $syntax.exit_code != 0 {
$problems = ($problems | append $"($shell) completion script does not parse: ($syntax.stderr | str trim)")
}
$problems = ($problems | append $"command '($RUNNER) ($name)' is not in the command table")
}
}
let footer = (help-footer)
if ($footer | str contains "source <(test completions") {
$problems = ($problems | append "help tells the shell to complete the builtin via 'test completions'")
if ($footer | str contains "shell builtin") or ($footer | str contains "builtin test") {
$problems = ($problems | append "help still warns about the test builtin")
}
if not ($footer | str contains "nu dev/test.nu completions") {
$problems = ($problems | append "help does not show 'nu dev/test.nu completions'")
if not ($footer | str contains $"nu ($SCRIPT_REL) completions") {
$problems = ($problems | append $"help does not show 'nu ($SCRIPT_REL) completions'")
}
# `(...)` in a double-quoted string runs the command; that would inline the script.
if ($footer | str contains "#compdef") or ($footer | str contains "complete -F") {
$problems = ($problems | append "help footer embeds a completion script; quote the <(...) example literally")
}
$problems
}
@@ -666,11 +645,11 @@ def check-dispatch [] {
}
let cli = (run-external "nu" $SCRIPT_PATH "unit" "--nocapture" | complete)
if $cli.exit_code == 0 or not ($cli.stderr | str contains "unknown flag") {
$problems = ($problems | append $"CLI 'test unit --nocapture' exit ($cli.exit_code): ($cli.stderr | str trim) ($cli.stdout | str trim)")
$problems = ($problems | append $"CLI '($RUNNER) unit --nocapture' exit ($cli.exit_code): ($cli.stderr | str trim) ($cli.stdout | str trim)")
}
let extra = (run-external "nu" $SCRIPT_PATH "integration" "a" "b" | complete)
if $extra.exit_code == 0 or not ($extra.stderr | str contains "at most one") {
$problems = ($problems | append $"CLI 'test integration a b' exit ($extra.exit_code): ($extra.stderr | str trim)")
$problems = ($problems | append $"CLI '($RUNNER) integration a b' exit ($extra.exit_code): ($extra.stderr | str trim)")
}
$problems
}
@@ -687,8 +666,8 @@ def check-filter-cache [] {
let cache = ($dir | path join "cache")
"0" | save -f $counter
let ran = (with-env { SKYG_TEST_LIST_STUB: $counter, SKYG_TEST_FILTER_CACHE: $cache } {
let first = (nu-complete test filters)
let second = (nu-complete test filters)
let first = (nu-complete dev-test filters)
let second = (nu-complete dev-test filters)
let n = (open --raw $counter | str trim | into int)
let key = (source-mtime-key)
let stored = (if ($cache | path exists) { open --raw $cache | lines | first } else { "" })
@@ -697,7 +676,7 @@ def check-filter-cache [] {
let names = (open --raw $cache | lines | skip 1)
(["stale"] | append $names | str join (char nl)) | save -f $cache
}
let third = (nu-complete test filters)
let third = (nu-complete dev-test filters)
let n2 = (open --raw $counter | str trim | into int)
{first: $first, second: $second, third: $third, n: $n, n2: $n2, key: $key, stored: $stored}
})
@@ -756,78 +735,204 @@ def check-kvm-step [] {
[]
}
def check-shell-wrapper [] {
# Words after a `TAG:` line, dropping the `--` that ends compadd's options.
def tagged-words [text: string, tag: string] {
let line = ($text | lines | where {|l| $l | str starts-with $"($tag):"} | get 0?)
if $line == null {
return []
}
$line
| str substring ($"($tag):" | str length)..
| split row " "
| where {|w| not ($w | is-empty) and $w != "--" }
}
def check-shell-completion [] {
mut problems = []
let dir = ($nu.temp-dir | path join $"runner-wrap-($nu.pid)")
let dir = ($nu.temp-dir | path join $"runner-complete-($nu.pid)")
rm -rf $dir
mkdir $dir
let fake = ($dir | path join "nu")
let nl = (char nl)
$"#!/bin/sh($nl)printf '%s\\n' \"\$*\"($nl)" | save -f $fake
run-external "chmod" "+x" $fake | complete | ignore
for shell in (nu-complete test shells) {
let subcommands = ($command_table | get name)
for shell in (nu-complete dev-test shells) {
let script = (completion-script $shell)
let file = ($dir | path join $"completions.($shell)")
$script | save -f $file
if ($script | str contains "builtin") {
$problems = ($problems | append $"($shell) completion still mentions the test builtin")
}
let wrappers = ($script | lines | where {|l| $l =~ '^(test|dev-test)\(\)' })
if not ($wrappers | is-empty) {
$problems = ($problems | append $"($shell) completion defines a command function")
}
if not ($script | str contains $COMP_FN) or not ($script | str contains $RUNNER) {
$problems = ($problems | append $"($shell) completion does not register ($COMP_FN) for ($RUNNER)")
}
if (which $shell | is-empty) {
continue
}
let script = ($dir | path join $"completions.($shell)")
completion-script $shell | save -f $script
let probe = ($dir | path join $"probe.($shell)")
[
$"source '($script)'"
"test unit"
$"test -f '($script)'"
"if [ $? -eq 0 ]; then echo POSIX:0; else echo POSIX:1; fi"
] | str join (char nl) | save -f $probe
let out = (with-env { PATH: ([$dir] | append $env.PATH) } {
run-external $shell $probe | complete
})
if $out.exit_code != 0 {
$problems = ($problems | append $"($shell) wrapper exited ($out.exit_code): ($out.stderr | str trim)")
let syntax_args = (if $shell == "zsh" { ["-f" "-n" $file] } else { ["--noprofile" "--norc" "-n" $file] })
let syntax = (run-external $shell ...$syntax_args | complete)
if $syntax.exit_code != 0 {
$problems = ($problems | append $"($shell) completion script does not parse: ($syntax.stderr | str trim)")
continue
}
if not ($out.stdout | str contains $"($SCRIPT_PATH) unit") {
$problems = ($problems | append $"($shell) did not dispatch 'test unit': ($out.stdout)")
let probe = ($dir | path join $"probe.($shell)")
let body = (if $shell == "zsh" {
[
$"source '($file)'"
"typeset -a got"
'compadd() { got+=("$*"); }'
("words=(" + $RUNNER + " '')")
"CURRENT=2"
$COMP_FN
'print -r -- "SUB:${got[*]}"'
"got=()"
("words=(" + $RUNNER + " vm '')")
"CURRENT=3"
$COMP_FN
'print -r -- "VM:${got[*]}"'
"got=()"
("words=(" + $RUNNER + " integration '')")
"CURRENT=3"
$"($COMP_FN)"
'print -r -- "FLAG:${got[*]}"'
"funcs="
'(( $+functions[dev-test] )) && funcs="${funcs}dev-test"'
'(( $+functions[test] )) && funcs="${funcs} test"'
'print -r -- "FUNCS:${funcs}"'
]
} else {
[
$"source '($file)'"
("COMP_WORDS=(" + $RUNNER + " '')")
"COMP_CWORD=1"
$COMP_FN
'printf "SUB:%s\n" "${COMPREPLY[*]}"'
("COMP_WORDS=(" + $RUNNER + " vm '')")
"COMP_CWORD=2"
$COMP_FN
'printf "VM:%s\n" "${COMPREPLY[*]}"'
("COMP_WORDS=(" + $RUNNER + " integration '')")
"COMP_CWORD=2"
$COMP_FN
'printf "FLAG:%s\n" "${COMPREPLY[*]}"'
"funcs="
'declare -F dev-test >/dev/null && funcs="${funcs}dev-test"'
'declare -F test >/dev/null && funcs="${funcs} test"'
'printf "FUNCS:%s\n" "$funcs"'
]
})
$body | str join (char nl) | save -f $probe
let args = (if $shell == "zsh" { ["-f" $probe] } else { ["--noprofile" "--norc" $probe] })
let out = (run-external $shell ...$args | complete)
if $out.exit_code != 0 {
$problems = ($problems | append $"($shell) completion probe exited ($out.exit_code): ($out.stderr | str trim) ($out.stdout | str trim)")
continue
}
if ($out.stdout | str contains $"($SCRIPT_PATH) -f") {
$problems = ($problems | append $"($shell) sent POSIX test to the runner: ($out.stdout)")
let sub_words = (tagged-words $out.stdout "SUB")
if ($sub_words | is-empty) {
$problems = ($problems | append $"($shell) completion output is empty: ($out.stdout | str trim)")
}
if not ($out.stdout | str contains "POSIX:0") {
$problems = ($problems | append $"($shell) POSIX test -f failed: ($out.stdout) ($out.stderr)")
for name in $subcommands {
if $name not-in $sub_words {
$problems = ($problems | append $"($shell) completion missing '($name)': ($sub_words | str join ' ')")
}
}
let vm_words = (tagged-words $out.stdout "VM")
if ("vm" not-in $vm_words) or ("peers" not-in $vm_words) {
$problems = ($problems | append $"($shell) vm completion is ($vm_words | str join ' ')")
}
let flag_words = (tagged-words $out.stdout "FLAG")
if "--nocapture" not-in $flag_words {
$problems = ($problems | append $"($shell) integration completion is ($flag_words | str join ' ')")
}
let funcs = (tagged-words $out.stdout "FUNCS")
if not ($funcs | is-empty) {
$problems = ($problems | append $"($shell) defined a command function: ($funcs | str join ' ')")
}
}
rm -rf $dir
$problems
}
# Cursor sits just after `marker` in the raw file, before any trailing newline.
def complete-at [body: string, marker: string] {
let file = ($nu.temp-dir | path join $"dev-test-ide-($nu.pid)-($marker | str replace ' ' '_').nu")
$body | save -f $file
let raw = (open --raw $file)
let idx = ($raw | str index-of $marker)
let pos = (if $idx == null or $idx < 0 { -1 } else { $idx + ($marker | str length) })
let got = (if $pos < 0 {
{stdout: "", stderr: "marker not found", exit_code: 1}
} else {
run-external "nu" "--no-config-file" "--ide-complete" ($pos | into string) $file | complete
})
rm -f $file
let comps = (if $got.exit_code != 0 {
[]
} else {
try { $got.stdout | from json | get completions } catch { [] }
})
{pos: $pos, comps: $comps, stdout: $got.stdout, stderr: $got.stderr, exit_code: $got.exit_code}
}
def check-nu-completions [] {
mut problems = []
let sub = (complete-at $"use ($SCRIPT_PATH) *\n($RUNNER) " $"($RUNNER) ")
let want = ($command_table | get name | each {|n| $"($RUNNER) ($n)" })
let missing = ($want | where {|n| $n not-in $sub.comps })
if ($sub.comps | is-empty) or ("alias" in $sub.comps) or (not ($missing | is-empty)) or (($sub.comps | length) != ($want | length)) {
$problems = ($problems | append $"nushell completion output is empty or wrong at ($sub.pos): ($sub.comps | to json -r) ($sub.stderr | str trim)")
}
let vm = (complete-at $"use ($SCRIPT_PATH) *\n($RUNNER) vm " $"($RUNNER) vm ")
if ($vm.comps | is-empty) or ("vm" not-in $vm.comps) or ("peers" not-in $vm.comps) or ("alias" in $vm.comps) {
$problems = ($problems | append $"nushell vm completion is empty or wrong: ($vm.comps | to json -r)")
}
$problems
}
def check-docs [] {
mut problems = []
let rel = $"nu ($SCRIPT_REL) completions"
for f in ["AGENTS.md" "README.md" "docs/consolidator.md" "devenv.nix"] {
let text = (open --raw $f)
if ($text | str contains "source <(test completions") {
$problems = ($problems | append $"($f) completes the shell builtin via 'test completions'")
if ($text | str contains "dev/test.nu") {
$problems = ($problems | append $"($f) still names dev/test.nu")
}
if ($text | str contains "shell builtin") or ($text | str contains "builtin test") {
$problems = ($problems | append $"($f) still documents the test builtin workaround")
}
if ($text | str contains "devenv shell -- test") {
$problems = ($problems | append $"($f) still invokes 'devenv shell -- test'")
}
}
for f in ["AGENTS.md" "README.md" "docs/consolidator.md"] {
let text = (open --raw $f)
if not ($text | str contains "nu dev/test.nu completions") {
$problems = ($problems | append $"($f) does not show 'nu dev/test.nu completions'")
if not ($text | str contains $rel) {
$problems = ($problems | append $"($f) does not show '($rel)'")
}
if not ($text | str contains "builtin") {
$problems = ($problems | append $"($f) does not mention the test builtin")
if not ($text | str contains $"devenv shell -- ($RUNNER)") {
$problems = ($problems | append $"($f) does not show 'devenv shell -- ($RUNNER)'")
}
}
let hook = (open --raw devenv.nix)
if not ($hook | str contains ">&2") {
$problems = ($problems | append "enterShell banner is not sent to stderr")
if not ($hook | str contains 'scripts."dev-test"') {
$problems = ($problems | append "devenv.nix does not define scripts.\"dev-test\"")
}
if not ($hook | str contains "completions bash") {
if ($hook | str contains "scripts.test") {
$problems = ($problems | append "devenv.nix still defines scripts.test")
}
if not ($hook | str contains $"($SCRIPT_REL) completions bash") {
$problems = ($problems | append "enterShell does not install bash completion")
}
if ($hook | str contains "completions zsh") {
$problems = ($problems | append "enterShell tries to install zsh completion from bash")
}
if not (($hook | str contains "case $-") and ($hook | str contains "*i*")) {
$problems = ($problems | append "enterShell banner is not limited to interactive shells")
$problems = ($problems | append "enterShell completion is not limited to interactive shells")
}
if ($hook | str contains 'echo "') {
$problems = ($problems | append "enterShell prints; non-interactive stdout must stay the command's own output")
}
$problems
}
+9 -9
View File
@@ -9,7 +9,7 @@
pkgs.systemd
# Local OTLP collector for the telemetry workstream (G7).
pkgs.opentelemetry-collector-contrib
# Test runner (dev/test.nu).
# Test runner (dev/dev-test.nu).
pkgs.nushell
];
@@ -22,22 +22,22 @@
scripts.run.exec = ''cargo run -- "$@"'';
scripts.smoke.exec = "cargo test --test binary_smoke -- --nocapture";
scripts.token.exec = ''cargo run -- token "$@"'';
# `devenv shell -- test unit|integration|lint|vm|all|ci`.
scripts.test.exec = ''nu dev/test.nu "$@"'';
# `devenv shell -- dev-test unit|integration|lint|vm|all|ci`.
# Quoted: a hyphen is not an identifier, so `scripts.dev-test` would subtract.
scripts."dev-test".exec = ''nu dev/dev-test.nu "$@"'';
enterShell = ''
export CLAUDE_CONFIG_DIR="$(pwd)/.claude-account"
# Interactive bash only. `devenv shell -- …` is non-interactive, so its
# stdout stays the command's own output, and the note goes to stderr.
# This hook cannot install zsh functions (direnv would not keep them
# either). zsh users source the completion script from .zshrc after
# compinit; see docs/consolidator.md.
# stdout stays the command's own output. This hook runs in bash, so it only
# installs bash completion, and direnv will not source it either. zsh users
# source the completion script from .zshrc after compinit; see
# docs/consolidator.md.
case $- in
*i*)
if [ -n "''${BASH_VERSION:-}" ] && command -v nu >/dev/null 2>&1; then
source <(nu dev/test.nu completions bash)
echo "test runner: 'test <subcommand>' in this shell ('test -f' stays the builtin). From outside: devenv shell -- test <subcommand>." >&2
source <(nu dev/dev-test.nu completions bash)
fi
;;
esac
+8 -8
View File
@@ -3,27 +3,27 @@
The peers acceptance test is a NixOS VM test. Three machines run the real `slinuxm`: `alpha` and `beta` consolidate each other over plain HTTP, and `alpha` also declares the TLS-only `gamma` with a private CA. It needs KVM and takes minutes.
```sh
devenv shell -- test vm peers # nix build .#tests.x86_64-linux.peers -L
devenv shell -- dev-test vm peers # nix build .#tests.x86_64-linux.peers -L
```
Against a tree without the consolidator the test stops at the first peers assertion; that is expected, the test is written ahead of the feature. `dev/test.nu` is the repository's test runner. `devenv shell -- test --help` lists its subcommands and the command behind each. Inside an interactive bash or zsh, `test` is the shell builtin, so `test vm` exits 0 without running anything. Use `devenv shell -- test …` or `nu dev/test.nu …`.
Against a tree without the consolidator the test stops at the first peers assertion; that is expected, the test is written ahead of the feature. `dev/dev-test.nu` is the repository's test runner. `devenv shell -- dev-test --help` lists its subcommands and the command behind each. In an interactive `devenv shell`, `dev-test …` is that same script.
In Nushell the runner is a module. Load it once, for example in `config.nu`, and `test unit`, `test vm peers`, ... complete natively. Integration-test name completion is cached in `target/skyg-test-filters.cache`, keyed on the newest mtime of the Rust sources and the Cargo manifests.
In Nushell the runner is a module. Load it once, for example in `config.nu`, and `dev-test unit`, `dev-test vm peers`, ... complete natively. Integration-test name completion is cached in `target/skyg-test-filters.cache`, keyed on the newest mtime of the Rust sources and the Cargo manifests.
```nu
use /home/you/src/skyg-linux-tracker/dev/test.nu * # absolute path to dev/test.nu
use /home/you/src/skyg-linux-tracker/dev/dev-test.nu * # absolute path to dev/dev-test.nu
```
zsh and bash completion comes from the same command table. The generated script defines a `test` function for known subcommands (`test unit`, `test vm peers`, ...) and leaves other invocations (`test -f`) to the builtin. An interactive `devenv shell` sources the bash script and prints a one-line note on stderr. That hook runs in bash, so it cannot install zsh functions, and direnv will not carry them either. zsh users add the following to `.zshrc` after `compinit`:
zsh and bash completion comes from the same command table. The generated script only registers completion for the external command `dev-test`. An interactive `devenv shell` sources the bash script. That hook runs in bash and direnv will not source it either, so zsh users add the following to `.zshrc` after `compinit`:
```sh
source <(nu /home/you/src/skyg-linux-tracker/dev/test.nu completions zsh)
source <(nu /home/you/src/skyg-linux-tracker/dev/dev-test.nu completions zsh)
```
bash, from the repo checkout:
```sh
source <(nu dev/test.nu completions bash)
source <(nu dev/dev-test.nu completions bash)
```
`nu dev/test.nu self-check` fails if the command table, the defined subcommands, the help text, the cargo argv, the filter cache or the shell wrapper drift apart.
`nu dev/dev-test.nu self-check` fails if the command table, the defined subcommands, the help text, the cargo argv, the filter cache or the shell completion drift apart.
+23 -16
View File
@@ -4,12 +4,12 @@ Part of [Phase 5: Consolidator](README.md). Contracts: [contracts.md](contracts.
| Field | Value |
|---|---|
| Status | T1 to T7 done; review round 1 addressed. `peers.nix` fails at the first peers assertion until WS-E lands (by design) |
| Status | T1 to T7 done; review round 1 addressed; runner renamed to `dev-test` (`dev/dev-test.nu`). `peers.nix` fails at the first peers assertion until WS-E lands (by design) |
| Owner | |
| Depends on | none (uses the module's freeform `settings` for `[peers]` and a systemd override for `LoadCredential`, so it needs no module changes). F1 later swaps this for the first-class module options |
| Blocks | nothing technically; it is the acceptance gate: goes green at the end of [WS-E](ws-e-telemetry.md) (with A, B, C, D landed), then F2 promotes it to `checks` |
| Can start when | day one |
| Owned files | `nix/tests/peers.nix` (done), `flake.nix` (the `tests.<system>.peers` output only), `dev/test.nu`, `devenv.nix` (nushell package, `test` script, completion hook), `AGENTS.md` and `README.md` (command lists only, see seams), `docs/consolidator.md` (creates the file with only a `## Testing` section, see seams) |
| Owned files | `nix/tests/peers.nix` (done), `flake.nix` (the `tests.<system>.peers` output only), `dev/dev-test.nu`, `devenv.nix` (nushell package, `dev-test` script, completion hook), `AGENTS.md` and `README.md` (command lists only, see seams), `docs/consolidator.md` (creates the file with only a `## Testing` section, see seams) |
| Do not touch | `src/**` (all streams), `nix/module.nix`, `nix/tests/vm.nix`, `systemd/skyg-linux-tracker.service` (F), `config.example.toml` (A/F), `tests/peers.rs` and `tests/common/mod.rs` (F; in-process variants by A/B live in their own test files, see [ws-a](ws-a-config.md), [ws-b](ws-b-registry.md)), `docs/api.md` (C) |
Shared-file seams:
@@ -17,7 +17,7 @@ Shared-file seams:
- `flake.nix`: T owns it until F2. F2 later moves exactly one attribute, `tests.${system}.peers`, into `checks` (rename `checks.vm-peers`). Nobody else edits `flake.nix`; it already carries the uncommitted peers output.
- `devenv.nix`: T only.
- `README.md`: T edits only the command/test list near the existing commands; F4 owns the new peers section. Different headings, so no overlap. `AGENTS.md`: T replaces the command list; F4 adds only the MCP tool-count note.
- `docs/consolidator.md`: T creates it with a single `## Testing` section (the one-line `use dev/test.nu *` and the completion setup). F4 adds every other section and must not edit `## Testing`.
- `docs/consolidator.md`: T creates it with a single `## Testing` section (the one-line `use dev/dev-test.nu *` and the completion setup). F4 adds every other section and must not edit `## Testing`.
## Goal
@@ -33,18 +33,18 @@ Note on the written test: `nix/tests/peers.nix` also contains a third node `gamm
- **T4** token rejected: revoke the token on `beta`; `alpha` reports server up, api up, `auth = rejected`, `peer.api.auth_ok = 0`, and proxy calls fail with 503 naming the peer, without becoming a 401 for the caller; restore with a new token file and no restart, status recovers. **Done** in the script except the `peer.api.auth_ok = 0` metric check (needs WS-E; add it to the script, or leave to F2 which may query the collector/metrics endpoint, note in Log).
- **T5** outage: stop `beta`'s service: `alpha` flips server and api after `fail_threshold`, `alpha`'s own endpoints stay healthy, and recovery is detected; stop `beta` entirely (VM down) and check that `alpha`'s requests are bounded by timeouts. **Done** in the script.
- **T6** no chaining and read-only: a request carrying `x-slinuxm-hop` is rejected; non-GET to a proxy path is 405; a TLS variant with a test CA via `ca_file`. **Done** (405 for POST/DELETE, hop header 4xx, `gamma` over HTTPS).
- **T7** **Done**: `dev/test.nu` with `unit`, `integration`, `lint`, `vm`, `all`, `ci`, `completions`, `self-check`; devenv `test` script plus zsh/bash completion hook; AGENTS.md and README.md command lists; `docs/consolidator.md` `## Testing`. A simple test runner CLI written in **Nushell**: `dev/test.nu` (devenv adds `pkgs.nushell`; also exposed as the devenv script `test`, so `devenv shell -- test unit` works from any shell). One entry point so contributors and agents do not memorise cargo/nix incantations. Structure it as a Nushell script with a `main` and `main <sub>` subcommands (`def "main unit"`, ...) so `--help` is generated and completion comes from the typed signatures:
- `test unit` runs `cargo test --lib --bins` (no sockets, no host access).
- `test integration [filter]` runs `cargo test --test '*'` (integration targets only: plain, tls, integration, binary_smoke, `peers` once it exists), so `test all` does not run the unit tests twice. `filter` is passed to cargo as a name filter. `--nocapture` is a libtest flag and is always placed after `--`.
- `test lint` runs `cargo fmt --check`, `cargo clippy --all-targets -- -D warnings` and the `cargo tree -i openssl` emptiness check.
- `test vm [name]` runs a NixOS VM test. `name` is one of `vm` (from `checks`) or `peers` (`nix build .#tests.x86_64-linux.peers -L`; moves to `checks` when green). A missing `/dev/kvm` fails that step and says why, after earlier `ci` steps, and the summary table still prints. The step prints that the build is slow.
- `test all` = lint, unit, integration, stopping at the first failure; `test ci` = `all` plus `vm`. No args prints help.
- **T7** **Done**: `dev/dev-test.nu` with `unit`, `integration`, `lint`, `vm`, `all`, `ci`, `completions`, `self-check`; devenv `dev-test` script plus zsh/bash completion hook; AGENTS.md and README.md command lists; `docs/consolidator.md` `## Testing`. A simple test runner CLI written in **Nushell**: `dev/dev-test.nu` (devenv adds `pkgs.nushell`; also exposed as the devenv script `dev-test`, so `devenv shell -- dev-test unit` works from any shell, including an interactive one). One entry point so contributors and agents do not memorise cargo/nix incantations. The command was renamed from `test` to `dev-test` (user decision, see Log) because `test` is a shell builtin. Structure it as a Nushell script with a `main` and subcommands literally named `dev-test <sub>` (`export def "dev-test unit"`, ...) so `--help` is generated and completion comes from the typed signatures. Nushell binds `main` to the file stem, so only commands named `dev-test ...` complete as `dev-test unit`:
- `dev-test unit` runs `cargo test --lib --bins` (no sockets, no host access).
- `dev-test integration [filter]` runs `cargo test --test '*'` (integration targets only: plain, tls, integration, binary_smoke, `peers` once it exists), so `dev-test all` does not run the unit tests twice. `filter` is passed to cargo as a name filter. `--nocapture` is a libtest flag and is always placed after `--`.
- `dev-test lint` runs `cargo fmt --check`, `cargo clippy --all-targets -- -D warnings` and the `cargo tree -i openssl` emptiness check.
- `dev-test vm [name]` runs a NixOS VM test. `name` is one of `vm` (from `checks`) or `peers` (`nix build .#tests.x86_64-linux.peers -L`; moves to `checks` when green). A missing `/dev/kvm` fails that step and says why, after earlier `ci` steps, and the summary table still prints. The step prints that the build is slow.
- `dev-test all` = lint, unit, integration, stopping at the first failure; `dev-test ci` = `all` plus `vm`. No args prints help.
- Each step prints a header and a final table (step, result, seconds); exit code non-zero on any failure (use `error make`/`exit 1`, never swallow cargo's exit status). Read-only against the host: no `/etc`, systemd or Docker changes (live-host checks stay manual, see Verification plan in the [README](README.md#verification-plan)).
- **Autocomplete (must work, and is a done-criterion):**
- Nushell: subcommands and flags complete natively once the script is loaded as a module. Dynamic values use custom completers (`def "nu-complete test vm" [] { ["vm" "peers"] }` and `name: string@"nu-complete test vm"`; integration filters complete from `cargo test --test '*' -- --list`, cached in `target/skyg-test-filters.cache` and keyed on the newest mtime of the Rust sources and Cargo manifests). A completer cannot update the caller's environment, so an env var does not survive Tab. `docs/consolidator.md` shows the one-line `use dev/test.nu *` for a permanent setup.
- Other shells (the repo owner's login shell is zsh): `nu dev/test.nu completions zsh|bash` prints a completion script generated from the same command table. `test` is a shell builtin, so in an interactive bash or zsh `test unit` exits 0 without running the suite. The script defines a `test` function that sends known subcommands to the runner and otherwise calls `builtin test`, which keeps `test -f` working. `devenv shell -- test …` does not need the function: it finds the devenv script on PATH. Interactive `devenv shell` sources the bash script and prints a one-line note on stderr. zsh is not started by that hook; after `compinit`, zsh users add `source <(nu dev/test.nu completions zsh)` to `.zshrc` themselves. direnv does not carry the function.
- Verified by a self-test: `test --help` and `test vm --help` list every subcommand/flag, and a check in `dev/test.nu` (`test self-check`) fails if the completion table and the defined subcommands diverge.
- AGENTS.md and README replace their scattered command lists with `test ...`, keeping the underlying commands visible in `--help`.
- Nushell: subcommands and flags complete natively once the script is loaded as a module. Dynamic values use custom completers (`def "nu-complete dev-test vm" [] { ["vm" "peers"] }` and `name: string@"nu-complete dev-test vm"`; integration filters complete from `cargo test --test '*' -- --list`, cached in `target/skyg-test-filters.cache` and keyed on the newest mtime of the Rust sources and Cargo manifests). A completer cannot update the caller's environment, so an env var does not survive Tab. `docs/consolidator.md` shows the one-line `use dev/dev-test.nu *` for a permanent setup.
- Other shells (the repo owner's login shell is zsh): `nu dev/dev-test.nu completions zsh|bash` prints a completion script generated from the same command table. `dev-test` is not a builtin, so the script only registers completion (`compadd` in zsh, `compgen` in bash) and does not define a function. Interactive `dev-test …` runs the devenv script on PATH. `self-check` invokes those completion functions and fails if the candidate list is empty. Interactive `devenv shell` sources the bash script and prints nothing, so `devenv shell -- dev-test completions zsh` stays pipe-clean. zsh is not started by that hook; after `compinit`, zsh users add `source <(nu dev/dev-test.nu completions zsh)` to `.zshrc` themselves. direnv does not source it for them.
- Verified by a self-test: `dev-test --help` and `dev-test vm --help` list every subcommand/flag, and a check in `dev/dev-test.nu` (`dev-test self-check`) fails if the completion table and the defined subcommands diverge.
- AGENTS.md and README replace their scattered command lists with `dev-test ...`, keeping the underlying commands visible in `--help`.
## Inputs
@@ -53,13 +53,13 @@ Note on the written test: `nix/tests/peers.nix` also contains a third node `gamm
## Done when
The test builds under `nix build .#tests.x86_64-linux.peers` (or equivalent) and evaluates, fails cleanly at the first peers assertion against the current binary, and is documented in `docs/consolidator.md`. `test unit|integration|lint|vm|all` each run and report correctly on the current tree, and tab completion works in Nushell, zsh and bash (T7). It must pass at the end of WS-E, then move to `checks.vm-peers` (F2 does the move).
The test builds under `nix build .#tests.x86_64-linux.peers` (or equivalent) and evaluates, fails cleanly at the first peers assertion against the current binary, and is documented in `docs/consolidator.md`. `dev-test unit|integration|lint|vm|all` each run and report correctly on the current tree, and tab completion works in Nushell, zsh and bash (T7). It must pass at the end of WS-E, then move to `checks.vm-peers` (F2 does the move).
## Handoff
- To everyone: the `test` runner and its subcommand names (referenced by AGENTS.md/README and by every stream's "Done when").
- To everyone: the `dev-test` runner and its subcommand names (referenced by AGENTS.md/README and by every stream's "Done when").
- To WS-F: a passing `peers.nix` (ready to promote) and any changes made to its assumptions, recorded in contracts.md section 10.
- To WS-A/B: the `test integration` step picks up any `tests/peers*.rs` automatically.
- To WS-A/B: the `dev-test integration` step picks up any `tests/peers*.rs` automatically.
## Log
@@ -78,3 +78,10 @@ The test builds under `nix build .#tests.x86_64-linux.peers` (or equivalent) and
9. Script mode is `--wrapped`, so flags reach our parser. Unknown flags and extra positionals are errors. `--nocapture` is valid only on `integration`. `self-check` covers the parser and the CLI forms `unit --nocapture` and `integration a b`.
10. `TODO.md:28` T7 remains unticked on purpose.
- Handoff 2026-10-19 (WS-T review round 1 -> all): same subcommand names. Invoke as `devenv shell -- test <subcommand>` or `nu dev/test.nu <subcommand>`. Interactive `devenv shell` installs the bash wrapper; zsh needs the `.zshrc` line after `compinit`. `nu dev/test.nu self-check` is the runner's test. Not run here (no nix builds in this worktree): `nu dev/test.nu vm peers`, which on a KVM host is `nix build .#tests.x86_64-linux.peers -L`. `nix/tests/peers.nix` is unchanged.
- 2026-10-19 (WS-T, user decision): Renamed the runner from `test` to `dev-test` and the module from `dev/test.nu` to `dev/dev-test.nu`. Supersedes the review-round-1 note that kept the name `test`. `test` is a shell builtin, so an interactive `test unit` never reached the runner and the completion script had to define a `test` function that fell through to `builtin test`. `dev-test` is an external command: `devenv shell -- dev-test unit|integration|lint|vm|all|ci` and an interactive `dev-test …` both run the script. Completion scripts only register completion (`compadd` / `compgen`); they do not define a function. `_describe` emitted no candidates outside a completion widget, so zsh now uses `compadd` and `self-check` fails if the bash or zsh candidate list is empty. Nushell commands are named `dev-test <sub>` because the file stem is `dev-test`. argv/`--nocapture` and the filter cache are unchanged. `nix/tests/peers.nix` is unchanged. Also fixed a latent help bug: the `<(...)` example sat in a double-quoted string, so Nushell ran the completions command and `--help` inlined the whole zsh script; the line is concatenated now and `self-check` rejects an embedded script. Earlier dated Log entries keep the old name because they record what happened then; the Status line, T7 text and Handoff use the current name.
Stale `dev/test.nu` names left for the merger (other streams' files, not edited here):
- `plan/phase-5-consolidator/README.md` workstream index
- `plan/phase-5-consolidator/ws-a-config.md` Do not touch
- `plan/phase-5-consolidator/ws-f-nixos-docs.md` Do not touch
- `TODO.md` T7 line (unticked on purpose; the merger ticks it)
- Handoff 2026-10-19 (WS-T rename -> all): invoke the runner as `devenv shell -- dev-test <subcommand>` or `nu dev/dev-test.nu <subcommand>`. Interactive bash completion is installed by `devenv shell`; zsh, after `compinit`: `source <(nu dev/dev-test.nu completions zsh)`. Nushell: `use dev/dev-test.nu *`. `nu dev/dev-test.nu self-check` is the runner's test. Verified here: `nu dev/dev-test.nu self-check` and `nu dev/dev-test.nu all` (fmt, clippy, openssl, unit, integration) green, plus `devenv shell -- dev-test --help`. Not run here (no nix builds in this worktree): `nu dev/dev-test.nu vm peers`, which on a KVM host is `nix build .#tests.x86_64-linux.peers -L`. `nix/tests/peers.nix` is unchanged.