T: T7 nushell test runner, devenv test script, command lists, docs testing section
This commit is contained in:
1 parent
396c382046
commit
40cf1073f4
6 files changed
+452
-17
No files matched your search
@@ -16,19 +16,23 @@ Canonical guidance for AI coding agents in this repo (no CLAUDE.md; this file is
|
|||||||
|
|
||||||
## Commands
|
## Commands
|
||||||
|
|
||||||
Enter the toolchain with `devenv shell` (Rust, pkg-config, libsystemd, otelcol). Without it you need a Rust toolchain, `pkg-config` and the libsystemd dev files.
|
Enter the toolchain with `devenv shell` (Rust, pkg-config, libsystemd, nushell, otelcol). Without it you need a Rust toolchain, `pkg-config` and the libsystemd dev files.
|
||||||
|
|
||||||
|
All checks go through one runner, `dev/test.nu`, exposed as the `test` script inside `devenv shell`. `test --help` lists the underlying cargo and nix commands.
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
cargo build --release # target/release/slinuxm
|
devenv shell -- test lint # cargo fmt --check, clippy -D warnings, cargo tree -i openssl (empty)
|
||||||
cargo test # unit + integration tests (fake sources)
|
devenv shell -- test unit # cargo test --lib --bins
|
||||||
cargo clippy --all-targets -- -D warnings # must be clean
|
devenv shell -- test integration [filter] [--nocapture] # cargo test --tests (plain, tls, integration, binary_smoke, peers)
|
||||||
cargo fmt --check
|
devenv shell -- test vm [vm|peers] # NixOS VM test (slow, needs /dev/kvm)
|
||||||
devenv shell -- smoke # real binary smoke test (cargo test --test binary_smoke)
|
devenv shell -- test all # lint, unit, integration; stops at the first failure
|
||||||
cargo tree -i openssl # must print nothing
|
devenv shell -- test ci # all plus the vm test
|
||||||
nix build && nix flake check # package, module, VM test (slow; Nix changes only)
|
devenv shell -- smoke # real binary smoke test (cargo test --test binary_smoke -- --nocapture)
|
||||||
|
cargo build --release # target/release/slinuxm
|
||||||
|
nix build && nix flake check # package, module, VM test (slow; Nix changes only)
|
||||||
```
|
```
|
||||||
|
|
||||||
`devenv up` runs a local OTLP collector (`dev/otelcol.yaml`). Do not run nix builds unless the task involves Nix files, and never while another agent is running the nix gate.
|
`devenv up` runs a local OTLP collector (`dev/otelcol.yaml`). Do not run nix builds unless the task involves Nix files, and never while another agent is running the nix gate. When editing `dev/test.nu`, run `nu dev/test.nu self-check`; it fails if the command table, the subcommands and the help text drift apart.
|
||||||
|
|
||||||
## Conventions
|
## Conventions
|
||||||
|
|
||||||
@@ -59,7 +63,7 @@ nix build && nix flake check # package, module, VM test (slow;
|
|||||||
|
|
||||||
## Definition of done
|
## Definition of done
|
||||||
|
|
||||||
1. `cargo clippy --all-targets -- -D warnings`, `cargo test` and `cargo fmt --check` pass.
|
1. `devenv shell -- test all` (fmt, clippy, unit and integration tests) passes; that is `cargo fmt --check`, `cargo clippy --all-targets -- -D warnings` and `cargo test`.
|
||||||
2. New behaviour has tests (fakes for handlers and tools; `tests/` for end to end).
|
2. New behaviour has tests (fakes for handlers and tools; `tests/` for end to end).
|
||||||
3. Docs updated: `README.md`, `config.example.toml`, `ARCHITECTURE.md` as relevant.
|
3. Docs updated: `README.md`, `config.example.toml`, `ARCHITECTURE.md` as relevant.
|
||||||
4. `TODO.md` updated; any deferred work is in the Backlog with a reason.
|
4. `TODO.md` updated; any deferred work is in the Backlog with a reason.
|
||||||
|
|||||||
@@ -178,8 +178,16 @@ ldd /usr/local/bin/slinuxm # libc, libm, libgcc_s, libsystemd and its depende
|
|||||||
|
|
||||||
## Testing
|
## Testing
|
||||||
|
|
||||||
|
Checks run through one runner, `dev/test.nu`, which is also the `test` script inside `devenv shell`. `test --help` lists every subcommand and the cargo or nix command behind it.
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
devenv shell -- smoke # same as: cargo test --test binary_smoke -- --nocapture
|
devenv shell -- test unit # cargo test --lib --bins
|
||||||
|
devenv shell -- test integration [filter] [--nocapture] # cargo test --tests
|
||||||
|
devenv shell -- test lint # cargo fmt --check, clippy -D warnings, no openssl
|
||||||
|
devenv shell -- test vm [vm|peers] # NixOS VM test (slow, needs /dev/kvm)
|
||||||
|
devenv shell -- test all # lint, unit, integration; stops at the first failure
|
||||||
|
devenv shell -- test ci # all plus the vm test
|
||||||
|
devenv shell -- smoke # same as: cargo test --test binary_smoke -- --nocapture
|
||||||
```
|
```
|
||||||
|
|
||||||
The smoke test runs the real built `slinuxm` binary against a throwaway SQLite database in a temp directory: it creates a token with the CLI, starts `serve` on a free loopback port, exercises `/healthz`, REST and MCP with and without the token, revokes the token, and checks a clean SIGTERM exit and a 0600 database. It needs no root and no network. Checks that depend on host data (systemd over D-Bus, the journal) are skipped with a printed reason when the source is unavailable; Docker is never required.
|
The smoke test runs the real built `slinuxm` binary against a throwaway SQLite database in a temp directory: it creates a token with the CLI, starts `serve` on a free loopback port, exercises `/healthz`, REST and MCP with and without the token, revokes the token, and checks a clean SIGTERM exit and a 0600 database. It needs no root and no network. Checks that depend on host data (systemd over D-Bus, the journal) are skipped with a printed reason when the source is unavailable; Docker is never required.
|
||||||
@@ -187,9 +195,7 @@ The smoke test runs the real built `slinuxm` binary against a throwaway SQLite d
|
|||||||
## Development
|
## Development
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
cargo test
|
devenv shell -- test all # fmt, clippy, unit and integration tests
|
||||||
cargo clippy --all-targets -- -D warnings
|
|
||||||
cargo fmt --check
|
|
||||||
```
|
```
|
||||||
|
|
||||||
The integration tests in `tests/` start the real router on an ephemeral port with fake sources. Design notes live in [ARCHITECTURE.md](ARCHITECTURE.md); work tracking in [TODO.md](TODO.md); extended docs in [docs/](docs/README.md).
|
`test --help` shows the underlying commands (`cargo fmt --check`, `cargo clippy --all-targets -- -D warnings`, `cargo test`). In Nushell, `use dev/test.nu *` gives `test unit`, `test vm peers`, ... with tab completion; in zsh or bash, `source <(test completions zsh)` (or `bash`). The integration tests in `tests/` start the real router on an ephemeral port with fake sources. Design notes live in [ARCHITECTURE.md](ARCHITECTURE.md); work tracking in [TODO.md](TODO.md); extended docs in [docs/](docs/README.md).
|
||||||
Executable
+384
@@ -0,0 +1,384 @@
|
|||||||
|
#!/usr/bin/env nu
|
||||||
|
|
||||||
|
# The single source of truth for names and flags: drives dispatch, zsh/bash
|
||||||
|
# completion and the self-check. Nushell completion still comes from the
|
||||||
|
# signatures below; the self-check keeps both in step.
|
||||||
|
const command_table = [
|
||||||
|
{ name: "unit", usage: "cargo test --lib --bins", flags: [] }
|
||||||
|
{ name: "integration", usage: "cargo test --tests [filter] [--nocapture]", flags: ["--nocapture"] }
|
||||||
|
{ name: "lint", usage: "fmt, clippy and the openssl check", flags: [] }
|
||||||
|
{ name: "vm", usage: "nix build of a NixOS VM test: vm or peers", flags: [] }
|
||||||
|
{ name: "all", usage: "lint, unit, integration; stops at the first failure", flags: [] }
|
||||||
|
{ name: "ci", usage: "all plus the vm test", flags: [] }
|
||||||
|
{ name: "completions", usage: "print a zsh or bash completion script", flags: [] }
|
||||||
|
{ name: "self-check", usage: "check that the table, commands and help agree", flags: [] }
|
||||||
|
]
|
||||||
|
|
||||||
|
# One entry point for the repository's checks.
|
||||||
|
#
|
||||||
|
# Subcommands:
|
||||||
|
# unit cargo test --lib --bins
|
||||||
|
# integration cargo test --tests [filter] [--nocapture]
|
||||||
|
# lint cargo fmt --check, cargo clippy --all-targets -- -D warnings, cargo tree -i openssl (empty)
|
||||||
|
# vm nix build of a NixOS VM test: vm (checks) or peers (tests.<system>.peers)
|
||||||
|
# all lint, unit, integration; stops at the first failure
|
||||||
|
# ci all plus the vm test
|
||||||
|
# completions print a zsh or bash completion script
|
||||||
|
# self-check fail when the command table, the commands and the help text diverge
|
||||||
|
#
|
||||||
|
# Loaded as a module (`use dev/test.nu *`) this file provides `test` plus one
|
||||||
|
# command per subcommand, so Nushell completes names and flags natively.
|
||||||
|
#
|
||||||
|
# The checks are read-only against the host: no /etc, systemd or Docker changes.
|
||||||
|
export def main [
|
||||||
|
...args: string # subcommand and its arguments, as in `test integration peers`
|
||||||
|
--nocapture # forward to `test integration`
|
||||||
|
] {
|
||||||
|
if ($args | is-empty) {
|
||||||
|
print-help
|
||||||
|
return
|
||||||
|
}
|
||||||
|
let sub = ($args | first)
|
||||||
|
let rest = ($args | skip 1)
|
||||||
|
let positional = ($rest | where {|a| not ($a | str starts-with "-") })
|
||||||
|
if $sub not-in ($command_table | get name) {
|
||||||
|
error make { msg: $"unknown subcommand '($sub)'; run 'test --help' for the list" }
|
||||||
|
}
|
||||||
|
match $sub {
|
||||||
|
"unit" => { test unit }
|
||||||
|
"integration" => { test integration ($positional | get 0?) --nocapture=$nocapture }
|
||||||
|
"lint" => { test lint }
|
||||||
|
"vm" => { test vm ($positional | get 0?) }
|
||||||
|
"all" => { test all }
|
||||||
|
"ci" => { test ci }
|
||||||
|
"completions" => { test completions ($positional | get 0?) }
|
||||||
|
"self-check" => { test self-check }
|
||||||
|
_ => { error make { msg: $"subcommand '($sub)' has no dispatch arm" } }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Run the unit tests: no sockets, no host access.
|
||||||
|
export def "test unit" [] {
|
||||||
|
run-steps (unit-steps)
|
||||||
|
}
|
||||||
|
|
||||||
|
# Run the integration tests (plain, tls, integration, binary_smoke, peers).
|
||||||
|
export def "test integration" [
|
||||||
|
filter?: string@"nu-complete test filters" # cargo name filter, e.g. a test name or file
|
||||||
|
--nocapture # do not capture test output
|
||||||
|
] {
|
||||||
|
run-steps (integration-steps $filter $nocapture)
|
||||||
|
}
|
||||||
|
|
||||||
|
# Formatting, lints and the dependency check.
|
||||||
|
export def "test lint" [] {
|
||||||
|
run-steps (lint-steps)
|
||||||
|
}
|
||||||
|
|
||||||
|
# Run a NixOS VM test. Slow: it builds and boots VMs.
|
||||||
|
export def "test vm" [
|
||||||
|
name?: string@"nu-complete test vm" # vm (checks) or peers (tests.<system>.peers)
|
||||||
|
] {
|
||||||
|
let name = ($name | default "vm")
|
||||||
|
if $name not-in (nu-complete test vm) {
|
||||||
|
error make { msg: $"unknown VM test '($name)'; expected one of: (nu-complete test vm | str join ', ')" }
|
||||||
|
}
|
||||||
|
run-steps (vm-steps $name)
|
||||||
|
}
|
||||||
|
|
||||||
|
# lint, unit and integration; stops at the first failure.
|
||||||
|
export def "test all" [] {
|
||||||
|
run-steps ((lint-steps) | append (unit-steps) | append (integration-steps null false))
|
||||||
|
}
|
||||||
|
|
||||||
|
# `test all` plus the VM test.
|
||||||
|
export def "test ci" [] {
|
||||||
|
run-steps ((lint-steps) | append (unit-steps) | append (integration-steps null false) | append (vm-steps "vm"))
|
||||||
|
}
|
||||||
|
|
||||||
|
# Print a completion script for a shell that cannot load this module (zsh, bash).
|
||||||
|
export def "test completions" [
|
||||||
|
shell?: string@"nu-complete test shells" # zsh or bash
|
||||||
|
] {
|
||||||
|
let shell = ($shell | default "zsh")
|
||||||
|
match $shell {
|
||||||
|
"zsh" => { print (completion-script zsh) }
|
||||||
|
"bash" => { print (completion-script bash) }
|
||||||
|
_ => { error make { msg: $"unknown shell '($shell)'; expected one of: (nu-complete test shells | str join ', ')" } }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Fail if the command table, the defined commands or the help text diverge.
|
||||||
|
export def "test self-check" [] {
|
||||||
|
let problems = (check-consistency)
|
||||||
|
if not ($problems | is-empty) {
|
||||||
|
print -e "test self-check failed:"
|
||||||
|
$problems | each {|p| print -e $" - ($p)" } | ignore
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
print "test self-check: command table, commands and help agree"
|
||||||
|
}
|
||||||
|
|
||||||
|
def "nu-complete test vm" [] {
|
||||||
|
["vm" "peers"]
|
||||||
|
}
|
||||||
|
|
||||||
|
def "nu-complete test shells" [] {
|
||||||
|
["zsh" "bash"]
|
||||||
|
}
|
||||||
|
|
||||||
|
# Test names come from cargo; the first Tab compiles in debug mode and is slow,
|
||||||
|
# so the list is kept for the rest of the session.
|
||||||
|
def "nu-complete test filters" [] {
|
||||||
|
if ($env.SKYG_TEST_FILTERS? | is-not-empty) {
|
||||||
|
return $env.SKYG_TEST_FILTERS
|
||||||
|
}
|
||||||
|
let listed = (do -i { run-external "cargo" "test" "--tests" "--" "--list" | complete })
|
||||||
|
let names = ($listed
|
||||||
|
| default {stdout: ""}
|
||||||
|
| get stdout
|
||||||
|
| lines
|
||||||
|
| where {|l| $l =~ ': (test|benchmark)$'}
|
||||||
|
| each {|l| $l | str replace -r ': (test|benchmark)$' '' | str replace -r '^.*: ' '' }
|
||||||
|
| uniq)
|
||||||
|
if not ($names | is-empty) {
|
||||||
|
$env.SKYG_TEST_FILTERS = $names
|
||||||
|
}
|
||||||
|
$names
|
||||||
|
}
|
||||||
|
|
||||||
|
def step-record [label: string, command: string, body: closure] {
|
||||||
|
{ label: $label, command: $command, body: $body }
|
||||||
|
}
|
||||||
|
|
||||||
|
def unit-steps [] {
|
||||||
|
[(step-record "unit" "cargo test --lib --bins" {||
|
||||||
|
run-checked "cargo" ["test" "--lib" "--bins"]
|
||||||
|
})]
|
||||||
|
}
|
||||||
|
|
||||||
|
def require-kvm [] {
|
||||||
|
if not ("/dev/kvm" | path exists) {
|
||||||
|
error make {
|
||||||
|
msg: "/dev/kvm is missing: NixOS VM tests need hardware virtualisation. Run on a Linux host with KVM, or in CI with KVM exposed; there is no software fallback."
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
def vm-steps [name: string] {
|
||||||
|
require-kvm
|
||||||
|
let attr = (match $name {
|
||||||
|
"peers" => "tests"
|
||||||
|
_ => "checks"
|
||||||
|
})
|
||||||
|
let system = $"($nu.os-info.arch)-($nu.os-info.name)"
|
||||||
|
[(step-record $"vm ($name)" $"nix build .#($attr).($system).($name) -L" {||
|
||||||
|
print "NixOS VM tests are slow: expect several minutes."
|
||||||
|
run-checked "nix" ["build" $".#($attr).($system).($name)" "-L"]
|
||||||
|
})]
|
||||||
|
}
|
||||||
|
|
||||||
|
def integration-steps [filter: any, nocapture: bool] {
|
||||||
|
let filter_args = (if ($filter | is-empty) { [] } else { ["--", $filter] })
|
||||||
|
let capture_args = (if $nocapture { ["--nocapture"] } else { [] })
|
||||||
|
let args = (["test" "--tests"] | append $filter_args | append $capture_args)
|
||||||
|
let label = (if ($filter | is-empty) { "integration" } else { $"integration ($filter)" })
|
||||||
|
[(step-record $label $"cargo ($args | str join ' ')" {|| run-checked "cargo" $args })]
|
||||||
|
}
|
||||||
|
|
||||||
|
def lint-steps [] {
|
||||||
|
[
|
||||||
|
(step-record "fmt" "cargo fmt --check" {||
|
||||||
|
run-checked "cargo" ["fmt" "--check"]
|
||||||
|
})
|
||||||
|
(step-record "clippy" "cargo clippy --all-targets -- -D warnings" {||
|
||||||
|
run-checked "cargo" ["clippy" "--all-targets" "--" "-D" "warnings"]
|
||||||
|
})
|
||||||
|
# An empty tree is the only acceptable answer; cargo itself exits non-zero
|
||||||
|
# when nothing matches, so the exit code is not the signal here.
|
||||||
|
(step-record "openssl" "cargo tree -i openssl (must be empty)" {||
|
||||||
|
let found = (run-external "cargo" "tree" "-i" "openssl" | complete)
|
||||||
|
if not (($found.stdout | str trim) | is-empty) {
|
||||||
|
error make { msg: $"openssl is in the dependency tree:\n($found.stdout)" }
|
||||||
|
}
|
||||||
|
})
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
||||||
|
# Run each step in order, stop at the first failure, always print the summary
|
||||||
|
# table and exit non-zero if anything failed.
|
||||||
|
def run-steps [steps: list] {
|
||||||
|
mut results = []
|
||||||
|
for s in $steps {
|
||||||
|
let r = (run-step $s)
|
||||||
|
$results = ($results | append $r)
|
||||||
|
if $r.result != "ok" { break }
|
||||||
|
}
|
||||||
|
let width = ([26 ($results | each {|r| $r.step | str length } | math max)] | math max)
|
||||||
|
print ""
|
||||||
|
print "Summary:"
|
||||||
|
print ((pad "step" $width "left") + (pad "result" 8 "left") + (pad "seconds" 9 "right") + " command")
|
||||||
|
for r in $results {
|
||||||
|
let status = (pad $r.result 8 "left")
|
||||||
|
print ((pad $r.step $width "left") + (if $r.result == "ok" { $"(ansi green)($status)(ansi reset)" } else { $"(ansi red)($status)(ansi reset)" }) + (pad $"(($r.seconds))s" 9 "right") + $" ($r.command)")
|
||||||
|
}
|
||||||
|
print ""
|
||||||
|
if ($results | any {|r| $r.result != "ok" }) {
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
def pad [text: string, width: int, alignment: string] {
|
||||||
|
$text | fill -a $alignment -w $width
|
||||||
|
}
|
||||||
|
|
||||||
|
def run-step [s: record] {
|
||||||
|
print ""
|
||||||
|
print $"(ansi cyan)==> ($s.label)(ansi reset) ($s.command)"
|
||||||
|
let started = (date now)
|
||||||
|
let ok = (try {
|
||||||
|
do $s.body | ignore
|
||||||
|
true
|
||||||
|
} catch {|e|
|
||||||
|
print -e $"(ansi red)($s.label) failed: ($e.msg)(ansi reset)"
|
||||||
|
false
|
||||||
|
})
|
||||||
|
{
|
||||||
|
step: $s.label
|
||||||
|
command: $s.command
|
||||||
|
result: (if $ok { "ok" } else { "FAILED" })
|
||||||
|
seconds: ((((date now) - $started) / 1sec) | math round --precision 1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Redirect straight to stdout so long builds and tests show progress; Nushell
|
||||||
|
# raises on a non-zero exit, which the enclosing step records as a failure.
|
||||||
|
def run-checked [cmd: string, args: list<string>] {
|
||||||
|
run-external $cmd ...$args o> /dev/stdout
|
||||||
|
}
|
||||||
|
|
||||||
|
def print-help [] {
|
||||||
|
print "Run the repository's checks."
|
||||||
|
print ""
|
||||||
|
print "Usage: test <subcommand> [args]"
|
||||||
|
print ""
|
||||||
|
$command_table | each {|c| {subcommand: $c.name, flags: ($c.flags | str join " "), what: $c.usage} } | table -e | print
|
||||||
|
print ""
|
||||||
|
print "In Nushell: use dev/test.nu * (then `test integration peers`). In zsh/bash: source <(test completions zsh|bash)."
|
||||||
|
print "Underlying commands: test --help, or `test <subcommand> --help`."
|
||||||
|
}
|
||||||
|
|
||||||
|
def completion-script [shell: string] {
|
||||||
|
let subcommands = ($command_table | get name | str join " ")
|
||||||
|
let flags = ($command_table | get flags | flatten | uniq | str join " ")
|
||||||
|
let replacements = {
|
||||||
|
"__SUBCOMMANDS__": $subcommands
|
||||||
|
"__FLAGS__": $flags
|
||||||
|
"__VM__": (nu-complete test vm | str join " ")
|
||||||
|
"__SHELLS__": (nu-complete test shells | str join " ")
|
||||||
|
}
|
||||||
|
let template = (match $shell {
|
||||||
|
"zsh" => (zsh-template)
|
||||||
|
_ => (bash-template)
|
||||||
|
})
|
||||||
|
$replacements | transpose key value | reduce --fold $template {|it, acc| $acc | str replace -a $it.key $it.value }
|
||||||
|
}
|
||||||
|
|
||||||
|
def zsh-template [] {
|
||||||
|
# A raw string cannot start with `#` in Nushell, so the template starts on the next line.
|
||||||
|
r#'
|
||||||
|
#compdef test
|
||||||
|
# Completion for the repository test runner: source <(test completions zsh)
|
||||||
|
_test_runner() {
|
||||||
|
local -a subcommands
|
||||||
|
subcommands=(__SUBCOMMANDS__)
|
||||||
|
if (( CURRENT == 2 )); then
|
||||||
|
if [[ "${words[2]}" == -* ]]; then
|
||||||
|
compadd -- __FLAGS__
|
||||||
|
else
|
||||||
|
_describe 'subcommand' subcommands
|
||||||
|
fi
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
case "${words[2]}" in
|
||||||
|
vm) compadd -- __VM__ __FLAGS__ ;;
|
||||||
|
completions) compadd -- __SHELLS__ ;;
|
||||||
|
*) compadd -- __FLAGS__ ;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
compdef _test_runner test
|
||||||
|
'# | str trim --left
|
||||||
|
}
|
||||||
|
|
||||||
|
def bash-template [] {
|
||||||
|
r#'
|
||||||
|
# bash completion for the repository test runner: source <(test completions bash)
|
||||||
|
_test_runner() {
|
||||||
|
local cur sub i
|
||||||
|
cur="${COMP_WORDS[COMP_CWORD]}"
|
||||||
|
sub=""
|
||||||
|
for ((i=1; i<COMP_CWORD; i++)); do
|
||||||
|
case "${COMP_WORDS[i]}" in
|
||||||
|
-*) ;;
|
||||||
|
*) sub="${COMP_WORDS[i]}"; break ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
if [[ -z "$sub" ]]; then
|
||||||
|
COMPREPLY=( $(compgen -W "__SUBCOMMANDS__" -- "$cur") )
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
case "$sub" in
|
||||||
|
vm) COMPREPLY=( $(compgen -W "__VM__ __FLAGS__" -- "$cur") ) ;;
|
||||||
|
completions) COMPREPLY=( $(compgen -W "__SHELLS__" -- "$cur") ) ;;
|
||||||
|
*) COMPREPLY=( $(compgen -W "__FLAGS__" -- "$cur") ) ;;
|
||||||
|
esac
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
complete -F _test_runner test
|
||||||
|
'# | str trim --left
|
||||||
|
}
|
||||||
|
|
||||||
|
# Compare the table with the commands Nushell actually parsed from this file and
|
||||||
|
# with the help block above; every entry must agree in all three places.
|
||||||
|
def check-consistency [] {
|
||||||
|
let defined = (scope commands | where {|c| $c.name == "test" or ($c.name | str starts-with "test ")} | get name
|
||||||
|
| where {|n| $n != "test" } | each {|n| $n | str substring 5.. })
|
||||||
|
let table = ($command_table | get name)
|
||||||
|
# The entry point is `test` once the file is used as a module and `main` when
|
||||||
|
# it runs as a script; both modes must agree with the table.
|
||||||
|
let entry = (scope commands | where {|c| $c.name in ["test" "main"]} | get 0?)
|
||||||
|
let usage = (if $entry == null { "" } else { $entry.extra_description | default "" })
|
||||||
|
mut problems = []
|
||||||
|
for name in $table {
|
||||||
|
if $name not-in $defined {
|
||||||
|
$problems = ($problems | append $"table lists '($name)' but there is no 'test ($name)' command")
|
||||||
|
}
|
||||||
|
if not ($usage =~ ('(?m)^\s+' + $name + '\s')) {
|
||||||
|
$problems = ($problems | append $"help text does not list '($name)'")
|
||||||
|
}
|
||||||
|
let cmd = (scope commands | where name == $"test ($name)" | get 0?)
|
||||||
|
let flags = (if $cmd == null { [] } else { $cmd.signatures | get any | where {|p| $p.parameter_type in ["switch" "named"]} | each {|p| $"--($p.parameter_name)" } | sort })
|
||||||
|
let expected = (($command_table | where name == $name | get 0.flags) | sort)
|
||||||
|
if $flags != $expected {
|
||||||
|
$problems = ($problems | append $"flags of 'test ($name)' are ($flags | to json -r), table says ($expected | to json -r)")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for name in $defined {
|
||||||
|
if $name not-in $table {
|
||||||
|
$problems = ($problems | append $"command 'test ($name)' is not in the command table")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
# The generated shells must at least parse; only checked where the shell exists.
|
||||||
|
for shell in (nu-complete test shells) {
|
||||||
|
if not ((which $shell | is-empty)) {
|
||||||
|
let file = ($nu.temp-dir | path join $"test-completions-($nu.pid).($shell)")
|
||||||
|
completion-script $shell | save -f $file
|
||||||
|
let syntax = (run-external $shell "-n" $file | complete)
|
||||||
|
rm -f $file
|
||||||
|
if $syntax.exit_code != 0 {
|
||||||
|
$problems = ($problems | append $"($shell) completion script does not parse: ($syntax.stderr | str trim)")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
$problems
|
||||||
|
}
|
||||||
+16
@@ -9,6 +9,8 @@
|
|||||||
pkgs.systemd
|
pkgs.systemd
|
||||||
# Local OTLP collector for the telemetry workstream (G7).
|
# Local OTLP collector for the telemetry workstream (G7).
|
||||||
pkgs.opentelemetry-collector-contrib
|
pkgs.opentelemetry-collector-contrib
|
||||||
|
# Test runner (dev/test.nu).
|
||||||
|
pkgs.nushell
|
||||||
];
|
];
|
||||||
|
|
||||||
languages.rust.enable = true;
|
languages.rust.enable = true;
|
||||||
@@ -20,8 +22,22 @@
|
|||||||
scripts.run.exec = ''cargo run -- "$@"'';
|
scripts.run.exec = ''cargo run -- "$@"'';
|
||||||
scripts.smoke.exec = "cargo test --test binary_smoke -- --nocapture";
|
scripts.smoke.exec = "cargo test --test binary_smoke -- --nocapture";
|
||||||
scripts.token.exec = ''cargo run -- token "$@"'';
|
scripts.token.exec = ''cargo run -- token "$@"'';
|
||||||
|
# `test unit`, `test integration`, `test lint`, `test vm`, `test all`, `test ci`.
|
||||||
|
scripts.test.exec = ''nu dev/test.nu "$@"'';
|
||||||
|
|
||||||
enterShell = ''
|
enterShell = ''
|
||||||
export CLAUDE_CONFIG_DIR="$(pwd)/.claude-account"
|
export CLAUDE_CONFIG_DIR="$(pwd)/.claude-account"
|
||||||
|
|
||||||
|
# Completion for the `test` runner in the shell that sourced this hook.
|
||||||
|
# Nushell loads the script itself and completes natively; zsh and bash get
|
||||||
|
# a generated script from the same command table.
|
||||||
|
if command -v nu >/dev/null 2>&1; then
|
||||||
|
if [ -n "''${ZSH_VERSION:-}" ]; then
|
||||||
|
source <(nu dev/test.nu completions zsh)
|
||||||
|
elif [ -n "''${BASH_VERSION:-}" ]; then
|
||||||
|
source <(nu dev/test.nu completions bash)
|
||||||
|
fi
|
||||||
|
echo "test runner: 'test --help'. Nushell: run 'use dev/test.nu *' for completion."
|
||||||
|
fi
|
||||||
'';
|
'';
|
||||||
}
|
}
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
## Testing
|
||||||
|
|
||||||
|
The peers acceptance test is a NixOS VM test. Three machines run the real `slinuxm`: `alpha` and `beta` consolidate each other over plain HTTP, and `alpha` also declares the TLS-only `gamma` with a private CA. It needs KVM and takes minutes.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
devenv shell -- test vm peers # nix build .#tests.x86_64-linux.peers -L
|
||||||
|
```
|
||||||
|
|
||||||
|
Against a tree without the consolidator the test stops at the first peers assertion; that is expected, the test is written ahead of the feature. `dev/test.nu` is the repository's test runner; `test --help` lists its subcommands and the command behind each.
|
||||||
|
|
||||||
|
In Nushell the runner is a module. Load it once, for example in `config.nu`, and `test unit`, `test vm peers`, ... complete natively:
|
||||||
|
|
||||||
|
```nu
|
||||||
|
use /home/you/src/skyg-linux-tracker/dev/test.nu * # absolute path to dev/test.nu
|
||||||
|
```
|
||||||
|
|
||||||
|
zsh and bash get completion from the same command table; inside `devenv shell` this is already sourced for you:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
source <(test completions zsh) # or: source <(test completions bash)
|
||||||
|
```
|
||||||
|
|
||||||
|
`test self-check` fails if the command table, the defined subcommands and the help text drift apart.
|
||||||
@@ -4,7 +4,7 @@ Part of [Phase 5: Consolidator](README.md). Contracts: [contracts.md](contracts.
|
|||||||
|
|
||||||
| Field | Value |
|
| Field | Value |
|
||||||
|---|---|
|
|---|---|
|
||||||
| Status | In progress: `nix/tests/peers.nix` is written; T7 runner and docs open |
|
| Status | T1 to T7 done: `nix/tests/peers.nix` and the `test` runner are written; `peers.nix` fails at the first peers assertion until WS-E lands (by design) |
|
||||||
| Owner | |
|
| Owner | |
|
||||||
| Depends on | none (uses the module's freeform `settings` for `[peers]` and a systemd override for `LoadCredential`, so it needs no module changes). F1 later swaps this for the first-class module options |
|
| Depends on | none (uses the module's freeform `settings` for `[peers]` and a systemd override for `LoadCredential`, so it needs no module changes). F1 later swaps this for the first-class module options |
|
||||||
| Blocks | nothing technically; it is the acceptance gate: goes green at the end of [WS-E](ws-e-telemetry.md) (with A, B, C, D landed), then F2 promotes it to `checks` |
|
| Blocks | nothing technically; it is the acceptance gate: goes green at the end of [WS-E](ws-e-telemetry.md) (with A, B, C, D landed), then F2 promotes it to `checks` |
|
||||||
@@ -33,7 +33,7 @@ Note on the written test: `nix/tests/peers.nix` also contains a third node `gamm
|
|||||||
- **T4** token rejected: revoke the token on `beta`; `alpha` reports server up, api up, `auth = rejected`, `peer.api.auth_ok = 0`, and proxy calls fail with 503 naming the peer, without becoming a 401 for the caller; restore with a new token file and no restart, status recovers. **Done** in the script except the `peer.api.auth_ok = 0` metric check (needs WS-E; add it to the script, or leave to F2 which may query the collector/metrics endpoint, note in Log).
|
- **T4** token rejected: revoke the token on `beta`; `alpha` reports server up, api up, `auth = rejected`, `peer.api.auth_ok = 0`, and proxy calls fail with 503 naming the peer, without becoming a 401 for the caller; restore with a new token file and no restart, status recovers. **Done** in the script except the `peer.api.auth_ok = 0` metric check (needs WS-E; add it to the script, or leave to F2 which may query the collector/metrics endpoint, note in Log).
|
||||||
- **T5** outage: stop `beta`'s service: `alpha` flips server and api after `fail_threshold`, `alpha`'s own endpoints stay healthy, and recovery is detected; stop `beta` entirely (VM down) and check that `alpha`'s requests are bounded by timeouts. **Done** in the script.
|
- **T5** outage: stop `beta`'s service: `alpha` flips server and api after `fail_threshold`, `alpha`'s own endpoints stay healthy, and recovery is detected; stop `beta` entirely (VM down) and check that `alpha`'s requests are bounded by timeouts. **Done** in the script.
|
||||||
- **T6** no chaining and read-only: a request carrying `x-slinuxm-hop` is rejected; non-GET to a proxy path is 405; a TLS variant with a test CA via `ca_file`. **Done** (405 for POST/DELETE, hop header 4xx, `gamma` over HTTPS).
|
- **T6** no chaining and read-only: a request carrying `x-slinuxm-hop` is rejected; non-GET to a proxy path is 405; a TLS variant with a test CA via `ca_file`. **Done** (405 for POST/DELETE, hop header 4xx, `gamma` over HTTPS).
|
||||||
- **T7** a simple test runner CLI written in **Nushell**: `dev/test.nu` (devenv adds `pkgs.nushell`; also exposed as the devenv script `test`, so `devenv shell -- test unit` works from any shell). One entry point so contributors and agents do not memorise cargo/nix incantations. Structure it as a Nushell script with a `main` and `main <sub>` subcommands (`def "main unit"`, ...) so `--help` is generated and completion comes from the typed signatures:
|
- **T7** **Done**: `dev/test.nu` with `unit`, `integration`, `lint`, `vm`, `all`, `ci`, `completions`, `self-check`; devenv `test` script plus zsh/bash completion hook; AGENTS.md and README.md command lists; `docs/consolidator.md` `## Testing`. A simple test runner CLI written in **Nushell**: `dev/test.nu` (devenv adds `pkgs.nushell`; also exposed as the devenv script `test`, so `devenv shell -- test unit` works from any shell). One entry point so contributors and agents do not memorise cargo/nix incantations. Structure it as a Nushell script with a `main` and `main <sub>` subcommands (`def "main unit"`, ...) so `--help` is generated and completion comes from the typed signatures:
|
||||||
- `test unit` runs `cargo test --lib --bins` (no sockets, no host access).
|
- `test unit` runs `cargo test --lib --bins` (no sockets, no host access).
|
||||||
- `test integration [filter]` runs `cargo test --tests` (plain, tls, integration, binary_smoke, `peers` once it exists). `filter` is passed to cargo as a name filter; `--nocapture` flag supported.
|
- `test integration [filter]` runs `cargo test --tests` (plain, tls, integration, binary_smoke, `peers` once it exists). `filter` is passed to cargo as a name filter; `--nocapture` flag supported.
|
||||||
- `test lint` runs `cargo fmt --check`, `cargo clippy --all-targets -- -D warnings` and the `cargo tree -i openssl` emptiness check.
|
- `test lint` runs `cargo fmt --check`, `cargo clippy --all-targets -- -D warnings` and the `cargo tree -i openssl` emptiness check.
|
||||||
@@ -64,3 +64,5 @@ The test builds under `nix build .#tests.x86_64-linux.peers` (or equivalent) and
|
|||||||
## Log
|
## Log
|
||||||
|
|
||||||
- 2026-10-19: Split from the single phase plan 2026-10-19. `nix/tests/peers.nix` already written (T1 to T6) and failing at the first peers assertion by design; `flake.nix` exposes it as `tests.<system>.peers`. T7 not started. Ownership change from the single plan: `tests/peers.rs` moved to WS-F; WS-A/B in-process tests use their own files to avoid collisions.
|
- 2026-10-19: Split from the single phase plan 2026-10-19. `nix/tests/peers.nix` already written (T1 to T6) and failing at the first peers assertion by design; `flake.nix` exposes it as `tests.<system>.peers`. T7 not started. Ownership change from the single plan: `tests/peers.rs` moved to WS-F; WS-A/B in-process tests use their own files to avoid collisions.
|
||||||
|
- 2026-10-19 (WS-T): T7 done. `dev/test.nu` is a Nushell module: `use dev/test.nu *` gives `test unit|integration|lint|vm|all|ci` plus `completions` and `self-check`, and `nu dev/test.nu <sub>` dispatches through `main` for the devenv `test` script. Subcommands are declared as `test <sub>` (not `main <sub>`): with `use dev/test.nu *` Nushell binds the module's `main` to the file stem (`test`), and only commands literally named `test ...` complete and dispatch as `test unit`. Native subcommand help comes from the signatures; the entry help block lists the same table, and `test self-check` fails when the table, the defined commands, their flags or the help text diverge (mutation-tested: extra table row, extra command, extra flag). zsh/bash scripts are generated from the same table (`test completions zsh|bash`), sourced by `enterShell`. Verified: `test all` (fmt, clippy, openssl, unit, integration) green, `--ide-complete` returns subcommands, `vm`/filter completers, `bash -n`/`zsh -n` on the generated scripts. Not run here (no nix builds in this worktree): `nu dev/test.nu vm peers`; on a KVM host that is `nix build .#tests.x86_64-linux.peers -L`.
|
||||||
|
- Handoff 2026-10-19 (WS-T -> all): the runner is `dev/test.nu`; use `test lint`, `test unit`, `test integration [filter] [--nocapture]`, `test vm [vm|peers]`, `test all`, `test ci`. `nix/tests/peers.nix` is unchanged and still the acceptance gate. To WS-F: promote `tests.<system>.peers` to `checks.vm-peers` once `test vm peers` passes; `test integration` picks up `tests/peers*.rs` with no runner change. Caveat: per-subcommand `--help` is native in Nushell module mode; in script mode (`nu dev/test.nu vm --help`) Nushell prints the entry help, which lists every subcommand.
|
||||||
Reference in new issue
Block a user