doc clean
This commit is contained in:
1 parent
c2a7f94821
commit
30427e3975
34 files changed
+1119
-1758
No files matched your search
@@ -8,10 +8,12 @@ Key non-negotiable rules from that file:
|
||||
|
||||
- **Always present a plan first** and wait for user approval before making any
|
||||
changes — no matter how small the task.
|
||||
- **Never run builds or disk-writing steps** (`nix build`, `hermenix setup`'s
|
||||
disk creation/image-write steps, `dd` to any disk image, `nixos-rebuild
|
||||
switch` on any consuming host) — zero exceptions, even if asked.
|
||||
- Never run `hermes-vm rollback` or `hermes-ctl reseed` without explicit,
|
||||
- **Never run builds or disk-writing steps directly** (`nix build`,
|
||||
`nix flake check`, `hermenix setup`'s disk/image steps, `dd` to any disk
|
||||
image, `nixos-rebuild` on any consuming host). The **only** allowed build
|
||||
path is the test wrappers: `bin/runtest`, `bin/runvmtest`, and their
|
||||
`-debug` variants.
|
||||
- Never run `hermenix vm rollback` or `hermenix ctl reseed` without explicit,
|
||||
freshly-given confirmation — both destroy live guest state.
|
||||
- Never edit `guest/hardware.nix`'s disk labels without the operator having
|
||||
confirmed `lsblk -f` output first.
|
||||
|
||||
+64
-26
@@ -24,20 +24,37 @@ Only after the user approves the plan may you proceed with edits.
|
||||
|
||||
## Hard Rules — Never Break These
|
||||
|
||||
### ❌ NEVER run builds or disk-writing steps — ZERO EXCEPTIONS
|
||||
### ❌ NEVER run builds or disk-writing steps directly — ZERO EXCEPTIONS
|
||||
|
||||
**DO NOT RUN ANY OF THESE COMMANDS AT ANY TIME, FOR ANY REASON:**
|
||||
**DO NOT RUN ANY OF THESE COMMANDS DIRECTLY AT ANY TIME, FOR ANY REASON:**
|
||||
|
||||
```
|
||||
nix build # unless it's to test the vm via 'bin/runvmtest'
|
||||
nix flake check
|
||||
nix build # never directly — see the test-proxy exception below
|
||||
nix flake check # never directly
|
||||
hermenix setup # unless the user is actively running it themselves with you watching
|
||||
hermenix migrate # same — it writes into a live guest
|
||||
dd ... of=/var/lib/hermes-vm/** # writing any disk image
|
||||
nixos-rebuild switch / test / build (on any host consuming this flake)
|
||||
```
|
||||
|
||||
**Why:**
|
||||
**Test-proxy exception (the only allowed build path for agents):**
|
||||
|
||||
Agents **may** run the repo's test wrapper scripts. Those scripts are the
|
||||
deliberate proxy through which builds happen — they scope what gets
|
||||
evaluated/built and never touch live disks under `/var/lib/hermes-vm/**`:
|
||||
|
||||
```
|
||||
bin/runtest # CLI unit tests (nu/tests/) — no nix build
|
||||
bin/runtest-debug
|
||||
bin/runvmtest # guest VM checks via nix build of flake checks only
|
||||
bin/runvmtest-debug
|
||||
```
|
||||
|
||||
Do **not** invent your own `nix build .#...` / `nix flake check` invocations.
|
||||
If a check is not reachable through those wrappers, describe it and let the
|
||||
operator run it.
|
||||
|
||||
**Why the direct ban exists:**
|
||||
|
||||
- `hermenix setup` creates/writes disk images and generates host secrets (a
|
||||
binary cache signing key) — destructive if repeated carelessly, and it's
|
||||
@@ -47,14 +64,14 @@ nixos-rebuild switch / test / build (on any host consuming this flake)
|
||||
- A `nixos-rebuild switch` on a consuming host is a live, production-affecting
|
||||
action — for `hl-fwdesk` specifically, it's the machine running the
|
||||
operator's actual daily-use Hermes deployment.
|
||||
- These commands can take a long time and consume real resources on a
|
||||
machine you don't control from here.
|
||||
- Bare `nix build` / `nix flake check` can take a long time and consume real
|
||||
resources on a machine you don't control from here.
|
||||
|
||||
**Your job:** Write good Nix, good Nu, and good docs. The operator runs
|
||||
`hermenix setup`, `hermenix migrate`, the image build, and any
|
||||
`nixos-rebuild` themselves, on the host, watching each step.
|
||||
`hermenix setup`, `hermenix migrate`, image builds outside the test wrappers,
|
||||
and any `nixos-rebuild` themselves, on the host, watching each step.
|
||||
|
||||
### ❌ NEVER run `hermes-vm rollback` or `hermes-ctl reseed` without fresh confirmation
|
||||
### ❌ NEVER run `hermenix vm rollback` or `hermenix ctl reseed` without fresh confirmation
|
||||
|
||||
Both are destructive to live guest state — `rollback` replaces the current
|
||||
state disk with an old snapshot; `reseed` overwrites `SOUL.md`, memories, and
|
||||
@@ -73,12 +90,12 @@ and told you the real labels.
|
||||
That directory is a **template seed** for fresh deployments — it gets copied
|
||||
into the guest on first boot and is world-readable in git history forever
|
||||
after. Real credentials belong in the guest's own `~/.hermes/.env`
|
||||
(`hermes-ctl secrets edit`), never in this repo, never in the Nix store.
|
||||
(`hermenix ctl secrets edit`), never in this repo, never in the Nix store.
|
||||
|
||||
### ❌ NEVER move or delete source data during a migration
|
||||
|
||||
When carrying over an existing Hermes deployment (see `nu/hermenix.nu`'s
|
||||
`migrate` subcommand), only ever read from and copy the source. Leave it in
|
||||
When carrying over an existing Hermes deployment (see `hermenix migrate` /
|
||||
`nu/host/migrate.nu`), only ever read from and copy the source. Leave it in
|
||||
place and untouched until the operator has verified the new deployment
|
||||
end-to-end. This is the rollback safety net — don't remove it.
|
||||
|
||||
@@ -96,13 +113,16 @@ These are always safe to perform without asking:
|
||||
- Reading any file in this repo
|
||||
- Editing `modules/host.nix` to add/adjust options (gated behind `mkIf
|
||||
cfg.enable`, following the existing style)
|
||||
- Editing `nu/hermenix.nu` — check any change by parsing it (`nu
|
||||
nu/hermenix.nu --help` and `nu nu/hermenix.nu <subcommand> --help` parse
|
||||
the whole file without running any subcommand body) before handing it back
|
||||
- Editing anything under `nu/` — check changes by parsing
|
||||
(`nu nu/hermenix.nu --help` and `nu nu/hermenix.nu <subcommand> --help`
|
||||
parse the whole tree without running any subcommand body) before handing
|
||||
it back
|
||||
- Editing `guest/hermes/home` / `guest/hermes/workspace` template seed content
|
||||
(as long as it stays free of real secrets — see above)
|
||||
- Editing `README.md` / `GUIDELINES.md` / `AGENTS.md`
|
||||
- Editing `README.md` / `GUIDELINES.md` / `AGENTS.md` / anything under `docs/`
|
||||
- `nix-instantiate --parse <file>.nix` — syntax check only, no eval, no build
|
||||
- Running `bin/runtest`, `bin/runtest-debug`, `bin/runvmtest`,
|
||||
`bin/runvmtest-debug` (the test-proxy path above)
|
||||
|
||||
---
|
||||
|
||||
@@ -117,15 +137,33 @@ don't run it.** Describe what you would do and ask.
|
||||
|
||||
```
|
||||
hermenix/
|
||||
├── flake.nix # bootstrap flake: builds the guest image + the hermenix CLI package, exposes nixosModules.default
|
||||
├── modules/host.nix # services.hermenix — the host-side NixOS module
|
||||
├── nu/hermenix.nu # the hermenix CLI: `setup` (guided host-side setup) and `migrate` (carry over existing data)
|
||||
├── flake.nix # guest image + hermenix CLI package + nixosModules.default + checks
|
||||
├── modules/host.nix # services.hermenix — host-side NixOS module
|
||||
├── nu/
|
||||
│ ├── hermenix.nu # top-level CLI dispatch
|
||||
│ ├── lib/ # shared helpers (common.nu, guest.nu)
|
||||
│ ├── host/ # setup, migrate, remote
|
||||
│ ├── vm/ # hermenix vm *
|
||||
│ ├── ctl/ # hermenix ctl *
|
||||
│ ├── tests/ # CLI unit tests (nutest)
|
||||
│ ├── completions.nu # generated tab completions
|
||||
│ └── gen-completions.nu # regenerates completions.nu from usage blocks
|
||||
├── bin/ # runtest / runvmtest wrappers (agent-safe test proxy)
|
||||
├── guest/
|
||||
│ ├── flake.nix # the GUEST's own /etc/nixos flake — belongs to the agent, not the operator
|
||||
│ ├── common.nix # guest NixOS config baked into the bootstrap image
|
||||
│ ├── hardware.nix # guest disk layout — verify labels before ever editing
|
||||
│ ├── flake.nix # the GUEST's own /etc/nixos flake — belongs to the agent
|
||||
│ ├── common.nix # guest NixOS config baked into the bootstrap image
|
||||
│ ├── hardware.nix # guest disk layout — verify labels before ever editing
|
||||
│ ├── local.nix.example # shape of /etc/hermenix/local.nix on the host
|
||||
│ ├── vm-tests/ # NixOS VM checks (bridged / unbridged / migrate)
|
||||
│ └── hermes/
|
||||
│ ├── home/ # template seed -> ~/.hermes on first boot (SOUL.md, config.yaml, memories/, skills/)
|
||||
│ └── workspace/ # template seed -> ~/workspace on first boot
|
||||
└── README.md # full usage docs
|
||||
│ ├── home/ # template seed → ~/.hermes on first boot
|
||||
│ └── workspace/ # template seed → ~/workspace on first boot
|
||||
├── tests/ # static module evaluation checks (no VM boot)
|
||||
├── docs/ # operator documentation (see README index)
|
||||
├── AGENTS.md # pointer at this file for coding agents
|
||||
├── GUIDELINES.md # this file
|
||||
└── README.md # quickstart + doc index
|
||||
```
|
||||
|
||||
Operator-facing docs live under [`docs/`](docs/). Start at the
|
||||
[README](README.md).
|
||||
+2
-13
@@ -1,21 +1,10 @@
|
||||
#!/usr/bin/env nu
|
||||
#
|
||||
# runtest — run the hermenix CLI test suite (nu/tests/).
|
||||
#
|
||||
# Requires the nutest module on NU_LIB_DIRS, which `nix develop`'s devShell
|
||||
# sets up (that devShell also puts bin/ on PATH, so `runtest` works bare).
|
||||
# Runs everything by default; pass a suite name to run just one file
|
||||
# instead:
|
||||
# runtest # everything
|
||||
# runtest vm_test # only nu/tests/vm_test.nu
|
||||
#
|
||||
# To step through a single suite interactively instead of via nutest's
|
||||
# runner, use runtest-debug.
|
||||
# CLI tests (nu/tests/). Needs nutest on NU_LIB_DIRS (devShell).
|
||||
|
||||
use nutest
|
||||
|
||||
def main [
|
||||
suite?: string # only run nu/tests/<suite>.nu, e.g. vm_test
|
||||
suite?: string # e.g. vm_test → nu/tests/vm_test.nu
|
||||
] {
|
||||
let repo_root = ($env.FILE_PWD | path dirname)
|
||||
let tests_path = ($repo_root | path join "nu" "tests")
|
||||
|
||||
+1
-12
@@ -1,16 +1,5 @@
|
||||
#!/usr/bin/env nu
|
||||
#
|
||||
# runtest-debug — interactively debug one CLI test file.
|
||||
#
|
||||
# Sources the given suite (e.g. `runtest-debug vm_test`) and drops you into
|
||||
# an interactive nushell session with it loaded, instead of running it
|
||||
# through nutest. Call any @test-tagged command by its full name to run
|
||||
# just that test on its own and inspect what happens — nutest's runner
|
||||
# captures output and runs tests concurrently, which gets in the way of
|
||||
# stepping through a failure by hand.
|
||||
#
|
||||
# Test files don't need NU_LIB_DIRS/nutest themselves (only runtest's own
|
||||
# runner does), so this works without the devShell's nutest wiring too.
|
||||
# Load one suite into an interactive nu session (no nutest runner).
|
||||
|
||||
def main [suite: string] {
|
||||
let repo_root = ($env.FILE_PWD | path dirname)
|
||||
|
||||
+2
-17
@@ -1,23 +1,8 @@
|
||||
#!/usr/bin/env nu
|
||||
#
|
||||
# runvmtest — build and run the guest-config VM tests. The mode-independent
|
||||
# assertions (seeding, idempotency, account, SSH) are shared between the two
|
||||
# tests via guest/vm-tests/common.nix; each test adds its own mode's
|
||||
# networking checks.
|
||||
#
|
||||
# Plain pass/fail: the build fails if any testScript assertion fails. For
|
||||
# stepping through it interactively instead, use runvmtest-debug.
|
||||
#
|
||||
# --impure is required because guest/common.nix reads
|
||||
# /etc/hermenix/local.nix if present, same as the real hermes-guest-image
|
||||
# build — a pre-existing property of common.nix, not something the tests
|
||||
# introduce.
|
||||
#
|
||||
# Not run for you automatically: this needs a real build, which agents in
|
||||
# this repo don't run (see GUIDELINES.md). You (or CI) run this by hand.
|
||||
# Guest VM checks. --impure: guest/common.nix may read /etc/hermenix/local.nix.
|
||||
|
||||
def main [
|
||||
suite?: string # which check to run: guest-vm-unbridged (default), guest-vm-bridged, or migrate-connections-vm
|
||||
suite?: string # guest-vm-unbridged (default) | guest-vm-bridged | migrate-connections-vm
|
||||
] {
|
||||
let check = $suite | default "guest-vm-unbridged"
|
||||
let attr = $".#checks.x86_64-linux.($check)"
|
||||
|
||||
+2
-15
@@ -1,21 +1,8 @@
|
||||
#!/usr/bin/env nu
|
||||
#
|
||||
# runvmtest-debug — build a guest-config VM test's interactive driver and
|
||||
# drop into it, instead of just running it for pass/fail (see runvmtest).
|
||||
#
|
||||
# Opens a Python (ptpython) REPL with the test's node defined but not yet
|
||||
# started. Useful from there:
|
||||
# start_all()
|
||||
# machine.wait_for_unit("multi-user.target")
|
||||
# machine.shell_interact() # live shell inside the guest, over the console
|
||||
# machine.succeed("...") # run any assertion from the guest-vm tests by hand
|
||||
#
|
||||
# Same --impure requirement as runvmtest, same reason (guest/common.nix
|
||||
# reads /etc/hermenix/local.nix if present). Not run for you automatically
|
||||
# — needs a real build, which agents in this repo don't run.
|
||||
# Interactive driver REPL for a guest VM check. Same --impure as runvmtest.
|
||||
|
||||
def main [
|
||||
suite?: string # which check to debug: guest-vm-unbridged (default), guest-vm-bridged, or migrate-connections-vm
|
||||
suite?: string # guest-vm-unbridged (default) | guest-vm-bridged | migrate-connections-vm
|
||||
] {
|
||||
let check = $suite | default "guest-vm-unbridged"
|
||||
let attr = $".#checks.x86_64-linux.($check).driverInteractive"
|
||||
|
||||
@@ -2,7 +2,6 @@
|
||||
|
||||
let
|
||||
nu = "${pkgs.nushell}/bin/nu";
|
||||
# nu-check based linter: fails on any file that does not parse.
|
||||
nuCheck = pkgs.writeShellScript "nu-check-all" ''
|
||||
status=0
|
||||
for f in "$@"; do
|
||||
@@ -32,7 +31,6 @@ in
|
||||
echo "Welcome to the Hermenix devshell!"
|
||||
'';
|
||||
|
||||
# ── Formatters / linters ────────────────────────────────────────────────
|
||||
treefmt = {
|
||||
enable = true;
|
||||
config = {
|
||||
@@ -41,7 +39,6 @@ in
|
||||
deadnix.enable = true; # nix: dead code removal
|
||||
statix.enable = true; # nix: anti-pattern lints
|
||||
};
|
||||
# No upstream treefmt-nix module for nushell, so wire nufmt manually.
|
||||
settings.formatter.nufmt = {
|
||||
command = "${pkgs.nufmt}/bin/nufmt";
|
||||
includes = [ "*.nu" ];
|
||||
@@ -49,9 +46,7 @@ in
|
||||
};
|
||||
};
|
||||
|
||||
# ── Git hooks ───────────────────────────────────────────────────────────
|
||||
git-hooks.hooks = {
|
||||
# Formats nix + nushell; package is wired up by the treefmt integration.
|
||||
treefmt.enable = true;
|
||||
|
||||
nu-check = {
|
||||
|
||||
+101
@@ -0,0 +1,101 @@
|
||||
# CLI reference
|
||||
|
||||
One binary, `hermenix`:
|
||||
|
||||
| | For |
|
||||
| -------------------------------- | ---------------------------------------------------------- |
|
||||
| `hermenix setup` | host provisioning — disks, cache key, image |
|
||||
| `hermenix migrate <src>` | existing flat `HERMES_HOME` → guest |
|
||||
| `hermenix migrate connections` | connection files only |
|
||||
| `hermenix remote <host> "<cmd>"` | `setup`/`migrate` over SSH |
|
||||
| `hermenix vm <sub>` | VM lifecycle, snapshots, backups |
|
||||
| `hermenix ctl <sub>` | host-side bridge into the guest (files, unit, passthrough) |
|
||||
| `hermenix version` | packaged version string |
|
||||
|
||||
Quote multi-word remote/passthrough args as one string —
|
||||
[CONCEPTS.md](CONCEPTS.md#quote-multi-word-subcommands).
|
||||
|
||||
Setup: [INSTALL.md](INSTALL.md). Migrate: [MIGRATE.md](MIGRATE.md).
|
||||
|
||||
---
|
||||
|
||||
## `hermenix vm`
|
||||
|
||||
```bash
|
||||
hermenix vm ssh # interactive shell (vsock)
|
||||
hermenix vm ssh "systemctl status" # run a command and exit
|
||||
hermenix vm console # journalctl -u hermes-vm -f (read-only)
|
||||
hermenix vm start | stop | restart
|
||||
|
||||
hermenix vm config # guest generation, history, git status
|
||||
hermenix vm config-pull [dir] # copy guest /etc/nixos here
|
||||
|
||||
hermenix vm snapshot | snapshots
|
||||
hermenix vm rollback <snap> # type 'rollback' to confirm
|
||||
|
||||
hermenix vm backup | backups
|
||||
```
|
||||
|
||||
`console` is follow-only (no input). No interactive fallback if the guest is
|
||||
dead.
|
||||
|
||||
Repo `guest/` is the **seed**; live guest `/etc/nixos` diverges after the
|
||||
agent edits it. `config-pull` is for reading/diffing — a host rebuild will not
|
||||
fix a guest config problem.
|
||||
|
||||
---
|
||||
|
||||
## `hermenix ctl`
|
||||
|
||||
Host-side helpers over vsock. They know where Hermes's files live in _this_
|
||||
VM layout and how to reach the gateway unit — they are not a second Hermes
|
||||
manual. For Hermes features (chat, sessions, providers, skills, …) use the
|
||||
upstream CLI or [Hermes Agent docs](https://github.com/NousResearch/hermes-agent); unknown `ctl` words are
|
||||
passed through to `hermes` in the guest.
|
||||
|
||||
```bash
|
||||
# Files under hermesHome (default /var/lib/hermes/.hermes)
|
||||
hermenix ctl soul [edit]
|
||||
hermenix ctl memory [edit] # memories/USER.md if present
|
||||
hermenix ctl conf [edit] # config.yaml
|
||||
hermenix ctl secrets [edit|show] # .env, mode 0600; masked by default
|
||||
|
||||
# Gateway user unit (name: services.hermenix.gatewayUnit)
|
||||
hermenix ctl status | logs | restart
|
||||
hermenix ctl units
|
||||
|
||||
# Seed / fold-back (destructive where noted)
|
||||
hermenix ctl reseed # overwrite from /etc/nixos/hermes seed
|
||||
hermenix ctl pull [dir] # live home → host (no .env/sessions/state.db)
|
||||
```
|
||||
|
||||
Passthrough examples (Hermes's own commands):
|
||||
|
||||
```bash
|
||||
hermenix ctl chat
|
||||
hermenix ctl update
|
||||
hermenix ctl "sessions list"
|
||||
hermenix ctl doctor
|
||||
```
|
||||
|
||||
`edit` pulls over vsock, opens local `$EDITOR`, writes back, offers restart.
|
||||
Secrets never enter the Nix store — no declarative option for them.
|
||||
|
||||
If `status`/`logs` are empty, check `hermenix ctl units` and set
|
||||
`services.hermenix.gatewayUnit` to the real unit name.
|
||||
|
||||
`services.hermenix.hermesHome` is where `ctl` looks. Override for a flat
|
||||
migrated layout; export `HERMES_HOME` inside the guest for the gateway
|
||||
process if it differs from the default.
|
||||
|
||||
---
|
||||
|
||||
## Env baked in by the host module
|
||||
|
||||
| Env | Option | Default |
|
||||
| ---------------------- | -------------------------------- | ------------------------- |
|
||||
| `HERMES_VSOCK_CID` | `services.hermenix.vsockCid` | `77` |
|
||||
| `HERMES_GUEST_NAME` | `services.hermenix.name` | `hermes` |
|
||||
| `HERMES_DISK_BASE_DIR` | `services.hermenix.disk.baseDir` | `/var/lib/hermes-vm` |
|
||||
| `HERMES_HOME_DIR` | `services.hermenix.hermesHome` | `/var/lib/hermes/.hermes` |
|
||||
| `HERMES_GATEWAY_UNIT` | `services.hermenix.gatewayUnit` | `hermes-agent` |
|
||||
@@ -0,0 +1,98 @@
|
||||
# Concepts
|
||||
|
||||
Shared model for everything else in these docs. Read once; other pages assume it.
|
||||
|
||||
## Two disks
|
||||
|
||||
| | Holds | Size (default) | Snapshotted | Backed up |
|
||||
| ---------- | ---------------------------------------------- | -------------- | ----------- | --------- |
|
||||
| **system** | NixOS, `/nix/store` | 80G | No | No |
|
||||
| **state** | identity, workspace, `/etc/nixos`, Hermes home | 20G | Yes | Yes |
|
||||
|
||||
System is large, churny, rebuildable from this flake. State holds what cannot
|
||||
be reproduced — snapshots stay cheap.
|
||||
|
||||
Host images under `services.hermenix.disk.baseDir` (default `/var/lib/hermes-vm`):
|
||||
|
||||
```
|
||||
$baseDir/system/system.raw
|
||||
$baseDir/state/state.raw
|
||||
$baseDir/snapshots/
|
||||
```
|
||||
|
||||
Guest: state disk at **`/var/lib/hermes`**. Hermes home defaults to
|
||||
`/var/lib/hermes/.hermes` (`~/.hermes` for user `hermes`). `/etc/nixos` →
|
||||
`/var/lib/hermes/nixos`.
|
||||
|
||||
What belongs _inside_ Hermes home (SOUL, memories, skills, sessions, …) is
|
||||
defined by Hermes itself — see [upstream docs](https://github.com/NousResearch/hermes-agent). hermenix only
|
||||
decides which disk it lives on and that the seed is applied once.
|
||||
|
||||
## The boundary is the host
|
||||
|
||||
The guest owns its kernel and config. Nothing _inside_ the guest is a security
|
||||
boundary; `modules/host.nix` nftables are. QEMU runs as unprivileged
|
||||
`hermes-vm` with seccomp + systemd hardening. Thin device model: virtio-blk,
|
||||
virtio-net, vsock, RNG, serial. No virtiofs. `vhost-net` off on purpose.
|
||||
|
||||
| | Default |
|
||||
| ------------------------- | --------------------------------------------------- |
|
||||
| Public internet | Yes |
|
||||
| Host Nix cache | Yes, one port; `hostCache.enable = false` closes it |
|
||||
| Anything else on the host | No, unless `hostAccess` |
|
||||
| LAN / other VMs | No, unless `lanAccess` |
|
||||
| Inbound from the network | No, unless `publish` (NAT mode) |
|
||||
| Host hardware | None, unless `passthrough` |
|
||||
|
||||
`network.bridged = true`: LAN inbound = every port the guest opens; `publish`
|
||||
ignored. Outbound gates unchanged. [NETWORK.md](NETWORK.md).
|
||||
|
||||
## Seed once
|
||||
|
||||
`guest/hermes/` is a **template** copied into the guest on first boot only
|
||||
(`/var/lib/hermes/.seeded`):
|
||||
|
||||
```
|
||||
guest/hermes/
|
||||
home/ → ~/.hermes (SOUL.md, config.yaml, skills/, …)
|
||||
workspace/ → ~/workspace
|
||||
```
|
||||
|
||||
Hermes owns and rewrites its home after that — re-seeding on every rebuild
|
||||
would wipe learned state. hermenix consequences:
|
||||
|
||||
- Editing `guest/hermes/` post-first-boot does nothing alone →
|
||||
`hermenix ctl reseed` (destructive; needs fresh confirmation).
|
||||
- Live files: `hermenix ctl soul` / `conf` / … or `hermenix ctl pull`.
|
||||
- Machine facts (not identity): `/etc/hermes/ENVIRONMENT.md`, regenerated
|
||||
every guest rebuild from `guest/common.nix`.
|
||||
|
||||
## vsock, not network SSH
|
||||
|
||||
Host → guest (`vm ssh`, `ctl`, backups) uses **vsock** (default CID 77). No
|
||||
management SSH port on the network path.
|
||||
|
||||
```
|
||||
ssh -o ProxyCommand='socat - VSOCK-CONNECT:<cid>:22' hermes@hermes
|
||||
```
|
||||
|
||||
## Determinate Nix in the guest
|
||||
|
||||
Guest: **Determinate Nix**, nixpkgs on FlakeHub. Host: stock Nix.
|
||||
|
||||
- Do not edit `/etc/nix/nix.conf` — custom settings in
|
||||
`/etc/nix/nix.custom.conf` (from `guest/common.nix`).
|
||||
- No guest `nix.settings` / `nix.gc`; `determinate-nixd` collects.
|
||||
- Substituters: FlakeHub Cache → host cache → `cache.nixos.org`.
|
||||
- Do not set `inputs.nixpkgs.follows` on `determinate` (cache misses).
|
||||
|
||||
## Quote multi-word subcommands
|
||||
|
||||
nu treats bare `--flag` tokens as flags on the _outer_ command. Quote anything
|
||||
past a single bare word:
|
||||
|
||||
```bash
|
||||
hermenix remote fwdesk.lab.internal "setup --clean"
|
||||
hermenix vm ssh "systemctl status"
|
||||
hermenix ctl "sessions list"
|
||||
```
|
||||
@@ -0,0 +1,82 @@
|
||||
# Hacking on this repo
|
||||
|
||||
## Devshell
|
||||
|
||||
```bash
|
||||
direnv allow
|
||||
```
|
||||
|
||||
Provides nushell, nufmt, nixfmt, and pre-commit hooks (installed on first
|
||||
entry):
|
||||
|
||||
- **treefmt** — nixfmt / deadnix / statix / nufmt
|
||||
- **nu-check** — parse-check `*.nu`
|
||||
- **gen-completions** — regenerates `nu/completions.nu` from usage blocks
|
||||
- **nu-tests** — `bin/runtest` when anything under `nu/` changes
|
||||
|
||||
## Layout
|
||||
|
||||
See [GUIDELINES.md](../GUIDELINES.md#repository-layout). The CLI is several
|
||||
nushell files concatenated by `flake.nix` in dependency order
|
||||
(`lib/` → `host/` → `vm/` → `ctl/` → `hermenix.nu`).
|
||||
|
||||
## Testing
|
||||
|
||||
Three suites. **Agents may run them only through the wrappers below** — never
|
||||
via bare `nix build` / `nix flake check`. See [GUIDELINES.md](../GUIDELINES.md).
|
||||
|
||||
### CLI tests (`nu/tests/`, nutest)
|
||||
|
||||
```bash
|
||||
runtest # everything (bin/ on PATH in the devShell)
|
||||
runtest vm_test
|
||||
runtest-debug vm_test
|
||||
```
|
||||
|
||||
Pure helpers + smoke that the subcommand tree parses — no ssh, no disk writes,
|
||||
no `nix build`.
|
||||
|
||||
### Guest VM tests (`guest/vm-tests/`)
|
||||
|
||||
Boot `guest/common.nix` in ephemeral NixOS test VMs. Never touches
|
||||
`/var/lib/hermes-vm/**`:
|
||||
|
||||
```bash
|
||||
runvmtest # NAT / isolated-bridge (default)
|
||||
runvmtest guest-vm-bridged # bridged mode with simulated LAN
|
||||
runvmtest-debug # interactive driver REPL
|
||||
```
|
||||
|
||||
Shared seeding/SSH/user assertions live in `guest/vm-tests/common.nix`.
|
||||
Mode-specific checks:
|
||||
|
||||
- **unbridged** — static `10.77.0.2/24` only, one default route via host
|
||||
- **bridged** — DHCP lease beside service net, dual default routes, LAN
|
||||
reachability both ways
|
||||
|
||||
Does not cover real `hermes-install` (no internet in the test VM) or the host
|
||||
`hermes-vm` QEMU unit.
|
||||
|
||||
### Module evaluation checks
|
||||
|
||||
Static — no VM boot. Evaluates `modules/host.nix` in both network modes plus
|
||||
`guest/common.nix`, asserts on nftables / networkd / guest DHCP metrics.
|
||||
|
||||
Operator/CI (not the agent-safe wrapper path today):
|
||||
|
||||
```bash
|
||||
nix build .#checks.x86_64-linux.network-modes
|
||||
```
|
||||
|
||||
Agents: describe the check and let the operator run it, unless/until a
|
||||
`bin/` wrapper exists for it.
|
||||
|
||||
## Parsing the CLI without running it
|
||||
|
||||
```bash
|
||||
nu nu/hermenix.nu --help
|
||||
nu nu/hermenix.nu setup --help
|
||||
# etc.
|
||||
```
|
||||
|
||||
Nix syntax only: `nix-instantiate --parse path/to/file.nix`.
|
||||
+117
@@ -0,0 +1,117 @@
|
||||
# Install
|
||||
|
||||
## Guided (recommended)
|
||||
|
||||
The `hermenix` CLI is a flake package — no checkout needed:
|
||||
|
||||
```bash
|
||||
nix run git+ssh://gitea@gitea.skyg.ca/ari/hermenix.git#hermenix -- setup
|
||||
```
|
||||
|
||||
Or install once:
|
||||
|
||||
```bash
|
||||
nix profile install git+ssh://gitea@gitea.skyg.ca/ari/hermenix.git#hermenix
|
||||
hermenix setup
|
||||
```
|
||||
|
||||
`setup` walks host-side steps: prerequisites, disks, cache signing key,
|
||||
`/etc/hermenix/local.nix` from your SSH keys, bootstrap image build, write to
|
||||
the system disk. It prints the module snippet to paste into your configuration
|
||||
(uplink filled in).
|
||||
|
||||
Safe to re-run — steps skip what's done; nothing destructive without
|
||||
confirmation. It never runs `nixos-rebuild` (that's your config).
|
||||
|
||||
Hand-run equivalent: [MANUAL_SETUP.md](MANUAL_SETUP.md).
|
||||
|
||||
## Add the module
|
||||
|
||||
```nix
|
||||
# flake.nix
|
||||
{
|
||||
inputs.hermenix.url = "git+ssh://gitea@gitea.skyg.ca/ari/hermenix.git";
|
||||
|
||||
outputs = { nixpkgs, hermenix, ... }: {
|
||||
nixosConfigurations.myserver = nixpkgs.lib.nixosSystem {
|
||||
system = "x86_64-linux";
|
||||
modules = [ hermenix.nixosModules.default ./configuration.nix ];
|
||||
};
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
```nix
|
||||
# configuration.nix
|
||||
services.hermenix = {
|
||||
enable = true;
|
||||
disk.baseDir = "/var/lib/hermes-vm";
|
||||
network.externalInterface = "eno1"; # your uplink
|
||||
# network.bridged = true; # see docs/NETWORK.md
|
||||
vcpu = 4;
|
||||
mem = 8192;
|
||||
};
|
||||
```
|
||||
|
||||
```bash
|
||||
sudo nixos-rebuild switch
|
||||
hermenix vm console # watch it boot
|
||||
```
|
||||
|
||||
## First boot (hermenix side)
|
||||
|
||||
```bash
|
||||
hermenix vm ssh
|
||||
|
||||
# Labels must match guest/hardware.nix BEFORE any guest rebuild
|
||||
lsblk -f
|
||||
|
||||
# FlakeHub Cache (device-code flow — approve URL from any browser)
|
||||
determinate-nixd login
|
||||
|
||||
sudo nix flake lock /etc/nixos
|
||||
```
|
||||
|
||||
The Hermes CLI installs unattended on first boot (`hermes-install.service`;
|
||||
a few minutes for native modules). After that, configure Hermes itself with
|
||||
its own CLI — provider, keys, gateway, etc. That is **upstream Hermes**, not
|
||||
hermenix; see the [Hermes Agent docs](https://github.com/NousResearch/hermes-agent).
|
||||
|
||||
From the host you can drive the guest install/gateway unit once it exists:
|
||||
|
||||
```bash
|
||||
hermenix ctl units
|
||||
hermenix ctl status
|
||||
```
|
||||
|
||||
> **Check labels before the first guest `nixos-rebuild`.**
|
||||
> `guest/hardware.nix` assumes root `nixos` and ESP `ESP`. Wrong labels →
|
||||
> unbootable system after the agent's first self-rebuild.
|
||||
|
||||
Guest NixOS changes are durable via
|
||||
`sudo nixos-rebuild switch --flake /etc/nixos#hermes` inside the VM. Hermes
|
||||
self-updates are upstream (`hermes update`); use `hermenix ctl` only as a
|
||||
vsock passthrough if you prefer running them from the host.
|
||||
|
||||
## Remote host
|
||||
|
||||
`setup` / `migrate` act on the machine they run on. `hermenix remote` SSHs:
|
||||
|
||||
```bash
|
||||
hermenix remote fwdesk.lab.internal setup
|
||||
hermenix remote fwdesk.lab.internal "setup --clean"
|
||||
hermenix remote fwdesk.lab.internal "migrate /var/lib/hermes"
|
||||
```
|
||||
|
||||
Quote subcommand + flags as **one string** when more than a bare name —
|
||||
[CONCEPTS.md](CONCEPTS.md#quote-multi-word-subcommands).
|
||||
|
||||
Runs `nix run <flake>#hermenix -- …` on the remote (no checkout). Remote needs
|
||||
SSH access to Gitea. Override URL with `--flake`. Uses `ssh -t` for prompts.
|
||||
|
||||
## Next
|
||||
|
||||
- [CLI.md](CLI.md) — day-to-day `vm` / `ctl`
|
||||
- [NETWORK.md](NETWORK.md) — allowlists / bridged
|
||||
- [MIGRATE.md](MIGRATE.md) — existing `HERMES_HOME`
|
||||
- [RECOVERY.md](RECOVERY.md) — snapshots / unbootable guest
|
||||
@@ -0,0 +1,84 @@
|
||||
# Manual setup
|
||||
|
||||
What `hermenix setup` does, step by step, for hand-running or when something
|
||||
goes sideways. Prefer the guided command in [INSTALL.md](INSTALL.md) when you
|
||||
can.
|
||||
|
||||
## 1. Create the disks
|
||||
|
||||
Two disks — design in [CONCEPTS.md](CONCEPTS.md#two-disks).
|
||||
|
||||
```bash
|
||||
sudo btrfs subvolume create /var/lib/hermes-vm/system
|
||||
sudo btrfs subvolume create /var/lib/hermes-vm/state
|
||||
sudo mkdir -p /var/lib/hermes-vm/snapshots
|
||||
|
||||
# BEFORE creating any file inside them
|
||||
sudo chattr +C /var/lib/hermes-vm/system /var/lib/hermes-vm/state
|
||||
sudo btrfs property set /var/lib/hermes-vm/system compression none
|
||||
sudo btrfs property set /var/lib/hermes-vm/state compression none
|
||||
|
||||
sudo truncate -s 80G /var/lib/hermes-vm/system/system.raw
|
||||
sudo truncate -s 20G /var/lib/hermes-vm/state/state.raw
|
||||
sudo mkfs.ext4 -L hermes-state /var/lib/hermes-vm/state/state.raw
|
||||
```
|
||||
|
||||
**`chattr +C` ordering matters.** Btrfs CoW stacked under a VM disk image
|
||||
fragments badly. `+C` disables that — but only for files created _after_.
|
||||
Set it on empty subvolumes and confirm with `lsattr` before writing anything.
|
||||
Wrong order → recreate the subvolume and copy the image in fresh.
|
||||
|
||||
Separate subvolumes so a btrfs snapshot of state does not include the system
|
||||
image.
|
||||
|
||||
## 2. Generate the cache signing key
|
||||
|
||||
The host serves its store to the guest as a read-only binary cache.
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /var/lib/secrets
|
||||
sudo nix-store --generate-binary-cache-key hermes-cache-1 \
|
||||
/var/lib/secrets/hermes-cache.secret /var/lib/secrets/hermes-cache.pub
|
||||
sudo chown nix-serve /var/lib/secrets/hermes-cache.secret
|
||||
sudo chmod 0400 /var/lib/secrets/hermes-cache.secret
|
||||
cat /var/lib/secrets/hermes-cache.pub
|
||||
```
|
||||
|
||||
## 3. Write `/etc/hermenix/local.nix`
|
||||
|
||||
Host-specific settings live here, not in this repo — see
|
||||
`guest/local.nix.example`:
|
||||
|
||||
```nix
|
||||
{
|
||||
cachePublicKey = "hermes-cache-1:AbCd..."; # key you just printed; empty = skip host cache
|
||||
authorizedKeys = [ "ssh-ed25519 AAAA... you@laptop" ];
|
||||
}
|
||||
```
|
||||
|
||||
`sudo mkdir -p /etc/hermenix` first if needed.
|
||||
|
||||
## 4. Build the image and write it to the system disk
|
||||
|
||||
`--impure` is required because `guest/common.nix` reads
|
||||
`/etc/hermenix/local.nix` outside the flake source tree. The extra substituter
|
||||
flags avoid compiling Determinate Nix from source. `setup` also passes
|
||||
`--no-write-lock-file --refresh` when building from the git URL — mirror that
|
||||
if you're doing the same by hand.
|
||||
|
||||
```bash
|
||||
nix build --impure --no-write-lock-file --refresh \
|
||||
git+ssh://gitea@gitea.skyg.ca/ari/hermenix.git#hermes-guest-image \
|
||||
--option extra-substituters https://install.determinate.systems \
|
||||
--option extra-trusted-public-keys cache.flakehub.com-3:hJuILl5sVK4iKm86JzgdXW12Y2Hwd5G07qKtHTOcDCM=
|
||||
|
||||
# notrunc keeps the disk at its full size; the image is smaller
|
||||
sudo dd if=result/nixos.img of=/var/lib/hermes-vm/system/system.raw \
|
||||
bs=4M conv=notrunc,fsync status=progress
|
||||
```
|
||||
|
||||
From a local checkout, use `.` instead of the git URL (still `--impure`).
|
||||
|
||||
## 5. Module + first boot
|
||||
|
||||
Continue from [INSTALL.md](INSTALL.md#add-the-module).
|
||||
@@ -0,0 +1,74 @@
|
||||
# Migrate an existing deployment
|
||||
|
||||
Copy an existing flat `HERMES_HOME` into the guest after hermenix setup + host
|
||||
`nixos-rebuild` and `hermenix vm ssh true` work:
|
||||
|
||||
```bash
|
||||
nix run git+ssh://gitea@gitea.skyg.ca/ari/hermenix.git#hermenix -- \
|
||||
migrate /path/to/existing/HERMES_HOME
|
||||
```
|
||||
|
||||
`migrate --help` for vsock CID, guest user, `--dest`, hostname rewrites.
|
||||
|
||||
**Copy only — never move or delete the source** until the new side is verified.
|
||||
|
||||
Remote: [INSTALL.md](INSTALL.md#remote-host) — put `migrate /path/...` in the
|
||||
quoted command string.
|
||||
|
||||
What a `HERMES_HOME` contains and how Hermes uses it is upstream territory —
|
||||
[Hermes Agent](https://github.com/NousResearch/hermes-agent). This page is only the hermenix carry path.
|
||||
|
||||
## `hermesHome`
|
||||
|
||||
`services.hermenix.hermesHome` is where `hermenix ctl` looks in the guest.
|
||||
|
||||
| Layout | Value |
|
||||
| -------------------------- | -------------------------------------------- |
|
||||
| Seeded `.hermes` (default) | `/var/lib/hermes/.hermes` |
|
||||
| Flat at state root | `migrate --dest` (default `/var/lib/hermes`) |
|
||||
|
||||
Export `HERMES_HOME` inside the guest to the same path for the gateway process.
|
||||
|
||||
## Don't use `sudo`
|
||||
|
||||
`migrate` uses your user's ssh-over-vsock key (same as `authorizedKeys`).
|
||||
`sudo` → root has no key → `Permission denied (publickey)`. If
|
||||
`hermenix vm ssh` works as you, run `migrate` the same way.
|
||||
|
||||
## `hermenix migrate connections`
|
||||
|
||||
Connection/integration files only (no identity wipe), local or `--to-guest`:
|
||||
|
||||
```bash
|
||||
hermenix migrate connections /old/HERMES_HOME ./staging
|
||||
hermenix migrate connections /old/HERMES_HOME --to-guest
|
||||
```
|
||||
|
||||
Defaults: files `config.yaml`, `auth.json`, `.env`; dir `connections/`.
|
||||
Rewrites apply in the destination only.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Unreadable source entries
|
||||
|
||||
Rootless-Docker leftovers may be unreadable; `tar --ignore-failed-read` makes
|
||||
that non-fatal. Pull those paths as root separately if needed:
|
||||
|
||||
```bash
|
||||
sudo tar -C /path/to/HERMES_HOME/.hermes -cf - . \
|
||||
| hermenix vm ssh 'mkdir -p /var/lib/hermes/.hermes-old && tar -C /var/lib/hermes/.hermes-old -xf -'
|
||||
```
|
||||
|
||||
### Stale Docker launcher after an old migrate
|
||||
|
||||
Current `migrate` excludes `.local/bin` and `.hermes-installed`. If an older
|
||||
run left a Docker-routing `~/.local/bin/hermes` ("docker not found on PATH"):
|
||||
|
||||
```bash
|
||||
rm -f ~/.local/bin/hermes ~/.hermes-installed
|
||||
sudo systemctl start hermes-install # or reinstall per upstream docs
|
||||
journalctl -fu hermes-install
|
||||
```
|
||||
|
||||
Then finish Hermes configuration with the upstream CLI
|
||||
([docs](https://github.com/NousResearch/hermes-agent)), not with hermenix-specific steps.
|
||||
+120
@@ -0,0 +1,120 @@
|
||||
# Network
|
||||
|
||||
Default model and reachability table: [CONCEPTS.md](CONCEPTS.md#the-boundary-is-the-host).
|
||||
|
||||
## Mode matrix
|
||||
|
||||
| | NAT (default) | Bridged (`network.bridged = true`) |
|
||||
| ------------------------------ | ------------------------------------- | ---------------------------------------- |
|
||||
| Guest address | `10.77.0.2` on isolated bridge | service net **plus** fixed LAN address |
|
||||
| Inbound from LAN / internet | only via `publish` | every port the guest opens |
|
||||
| Guest → public internet | yes (masqueraded) | yes (L2 / LAN gateway) |
|
||||
| Guest → private LAN | `lanAccess` / `allowLan` only | same gates |
|
||||
| Guest → host services | `hostAccess` + host cache on `10.77.0.1` | same over service net |
|
||||
| `publish` | DNAT on uplink | **ignored** |
|
||||
|
||||
## Guest → a service on your LAN
|
||||
|
||||
```nix
|
||||
services.hermenix.lanAccess = [
|
||||
{ address = "192.168.1.50"; port = 11434; } # ollama
|
||||
{ address = "192.168.1.50"; port = 9000; proto = "udp"; }
|
||||
];
|
||||
```
|
||||
|
||||
One destination:port per entry, punched through the RFC1918 drop. Traffic is
|
||||
masqueraded; the peer sees the host's address.
|
||||
|
||||
`allowLan = true` opens all private space at once. Prefer `lanAccess` — an
|
||||
agent that goes wrong should not wander.
|
||||
|
||||
## Guest → a service on the host
|
||||
|
||||
```nix
|
||||
services.hermenix.hostAccess = [ { port = 8188; } ];
|
||||
```
|
||||
|
||||
The service must bind somewhere the guest can see. `127.0.0.1` will not work
|
||||
(guest arrives on the bridge). Bind to `10.77.0.1` (bridge only) or
|
||||
`0.0.0.0`:
|
||||
|
||||
```nix
|
||||
services.comfyui.listenAddress = "10.77.0.1";
|
||||
```
|
||||
|
||||
From the guest: `http://10.77.0.1:8188`.
|
||||
|
||||
Often better than GPU passthrough when you only need the capability over HTTP
|
||||
— see [PASSTHROUGH.md](PASSTHROUGH.md).
|
||||
|
||||
## Expose a guest service
|
||||
|
||||
**Guest side** — agent opens its firewall in `/etc/nixos`:
|
||||
|
||||
```nix
|
||||
networking.firewall.allowedTCPPorts = [ 22 8080 ];
|
||||
```
|
||||
|
||||
**Host only** — already allowed (outbound host→guest):
|
||||
|
||||
```bash
|
||||
curl http://10.77.0.2:8080
|
||||
```
|
||||
|
||||
**Wider network** (NAT mode):
|
||||
|
||||
```nix
|
||||
services.hermenix.publish = [
|
||||
{ hostPort = 8080; guestPort = 8080; }
|
||||
];
|
||||
```
|
||||
|
||||
DNATs the uplink port and opens the forward rule. Ports need not match.
|
||||
Publishing is the one change that lets the outside world start conversations
|
||||
with the agent's machine — put TLS/auth in front of anything that isn't a toy.
|
||||
|
||||
## Bridged mode (guest on the LAN)
|
||||
|
||||
Guest becomes a first-class LAN device: fixed address, reachable on any port it
|
||||
opens, outbound still gated.
|
||||
|
||||
```nix
|
||||
services.hermenix.network = {
|
||||
bridged = true;
|
||||
externalInterface = "enp191s0";
|
||||
|
||||
lanAddress = "10.0.12.50"; # guest's fixed LAN address
|
||||
hostLanAddress = "10.0.12.10"; # this host's LAN address
|
||||
lanGateway = "10.0.12.1";
|
||||
lanPrefixLength = 24;
|
||||
# lanDns = [ "1.1.1.1" ]; # optional; defaults to lanGateway
|
||||
};
|
||||
```
|
||||
|
||||
The bridge absorbs the uplink. The host-side DHCP on the bridge reserves
|
||||
`lanAddress` for the guest MAC; the guest stays a plain DHCP client. The
|
||||
bridge clones the uplink MAC so the host keeps its LAN identity. Guest DHCP
|
||||
broadcast never leaves the bridge.
|
||||
|
||||
### Switching over
|
||||
|
||||
1. Pick free addresses for guest, host (use the host's existing LAN address if
|
||||
reserved), and gateway. Keep them outside the DHCP pool or reserve them.
|
||||
2. Set options above, `sudo nixos-rebuild switch` (brief network drop).
|
||||
3. Verify: `hermenix vm ssh`; inside, `ip -4 addr` shows `10.77.0.2` **and**
|
||||
the LAN address. From another LAN device:
|
||||
`ssh hermes@<guest-lan-address>`.
|
||||
|
||||
### Switching back
|
||||
|
||||
`bridged = false` and rebuild. Guest DHCP finds no server on the isolated
|
||||
bridge and falls back to the static route via the host. No guest-side change.
|
||||
|
||||
### Gotchas
|
||||
|
||||
- `hostLanAddress` is a static LAN claim — reserve it.
|
||||
- Uplink network config is replaced by the bridge-port role while bridged.
|
||||
- Older guest images without DHCP-on-NIC in `guest/common.nix` won't take a
|
||||
bridged address; upgrade the image or apply that config inside the VM first.
|
||||
- Drop `publish` entries if you return to NAT mode; they do nothing while
|
||||
bridged.
|
||||
@@ -0,0 +1,124 @@
|
||||
# Hardware passthrough & virtual devices
|
||||
|
||||
## Before reaching for passthrough
|
||||
|
||||
If the goal is "the agent can use the GPU", passthrough is often wrong. It is
|
||||
exclusive (host loses the card while the VM runs) and is the largest isolation
|
||||
reduction in this setup.
|
||||
|
||||
Prefer: keep Ollama/ComfyUI on the host, open the port with `hostAccess`
|
||||
([NETWORK.md](NETWORK.md#guest--a-service-on-the-host)). Pass hardware through
|
||||
when the guest genuinely needs the device — CUDA in-process, capture card,
|
||||
radio, display output.
|
||||
|
||||
## What PCI passthrough costs
|
||||
|
||||
A PCI device does DMA. The IOMMU is the only thing between it and host memory,
|
||||
so `passthrough.iommu` is required; the module refuses to build without it.
|
||||
|
||||
- **Whole IOMMU groups only.** Check before planning:
|
||||
|
||||
```bash
|
||||
for d in /sys/kernel/iommu_groups/*/devices/*; do
|
||||
echo "group $(basename "$(dirname "$(dirname "$d")")"): $(lspci -nns "$(basename "$d")")"
|
||||
done | sort -V
|
||||
```
|
||||
|
||||
ACS override patches lie about isolation — not appropriate here.
|
||||
|
||||
- **Every function.** A GPU is at least graphics + HDMI audio.
|
||||
|
||||
## GPU
|
||||
|
||||
```bash
|
||||
lspci -Dnn | grep -i vga
|
||||
```
|
||||
|
||||
```nix
|
||||
services.hermenix.passthrough = {
|
||||
iommu = "amd"; # or "intel"
|
||||
pci = [
|
||||
{ address = "0000:01:00.0"; id = "10de:2204"; } # graphics
|
||||
{ address = "0000:01:00.1"; id = "10de:1aef"; } # HDMI audio
|
||||
];
|
||||
};
|
||||
```
|
||||
|
||||
Setting `id` makes vfio-pci claim the card in the initrd before `nvidia` /
|
||||
`amdgpu`. Do this for GPUs. Address-only devices bind at boot via a oneshot
|
||||
instead.
|
||||
|
||||
Rebuild and **reboot**, then:
|
||||
|
||||
```bash
|
||||
lspci -nnk -d 10de:2204 | grep "Kernel driver" # want: vfio-pci
|
||||
```
|
||||
|
||||
Guest-side drivers are the agent's problem (`/etc/nixos`).
|
||||
|
||||
## USB / Bluetooth
|
||||
|
||||
```bash
|
||||
lsusb
|
||||
```
|
||||
|
||||
```nix
|
||||
services.hermenix.passthrough.usb = [
|
||||
{ vendorId = "8087"; productId = "0026"; }
|
||||
{ vendorId = "0403"; productId = "6001"; }
|
||||
];
|
||||
```
|
||||
|
||||
QEMU mediates (no IOMMU/VFIO). Host must not be using the device (`btusb`
|
||||
blacklist, no `hardware.bluetooth` for an integrated radio). No hotplug —
|
||||
`hermenix vm restart` after plugging. Whole USB controller as PCI gives real
|
||||
hotplug at the cost of exclusivity.
|
||||
|
||||
## When the VM won't start
|
||||
|
||||
`hermenix vm console` first. Then: device not on vfio-pci (`lspci -nnk`),
|
||||
IOMMU off (`dmesg | grep -i iommu`), or something in the group still held by
|
||||
the host. The module raises `LimitMEMLOCK` when PCI passthrough is configured.
|
||||
|
||||
---
|
||||
|
||||
## Virtual devices
|
||||
|
||||
`services.hermenix.devices` attaches optional QEMU devices (no host hardware):
|
||||
|
||||
| Option | Default | Attaches |
|
||||
| ------------------------- | ---------- | ----------------------------------------------------- |
|
||||
| `devices.rng.enable` | `true` | virtio entropy |
|
||||
| `devices.vsock.enable` | `true` | host↔guest vsock (`ctl`, backups) |
|
||||
| `devices.video.enable` | `false` | virtual display / render node |
|
||||
| `devices.video.model` | `"virtio"` | `virtio` (render node) or legacy VGA |
|
||||
| `devices.balloon.enable` | `false` | virtio memory balloon |
|
||||
| `devices.watchdog.enable` | `false` | `i6300esb`/`ib700` |
|
||||
| `devices.input.enable` | `false` | virtio keyboard/tablet |
|
||||
| `devices.audio.enable` | `false` | `intel-hda`/`virtio-sound` (see caveat) |
|
||||
| `devices.tpm.enable` | `false` | TPM 2.0 via swtpm |
|
||||
| `devices.cdroms` | `[]` | read-only CD images |
|
||||
| `devices.extraDisks` | `[]` | extra raw disks |
|
||||
| `devices.usbController` | `"auto"` | `qemu-xhci` iff `passthrough.usb` set |
|
||||
|
||||
`extraQemuArgs` appends raw QEMU args (unvalidated). Prefer typed options.
|
||||
|
||||
### Render node
|
||||
|
||||
```nix
|
||||
services.hermenix.devices.video.enable = true;
|
||||
```
|
||||
|
||||
Gives `/dev/dri/card0` and `/dev/dri/renderD128` (`virtio` model only).
|
||||
Console stays serial.
|
||||
|
||||
### TPM
|
||||
|
||||
`devices.tpm.enable = true` starts swtpm; state under
|
||||
`/var/lib/hermes-vm-tpm` by default.
|
||||
|
||||
### Audio caveat
|
||||
|
||||
`hermes-vm.service` is hardened (`ProtectHome`, `PrivateTmp`, dedicated user)
|
||||
and cannot reach PipeWire without loosening that. Leave audio off unless you
|
||||
mean to relax the unit.
|
||||
@@ -0,0 +1,63 @@
|
||||
# Recovery
|
||||
|
||||
Cheapest first. Layers 1–2 are inside the guest; 3–4 are host-side hermenix.
|
||||
|
||||
```bash
|
||||
# Guest NixOS history
|
||||
hermenix vm ssh "git -C /etc/nixos log --oneline"
|
||||
hermenix vm ssh "sudo nixos-rebuild switch --rollback" # also in boot menu
|
||||
|
||||
# State-disk snapshot
|
||||
hermenix vm snapshot
|
||||
hermenix vm snapshots
|
||||
hermenix vm rollback state-20260914-1400 # type 'rollback' to confirm
|
||||
|
||||
# Restic backup of state files
|
||||
hermenix vm backup
|
||||
hermenix vm backups
|
||||
```
|
||||
|
||||
Identity files under Hermes home can be inspected with `hermenix ctl soul`
|
||||
etc.; restoring _Hermes_ app state is mostly "roll the state disk" or restore
|
||||
from backup — not a separate product flow.
|
||||
|
||||
## Snapshots
|
||||
|
||||
Hourly, keep 48, **state disk only**. System disk: re-image with
|
||||
`hermenix setup`.
|
||||
|
||||
fsfreeze via qemu-guest-agent around each shot. `rollback` stops the VM, moves
|
||||
current state to `state.old-<timestamp>`, restores snapshot — reversible until
|
||||
you delete `state.old-*`.
|
||||
|
||||
Keep retention tighter than on ZFS: state image is nodatacow; live snapshots
|
||||
reintroduce CoW on referenced blocks.
|
||||
|
||||
## Backups
|
||||
|
||||
```nix
|
||||
services.hermenix.backup = {
|
||||
enable = true;
|
||||
repository = "sftp:backup@nas:/hermes";
|
||||
passwordFile = "/var/lib/secrets/hermes-restic";
|
||||
};
|
||||
```
|
||||
|
||||
State **files** over vsock/SSH into restic — not the raw disk. Portable;
|
||||
avoids loop-mounting agent-controlled FS on the host. Runs on the host (no
|
||||
`lanAccess` needed for a LAN repo).
|
||||
|
||||
## Guest won't boot
|
||||
|
||||
Re-image **system** only (`hermenix setup` write step); leave state disk.
|
||||
Identity + `/etc/nixos` + workspace return with it.
|
||||
|
||||
## Starting completely fresh
|
||||
|
||||
```bash
|
||||
hermenix setup --clean
|
||||
```
|
||||
|
||||
Stops VM; deletes `system.raw` / `state.raw` / UEFI vars; drops stale
|
||||
`known_hosts` for the guest host key; walks setup again. Subvolumes/snapshots
|
||||
kept. Type `clean` to confirm.
|
||||
@@ -0,0 +1,21 @@
|
||||
# Known rough edges
|
||||
|
||||
- The Determinate NixOS module's option names for custom Nix settings aren't
|
||||
published — only the nix-darwin `determinateNix.customSettings` equivalent
|
||||
is. So `guest/common.nix` writes `/etc/nix/nix.custom.conf` via
|
||||
`environment.etc`. If a future module version claims that file, switch to
|
||||
its option.
|
||||
- `determinate-nixd login` is manual. No safe way to bake the credential into
|
||||
the image.
|
||||
- `MemoryDenyWriteExecute = true` is correct under KVM but is the first thing
|
||||
to flip if QEMU refuses to start.
|
||||
- `pkgs.OVMF.firmware` / `pkgs.OVMF.variables` are current attribute names;
|
||||
older nixpkgs put these under `pkgs.OVMF.fd` in an `FV/` subdirectory.
|
||||
- PCI passthrough needs a reboot (early vfio binding in initrd).
|
||||
- Snapshots need btrfs. Everything else works on any filesystem; without btrfs
|
||||
you rely on backups alone.
|
||||
- `state.old-*` left by `hermenix vm rollback` are not cleaned automatically.
|
||||
Delete them once you're sure.
|
||||
- First boot builds a lot. Give it time.
|
||||
- `devices.audio.enable` fights unit hardening — see
|
||||
[PASSTHROUGH.md](PASSTHROUGH.md#audio-caveat).
|
||||
@@ -20,8 +20,7 @@
|
||||
url = "github:numtide/treefmt-nix";
|
||||
inputs.nixpkgs.follows = "nixpkgs";
|
||||
};
|
||||
# A nushell test framework, used by nu/tests/. No nixpkgs package exists
|
||||
# for it, so it's pulled in as a plain source input instead.
|
||||
# No nixpkgs package; plain source for nu/tests.
|
||||
nutest = {
|
||||
url = "github:vyadh/nutest/v1.2.0";
|
||||
flake = false;
|
||||
@@ -117,23 +116,12 @@
|
||||
modules = [ ./devenv.nix ];
|
||||
};
|
||||
|
||||
# Never run by an agent — operator/CI only:
|
||||
# nix build .#checks.x86_64-linux.guest-vm-unbridged --impure -L
|
||||
# nix build .#checks.x86_64-linux.guest-vm-bridged --impure -L
|
||||
# nix build .#checks.x86_64-linux.migrate-connections-vm --impure -L
|
||||
#
|
||||
# The guest VM tests (guest/vm-tests/) share their node config and
|
||||
# mode-independent assertions via guest/vm-tests/common.nix; each test
|
||||
# adds the checks that only apply to its mode. default.nix aggregates
|
||||
# them into this attrset, which the static network-modes check joins.
|
||||
checks.${system} =
|
||||
(import ./guest/vm-tests {
|
||||
inherit pkgs determinate;
|
||||
hermenix = self.packages.${system}.hermenix;
|
||||
})
|
||||
// {
|
||||
# No build, no VM boot: evaluates host.nix in both networking modes
|
||||
# and guest/common.nix, and asserts on the generated rulesets/units.
|
||||
network-modes = import ./tests/network-modes-check.nix { inherit pkgs; };
|
||||
};
|
||||
};
|
||||
|
||||
+17
-126
@@ -6,22 +6,13 @@
|
||||
}:
|
||||
|
||||
let
|
||||
# These must match services.hermenix.* on the host. If you changed the
|
||||
# defaults there, change them here too before building the image.
|
||||
# Must match services.hermenix.* defaults on the host.
|
||||
hostAddress = "10.77.0.1";
|
||||
guestAddress = "10.77.0.2";
|
||||
prefixLength = 24;
|
||||
cachePort = 5000;
|
||||
|
||||
# Host-specific settings (cache signing key, your SSH key) live at a fixed
|
||||
# path on the HOST, not in this repo - see local.nix.example for the shape.
|
||||
# `hermenix setup` writes it there. Deliberately outside the flake's own
|
||||
# source tree: this flake is meant to be built straight from its git+ssh
|
||||
# URL with no local checkout at all (hence --impure, to allow reading a
|
||||
# path Nix can't otherwise prove is pure), and a repo-relative file would
|
||||
# either need a checkout to hold it or get baked into the immutable store
|
||||
# copy Nix fetches. Falls back to safe defaults (no host cache, no SSH
|
||||
# access) if it hasn't been created yet.
|
||||
# Host-only settings outside the flake (--impure). See local.nix.example.
|
||||
local = if builtins.pathExists /etc/hermenix/local.nix then import /etc/hermenix/local.nix else { };
|
||||
cachePublicKey = local.cachePublicKey or "";
|
||||
authorizedKeys = local.authorizedKeys or [ ];
|
||||
@@ -29,19 +20,7 @@ let
|
||||
stateDir = "/var/lib/hermes";
|
||||
useHostCache = cachePublicKey != "";
|
||||
|
||||
# Shared between environment.systemPackages (the interactive PATH) and
|
||||
# hermes-install's own `path` (below). environment.systemPackages does NOT
|
||||
# propagate into systemd services' PATH on its own - a bare `curl` inside
|
||||
# a systemd script only resolves if the unit is explicitly given a `path`,
|
||||
# which is what actually failed here (confirmed live: "curl: command not
|
||||
# found", then separately "sh: command not found" from the upstream
|
||||
# installer's uv-install step, both from inside hermes-install despite the
|
||||
# tools existing in systemPackages). `bash` is included explicitly because
|
||||
# its output provides `bin/sh` (the same symlink NixOS's own `/bin/sh`
|
||||
# compat path points at) - nothing else in this list provides a bare `sh`.
|
||||
# The rest of the coreutils-adjacent tools are a baseline so the next
|
||||
# third-party install script that shells out to some common Unix tool by
|
||||
# bare name doesn't trigger another one of these round-trips.
|
||||
# systemPackages does not populate systemd unit PATH; list deps once for both.
|
||||
hermesInstallDeps = with pkgs; [
|
||||
git
|
||||
curl
|
||||
@@ -74,11 +53,7 @@ let
|
||||
];
|
||||
in
|
||||
{
|
||||
# Both disks are virtio-blk, including the one this very image boots from
|
||||
# (see modules/host.nix's qemu invocation) - so this has to be here, not
|
||||
# only in hardware.nix. hardware.nix is excluded from this image's build
|
||||
# (nixos-generators supplies its own partitioning/bootloader instead), but
|
||||
# driver support for the VM environment itself is needed either way.
|
||||
# virtio drivers needed in bootstrap image too (hardware.nix is not imported there).
|
||||
imports = [ (modulesPath + "/profiles/qemu-guest.nix") ];
|
||||
boot.initrd.availableKernelModules = [
|
||||
"virtio_pci"
|
||||
@@ -91,17 +66,7 @@ in
|
||||
|
||||
networking.hostName = "hermes";
|
||||
|
||||
# =====================================================================
|
||||
# The state disk.
|
||||
#
|
||||
# Second virtio disk, holding everything that cannot be rebuilt: Hermes's
|
||||
# identity and memories, its workspace, and the NixOS config it maintains
|
||||
# for itself. The system disk holds NixOS and the Nix store and is
|
||||
# deliberately disposable.
|
||||
#
|
||||
# neededForBoot so it is mounted in the initrd, before activation runs and
|
||||
# tries to seed into it.
|
||||
# =====================================================================
|
||||
# State disk (identity, workspace, /etc/nixos). neededForBoot: seed runs before activation.
|
||||
fileSystems."${stateDir}" = {
|
||||
device = "/dev/disk/by-label/hermes-state";
|
||||
fsType = "ext4";
|
||||
@@ -112,14 +77,7 @@ in
|
||||
];
|
||||
};
|
||||
|
||||
# =====================================================================
|
||||
# Seed identity and configuration on first boot, then get out of the way.
|
||||
#
|
||||
# FIRST BOOT ONLY, deliberately. Hermes rewrites SOUL.md, its memories and
|
||||
# its own NixOS config as it goes - that is the point of the thing. Copying
|
||||
# the seed over it on every rebuild would quietly delete everything it had
|
||||
# worked out. Use `hermes-ctl reseed` when you actually mean to overwrite.
|
||||
# =====================================================================
|
||||
# First boot only — re-applying would wipe agent-owned SOUL/memories. Use hermenix ctl reseed.
|
||||
system.activationScripts.seedHermesState = {
|
||||
deps = [
|
||||
"users"
|
||||
@@ -139,15 +97,7 @@ in
|
||||
|
||||
chmod -R u+w ${stateDir}/nixos ${stateDir}/.hermes ${stateDir}/workspace
|
||||
|
||||
# Git history here is the agent's cheapest undo, and it stops
|
||||
# nixos-rebuild complaining about an untracked flake.
|
||||
#
|
||||
# Must run before the chown below, while ${stateDir}/nixos is still
|
||||
# root-owned: activation runs as root, and git refuses to operate on
|
||||
# a repository whose working tree is owned by a different user
|
||||
# ("detected dubious ownership") - it would otherwise fail here on
|
||||
# every boot, silently (activation script failures are non-fatal),
|
||||
# and the seed would never actually get committed.
|
||||
# Commit while still root-owned (git refuses foreign ownership; activation failures are silent).
|
||||
if [ ! -d ${stateDir}/nixos/.git ]; then
|
||||
${pkgs.git}/bin/git -C ${stateDir}/nixos init -q -b main
|
||||
${pkgs.git}/bin/git -C ${stateDir}/nixos add -A
|
||||
@@ -162,33 +112,13 @@ in
|
||||
chown hermes:hermes ${stateDir}/.seeded
|
||||
fi
|
||||
|
||||
# /etc/nixos points at the state disk, so the config is snapshotted and
|
||||
# backed up along with everything else the agent owns.
|
||||
#
|
||||
# /etc/nixos is a real directory by default on NixOS, not a symlink or
|
||||
# missing path - `ln -sfn` only replaces an existing LINKNAME in place
|
||||
# if it's already a symlink; against a plain directory it instead
|
||||
# creates the link *inside* it (`/etc/nixos/nixos -> ...`), leaving
|
||||
# /etc/nixos itself untouched. rm -rf first so this is idempotent
|
||||
# either way.
|
||||
# /etc/nixos is a real dir by default; ln -sfn would nest inside it — rm first.
|
||||
rm -rf /etc/nixos
|
||||
ln -sfn ${stateDir}/nixos /etc/nixos
|
||||
'';
|
||||
};
|
||||
|
||||
# =====================================================================
|
||||
# Networking: a static service-net address for talking to the host (nix
|
||||
# cache, hostAccess), plus DHCP so an operator can later switch this VM
|
||||
# onto the LAN (services.hermenix.network.bridged = true on the host).
|
||||
# - NAT mode: no DHCP server on the host bridge; the static default
|
||||
# route via the host is the only one and carries all traffic.
|
||||
# - bridged mode: the host runs a DHCP server on its bridge that hands
|
||||
# this NIC its fixed LAN address (services.hermenix.network.lanAddress,
|
||||
# keyed to this NIC's MAC); that lease's lower-metric default route
|
||||
# (100) wins over the static fallback (200), and LAN devices reach
|
||||
# the VM at the fixed address. The VM never needs to know the address.
|
||||
# Resolvers stay QUAD9 (UseDNS = false) in both modes.
|
||||
# =====================================================================
|
||||
# Static service-net + DHCP (bridged mode gets LAN lease at metric 100; host route at 200).
|
||||
networking.useNetworkd = true;
|
||||
systemd.network.enable = true;
|
||||
systemd.network.networks."10-eth" = {
|
||||
@@ -224,22 +154,9 @@ in
|
||||
};
|
||||
networking.firewall.allowedTCPPorts = [ 22 ];
|
||||
|
||||
# Lets the host fsfreeze before taking a snapshot.
|
||||
services.qemuGuest.enable = true;
|
||||
|
||||
# =====================================================================
|
||||
# Nix
|
||||
#
|
||||
# Determinate Nix owns /etc/nix/nix.conf and their docs are explicit that
|
||||
# nix.custom.conf is the only supported place for your own settings. So
|
||||
# there is no nix.settings block here, and no nix.gc either: determinate-nixd
|
||||
# runs the collector itself.
|
||||
#
|
||||
# If the Determinate NixOS module on your version manages this file itself,
|
||||
# drop this block and use its option instead. The documented nix-darwin
|
||||
# equivalent is determinateNix.customSettings; the NixOS module's own option
|
||||
# names are not published, so this writes the file directly.
|
||||
# =====================================================================
|
||||
# Determinate owns nix.conf — custom settings only via nix.custom.conf (no nix.settings).
|
||||
environment.etc."nix/nix.custom.conf".text = ''
|
||||
trusted-users = root hermes
|
||||
extra-experimental-features = nix-command flakes
|
||||
@@ -248,17 +165,8 @@ in
|
||||
extra-trusted-public-keys = cache.flakehub.com-3:hJuILl5sVK4iKm86JzgdXW12Y2Hwd5G07qKtHTOcDCM=${lib.optionalString useHostCache " ${cachePublicKey}"}
|
||||
'';
|
||||
|
||||
# Keep enough generations that a bad self-rebuild is one boot-menu entry
|
||||
# away from being undone.
|
||||
boot.loader.systemd-boot.configurationLimit = lib.mkDefault 20;
|
||||
|
||||
# =====================================================================
|
||||
# The agent's account
|
||||
#
|
||||
# Inside the VM the agent is effectively root. That is the point: the VM
|
||||
# is the boundary, not the account. It can rebuild the system, open ports,
|
||||
# add services, and none of it reaches the host.
|
||||
# =====================================================================
|
||||
users.users.hermes = {
|
||||
isNormalUser = true;
|
||||
home = stateDir;
|
||||
@@ -274,22 +182,7 @@ in
|
||||
users.groups.hermes = { };
|
||||
security.sudo.wheelNeedsPassword = false;
|
||||
|
||||
# =====================================================================
|
||||
# Hermes itself
|
||||
#
|
||||
# Installed with upstream's own installer rather than the Nix flake, so
|
||||
# that `hermes update` works natively and the agent owns its own config.
|
||||
# Upstream classes Nix/NixOS as a best-effort Tier 2 platform; inside a VM
|
||||
# built to contain mess, taking the supported path is the cheaper trade.
|
||||
#
|
||||
# The installer itself runs automatically on first boot (below) - it
|
||||
# supports a fully unattended mode and has no need for operator input.
|
||||
# `hermes setup` and `hermes gateway install` do need real input (model
|
||||
# provider, API keys, messaging platform tokens), so those stay manual:
|
||||
# hermes setup
|
||||
# hermes gateway install
|
||||
# loginctl enable-linger hermes
|
||||
# =====================================================================
|
||||
# Upstream installer (not Nix) so hermes update works. Interactive config is upstream docs.
|
||||
systemd.services.hermes-install = {
|
||||
description = "Install the Hermes Agent CLI (first boot only)";
|
||||
after = [ "network-online.target" ];
|
||||
@@ -335,13 +228,7 @@ in
|
||||
|
||||
environment.systemPackages = hermesInstallDeps;
|
||||
|
||||
# hermes has createHome = false (its home is the state disk, seeded above,
|
||||
# not NixOS's usual skeleton) so it never gets a ~/.bashrc to append to.
|
||||
# Upstream's installer puts the `hermes` launcher in ~/.local/bin and says
|
||||
# as much ("Could not detect shell config file to add ~/.local/bin to
|
||||
# PATH"). This option adds it via /etc/set-environment (PAM, every login
|
||||
# session) instead of relying on a per-user dotfile that will never exist
|
||||
# here. Same option hl-fwdesk.nix already sets on the physical host.
|
||||
# createHome=false so no ~/.bashrc; put ~/.local/bin on PATH via PAM instead.
|
||||
environment.localBinInPath = true;
|
||||
|
||||
programs.direnv = {
|
||||
@@ -376,9 +263,13 @@ in
|
||||
keep anything you care about outside /var/lib/hermes.
|
||||
- Your identity is ~/.hermes/SOUL.md and it is yours to edit. It was
|
||||
seeded once on first boot and is never overwritten by a rebuild, so
|
||||
changes you make there survive. Same for ~/.hermes/memories/.
|
||||
changes you make there survive. Same for ~/.hermes/memories/ and
|
||||
anything else you keep under ~/.hermes.
|
||||
- This file is regenerated on every system rebuild. Editing it is
|
||||
pointless; it will be replaced. Put durable notes in SOUL.md.
|
||||
- Before your first `nixos-rebuild switch`, commit /etc/nixos. That
|
||||
commit is the cheapest undo; previous generations are in the boot
|
||||
menu if a rebuild goes badly.
|
||||
- The skill at ~/.hermes/skills/nixos-self-config covers how to change
|
||||
this machine safely. Read it before your first rebuild.
|
||||
- In NAT mode you have no route to the host or to any local network:
|
||||
|
||||
+3
-11
@@ -1,16 +1,8 @@
|
||||
{ lib, ... }:
|
||||
|
||||
# Only used by the guest's own /etc/nixos flake (see guest/flake.nix). The
|
||||
# bootstrap image gets its partitioning and bootloader from nixos-generators
|
||||
# instead, which is why fileSystems/bootloader are kept out of common.nix -
|
||||
# but driver support for the VM environment (qemu-guest.nix profile, virtio
|
||||
# initrd modules) lives in common.nix instead, since the bootstrap image
|
||||
# needs that too. See common.nix if you're looking for that.
|
||||
#
|
||||
# VERIFY THE LABELS on first boot with `lsblk -f`. nixos-generators names the
|
||||
# root partition "nixos" and the ESP "ESP", but check rather than trust, and
|
||||
# fix here if they differ. Getting this wrong means the first self-rebuild
|
||||
# produces a system that will not boot.
|
||||
# Used only by the guest's /etc/nixos flake. Bootstrap image partitioning
|
||||
# comes from nixos-generators (see common.nix for virtio drivers).
|
||||
# Verify labels with `lsblk -f` before the first self-rebuild.
|
||||
|
||||
{
|
||||
fileSystems."/" = {
|
||||
|
||||
@@ -1,11 +1,9 @@
|
||||
# Minimal starter config.yaml.
|
||||
#
|
||||
# Seeded once on first boot to ~/.hermes/config.yaml, then never touched
|
||||
# again by a rebuild — edit it live with `hermes-ctl conf edit`, or just let
|
||||
# Hermes manage it. Fill in a model provider before first use.
|
||||
# Seeded once to ~/.hermes/config.yaml; never re-applied by a rebuild.
|
||||
# Shape and keys: upstream Hermes docs. Edit live via `hermenix ctl conf edit`
|
||||
# or let Hermes manage it.
|
||||
model:
|
||||
provider: ''
|
||||
default: ''
|
||||
provider: ""
|
||||
default: ""
|
||||
|
||||
agent:
|
||||
reasoning_effort: medium
|
||||
|
||||
+4
-11
@@ -1,17 +1,10 @@
|
||||
# Host-specific settings, imported by common.nix from /etc/hermenix/local.nix
|
||||
# on the HOST (not from this repo - it lives outside the flake's own source
|
||||
# tree on purpose, since this flake is meant to be built from its git+ssh URL
|
||||
# with no local checkout at all).
|
||||
#
|
||||
# `hermenix setup` writes /etc/hermenix/local.nix for you. This file is just
|
||||
# the reference for its shape.
|
||||
# Written to /etc/hermenix/local.nix on the host (outside the flake; --impure).
|
||||
# `hermenix setup` creates it. See docs/INSTALL.md.
|
||||
{
|
||||
# Output of `cat /var/lib/secrets/hermes-cache.pub` on the host, once
|
||||
# generated. Leave empty (or omit the file entirely) and the guest just
|
||||
# skips the host cache, falling back to cache.nixos.org.
|
||||
# cat /var/lib/secrets/hermes-cache.pub — empty skips the host cache.
|
||||
cachePublicKey = ""; # e.g. "hermes-cache-1:AbCd..."
|
||||
|
||||
# Your SSH public key(s), for `hermes-vm ssh`.
|
||||
# SSH keys for hermenix vm ssh.
|
||||
authorizedKeys = [
|
||||
# "ssh-ed25519 AAAA... you@laptop"
|
||||
];
|
||||
|
||||
+35
-199
@@ -19,14 +19,6 @@ let
|
||||
vmUser = "hermes-vm";
|
||||
runDir = "/run/hermes-vm";
|
||||
stateDir = "/var/lib/hermes-vm";
|
||||
# Two disks, deliberately.
|
||||
#
|
||||
# The system disk is large, churny and entirely rebuildable - it is the
|
||||
# NixOS install and the Nix store. We never snapshot it.
|
||||
#
|
||||
# The state disk holds the only things that cannot be reproduced: Hermes's
|
||||
# identity and memories, its workspace, and the NixOS config it has written
|
||||
# for itself. Small, so snapshots are cheap and backups are quick.
|
||||
systemImage = "${cfg.disk.baseDir}/system/system.raw";
|
||||
stateImage = "${cfg.disk.baseDir}/state/state.raw";
|
||||
snapDir = "${cfg.disk.baseDir}/snapshots";
|
||||
@@ -36,25 +28,15 @@ let
|
||||
anyUsb = pt.usb != [ ];
|
||||
earlyIds = lib.filter (i: i != null) (map (d: d.id) pt.pci);
|
||||
|
||||
# "auto" adds a USB controller only when something needs plugging into it;
|
||||
# "on"/"off" force the decision.
|
||||
usbControllerOn =
|
||||
cfg.devices.usbController == "on" || (cfg.devices.usbController == "auto" && anyUsb);
|
||||
|
||||
# swtpm state. Must live under /var/lib/ so the unit's StateDirectory owns it.
|
||||
tpmStateDir = "/var/lib/hermes-vm-tpm";
|
||||
|
||||
# Netfilter mark tagging guest-originated traffic in bridged mode. The
|
||||
# guest's only way in is the tap interface, so marking at the bridge
|
||||
# prerouting hook identifies the guest regardless of which address it
|
||||
# currently presents - its static service-net address, or the fixed LAN
|
||||
# address served to it by the host-side DHCP on the bridge. 18501 ==
|
||||
# 0x4845 ("HE"), for hermenix.
|
||||
# 18501 == 0x4845 ("HE"); marks guest traffic from the tap (bridged mode).
|
||||
bridgeMark = 18501;
|
||||
|
||||
# Dotted-quad netmask for a prefix length, used by the bridged-mode DHCP
|
||||
# server's single-address lease (dnsmasq needs an explicit mask for a
|
||||
# one-address range).
|
||||
# dnsmasq needs an explicit mask for a one-address DHCP range.
|
||||
netmaskFromPrefix =
|
||||
p:
|
||||
lib.concatStringsSep "." (
|
||||
@@ -76,15 +58,12 @@ let
|
||||
) 4
|
||||
);
|
||||
|
||||
# Graceful ACPI power button via QMP, then wait for QEMU to exit.
|
||||
stopScript = pkgs.writeShellScript "hermes-vm-stop" ''
|
||||
set -eu
|
||||
printf '%s' '{"execute":"qmp_capabilities"}{"execute":"system_powerdown"}' \
|
||||
| ${pkgs.socat}/bin/socat - UNIX-CONNECT:${runDir}/qmp.sock >/dev/null 2>&1 || exit 0
|
||||
'';
|
||||
|
||||
# Freeze the guest filesystems around a snapshot so it captures a consistent
|
||||
# state rather than the equivalent of pulling the power.
|
||||
qga =
|
||||
verb:
|
||||
pkgs.writeShellScript "hermes-vm-fs${verb}" ''
|
||||
@@ -419,13 +398,14 @@ in
|
||||
NAT. The bridge absorbs `externalInterface`, so any device on the
|
||||
LAN can reach any port the guest exposes.
|
||||
|
||||
Addressing is fully static - no DHCP is involved. You must give
|
||||
the guest a fixed LAN address (`lanAddress`), the host's own LAN
|
||||
address (`hostLanAddress` - it moves onto the bridge), and the
|
||||
LAN gateway (`lanGateway`). The guest is handed `lanAddress` by a
|
||||
DHCP server this module runs on the bridge, keyed to the guest's
|
||||
MAC (`network.mac`), so the guest itself stays a plain DHCP
|
||||
client and never needs to know its own address.
|
||||
Addressing is static from the operator's point of view: you pick
|
||||
fixed addresses for the guest (`lanAddress`), this host
|
||||
(`hostLanAddress` — it moves onto the bridge), and the LAN
|
||||
gateway (`lanGateway`). A DHCP server this module runs on the
|
||||
bridge hands `lanAddress` to the guest, keyed to `network.mac`,
|
||||
so the guest stays a plain DHCP client and never needs to know
|
||||
its own address. That lease never leaves the bridge; the LAN's
|
||||
own DHCP cannot race it.
|
||||
|
||||
Inbound is no longer limited to `publish` (which is ignored in
|
||||
this mode): the LAN may reach every guest port. Outbound is still
|
||||
@@ -761,18 +741,20 @@ in
|
||||
|
||||
hermesHome = mkOption {
|
||||
type = types.str;
|
||||
default = "${stateDir}/.hermes";
|
||||
# Guest path (not host disk.baseDir).
|
||||
default = "/var/lib/hermes/.hermes";
|
||||
description = ''
|
||||
Where Hermes's own data (SOUL.md, config.yaml, memories/, sessions/,
|
||||
state.db, and so on) lives inside the guest. Defaults to the seeded
|
||||
layout under the state disk mount. Override this when carrying over
|
||||
an existing flat HERMES_HOME from elsewhere (e.g. a prior Docker
|
||||
deployment) that you're placing directly at the root of the state
|
||||
disk instead of under .hermes - see `hermenix migrate --help`.
|
||||
layout under the state disk mount (`/var/lib/hermes/.hermes`).
|
||||
Override when carrying over an existing flat HERMES_HOME from
|
||||
elsewhere (e.g. a prior Docker deployment) that you're placing
|
||||
directly at the root of the state disk instead of under .hermes —
|
||||
see `hermenix migrate --help`.
|
||||
|
||||
This only changes where the host-side `hermenix ctl` commands (soul,
|
||||
memory, conf, secrets, reseed, pull) looks. It does not change what
|
||||
the gateway process itself uses - export HERMES_HOME to match inside
|
||||
memory, conf, secrets, reseed, pull) look. It does not change what
|
||||
the gateway process itself uses — export HERMES_HOME to match inside
|
||||
the guest when you install it.
|
||||
'';
|
||||
};
|
||||
@@ -856,12 +838,6 @@ in
|
||||
};
|
||||
users.groups.${vmUser} = { };
|
||||
|
||||
# =====================================================================
|
||||
# The VM itself
|
||||
#
|
||||
# No virtiofs, no shared filesystem of any kind, no vhost-net. The guest
|
||||
# gets a block device, a tap, a vsock and a serial port. That is all.
|
||||
# =====================================================================
|
||||
systemd.services.hermes-vm = {
|
||||
description = "Hermes Agent VM";
|
||||
wantedBy = [ "multi-user.target" ];
|
||||
@@ -883,7 +859,6 @@ in
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
# Per-VM writable UEFI variable store, seeded once.
|
||||
if [ ! -e ${stateDir}/OVMF_VARS.fd ]; then
|
||||
install -m 0600 ${pkgs.OVMF.variables} ${stateDir}/OVMF_VARS.fd
|
||||
fi
|
||||
@@ -894,8 +869,6 @@ in
|
||||
Group = vmUser;
|
||||
SupplementaryGroups = [ "kvm" ];
|
||||
RuntimeDirectory = "hermes-vm";
|
||||
# ProtectSystem=strict makes everything read-only; punch through for
|
||||
# the images and the UEFI variable store only.
|
||||
ReadWritePaths = [ cfg.disk.baseDir ];
|
||||
StateDirectory = "hermes-vm";
|
||||
RuntimeDirectoryMode = "0750";
|
||||
@@ -910,32 +883,21 @@ in
|
||||
"-m ${toString cfg.mem}"
|
||||
"-nodefaults"
|
||||
"-no-user-config"
|
||||
# Display Settings
|
||||
"-display none"
|
||||
"-serial stdio"
|
||||
# QEMU's own seccomp sandbox, everything denied that we don't need.
|
||||
"-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny"
|
||||
# UEFI: firmware read-only, variables per-VM and writable.
|
||||
"-drive if=pflash,format=raw,unit=0,readonly=on,file=${pkgs.OVMF.firmware}"
|
||||
"-drive if=pflash,format=raw,unit=1,file=${stateDir}/OVMF_VARS.fd"
|
||||
# System disk: NixOS and the Nix store. Rebuildable, never snapshotted.
|
||||
# aio=threads and a writeback cache because these are files on a
|
||||
# filesystem, not block devices; aio=native wants the latter.
|
||||
# aio=threads+writeback: images are files, not blockdevs.
|
||||
"-drive if=none,id=system,format=raw,file=${systemImage},cache=writeback,aio=threads,discard=unmap"
|
||||
"-device virtio-blk-pci,drive=system"
|
||||
# State disk: everything irreplaceable. Mounted at /var/lib/hermes.
|
||||
"-drive if=none,id=state,format=raw,file=${stateImage},cache=writeback,aio=threads,discard=unmap"
|
||||
"-device virtio-blk-pci,drive=state"
|
||||
# Pre-created tap. vhost=off on purpose: vhost-net would move packet
|
||||
# handling into the host kernel, which is exactly the surface we are
|
||||
# trying not to expose. Costs throughput we do not need.
|
||||
# vhost=off: keep packet handling out of the host kernel.
|
||||
"-netdev tap,id=net0,ifname=${net.tapId},script=no,downscript=no,vhost=off"
|
||||
"-device virtio-net-pci,netdev=net0,mac=${net.mac}"
|
||||
# Host -> guest management, no network involved.
|
||||
"-qmp unix:${runDir}/qmp.sock,server=on,wait=off"
|
||||
]
|
||||
# Optional devices. rng and vsock are on by default; everything else
|
||||
# is off unless enabled via services.hermenix.devices.
|
||||
++ lib.optionals cfg.devices.rng.enable [
|
||||
"-device virtio-rng-pci"
|
||||
]
|
||||
@@ -994,11 +956,8 @@ in
|
||||
"-device virtio-serial-pci"
|
||||
"-device virtserialport,chardev=qga0,name=org.qemu.guest_agent.0"
|
||||
]
|
||||
# Passed-through PCI devices. The host cannot use these while the VM
|
||||
# is running.
|
||||
++ map (d: "-device vfio-pci,host=${d.address}") pt.pci
|
||||
++ map (u: "-device usb-host,vendorid=0x${u.vendorId},productid=0x${u.productId}") pt.usb
|
||||
# Raw escape hatch. Unvalidated - prefer the typed devices.* options.
|
||||
++ cfg.extraQemuArgs
|
||||
);
|
||||
|
||||
@@ -1007,7 +966,6 @@ in
|
||||
Restart = "always";
|
||||
RestartSec = 5;
|
||||
|
||||
# ---- hardening -------------------------------------------------
|
||||
DevicePolicy = "closed";
|
||||
DeviceAllow = [
|
||||
"/dev/kvm rw"
|
||||
@@ -1017,8 +975,7 @@ in
|
||||
++ lib.optionals anyPci [ "char-vfio rw" ]
|
||||
++ lib.optionals anyUsb [ "char-usb_device rw" ];
|
||||
|
||||
# VFIO pins the entire guest RAM allocation, so the default memlock
|
||||
# limit has to go or QEMU fails to start with an obscure error.
|
||||
# VFIO pins guest RAM; default memlock is too low.
|
||||
LimitMEMLOCK = lib.mkIf anyPci "infinity";
|
||||
NoNewPrivileges = true;
|
||||
ProtectSystem = "strict";
|
||||
@@ -1047,16 +1004,13 @@ in
|
||||
"~@resources"
|
||||
"~@obsolete"
|
||||
];
|
||||
# KVM means no JIT, so W^X is safe. If QEMU refuses to start, this is
|
||||
# the first thing to flip.
|
||||
# Safe under KVM (no JIT). First flip if QEMU won't start.
|
||||
MemoryDenyWriteExecute = true;
|
||||
CapabilityBoundingSet = [ "" ];
|
||||
AmbientCapabilities = [ "" ];
|
||||
};
|
||||
};
|
||||
|
||||
# TPM 2.0 emulator (swtpm), started before the VM. QEMU connects to its
|
||||
# control socket in the runtime dir; state persists under its StateDirectory.
|
||||
systemd.services.hermes-vm-swtpm = mkIf cfg.devices.tpm.enable {
|
||||
description = "swtpm TPM emulator for the Hermes VM";
|
||||
wantedBy = [ "multi-user.target" ];
|
||||
@@ -1104,7 +1058,6 @@ in
|
||||
'';
|
||||
};
|
||||
|
||||
# Tap device, created as root, owned by the VM user, bridged by networkd.
|
||||
systemd.services.hermes-vm-tap = {
|
||||
description = "TAP interface for the Hermes VM";
|
||||
after = [ "systemd-networkd.service" ];
|
||||
@@ -1123,12 +1076,6 @@ in
|
||||
'';
|
||||
};
|
||||
|
||||
# =====================================================================
|
||||
# Device passthrough
|
||||
# =====================================================================
|
||||
|
||||
# The IOMMU is what keeps a passed-through device from reading host
|
||||
# memory. Without it, PCI passthrough is not a boundary at all.
|
||||
boot.kernelParams = lib.optionals (pt.iommu != null) [
|
||||
(if pt.iommu == "intel" then "intel_iommu=on" else "amd_iommu=on")
|
||||
"iommu=pt"
|
||||
@@ -1140,12 +1087,10 @@ in
|
||||
"vfio_iommu_type1"
|
||||
];
|
||||
|
||||
# Claim the device before any host driver gets to it.
|
||||
boot.extraModprobeConfig = lib.optionalString (earlyIds != [ ]) ''
|
||||
options vfio-pci ids=${lib.concatStringsSep "," earlyIds}
|
||||
'';
|
||||
|
||||
# Catch anything that was not early-bound, by address. Idempotent.
|
||||
systemd.services.hermes-vm-vfio = mkIf anyPci {
|
||||
description = "Bind passthrough PCI devices to vfio-pci";
|
||||
before = [ "hermes-vm.service" ];
|
||||
@@ -1182,9 +1127,6 @@ in
|
||||
SUBSYSTEM=="usb", ATTR{idVendor}=="${u.vendorId}", ATTR{idProduct}=="${u.productId}", OWNER="${vmUser}", MODE="0660"
|
||||
'') pt.usb;
|
||||
|
||||
# =====================================================================
|
||||
# Networking
|
||||
# =====================================================================
|
||||
networking.useNetworkd = true;
|
||||
|
||||
systemd.network.netdevs."10-${net.bridge}".netdevConfig = {
|
||||
@@ -1194,10 +1136,7 @@ in
|
||||
|
||||
systemd.network.networks."10-${net.bridge}" = {
|
||||
matchConfig.Name = net.bridge;
|
||||
# hostAddress stays on the bridge in both modes: it is the guest's
|
||||
# service net (host cache, hostAccess). In bridged mode the host's
|
||||
# static LAN address is added on top of it - the bridge only ever has
|
||||
# static addresses, never a DHCP lease.
|
||||
# hostAddress = service net; bridged also adds hostLanAddress (static only).
|
||||
address = [
|
||||
"${net.hostAddress}/${toString net.prefixLength}"
|
||||
]
|
||||
@@ -1207,16 +1146,12 @@ in
|
||||
IPv4Forwarding = true;
|
||||
}
|
||||
// lib.optionalAttrs net.bridged {
|
||||
# Static LAN config for the host: the bridge no longer takes a DHCP
|
||||
# lease (bridged mode is static-only), so its gateway and DNS come
|
||||
# from the module options instead.
|
||||
Gateway = net.lanGateway;
|
||||
DNS = if net.lanDns == [ ] then [ net.lanGateway ] else net.lanDns;
|
||||
};
|
||||
};
|
||||
|
||||
# The host's DNS, now that the bridge cannot learn it from a DHCP lease.
|
||||
# mkDefault so the operator's own nameservers win if they set any.
|
||||
# mkDefault: operator nameservers win if set.
|
||||
networking.nameservers = lib.mkIf net.bridged (
|
||||
lib.mkDefault (if net.lanDns == [ ] then [ net.lanGateway ] else net.lanDns)
|
||||
);
|
||||
@@ -1226,21 +1161,12 @@ in
|
||||
networkConfig.Bridge = net.bridge;
|
||||
};
|
||||
|
||||
# Bridged mode: the physical uplink becomes a bridge port so the LAN
|
||||
# shares an L2 domain with the guest. The port must not carry its own
|
||||
# address config - the bridge owns the LAN address now (static, via
|
||||
# hostLanAddress; there is no DHCP on the bridge).
|
||||
systemd.network.networks."12-${net.externalInterface}" = lib.mkIf net.bridged {
|
||||
matchConfig.Name = net.externalInterface;
|
||||
networkConfig.Bridge = net.bridge;
|
||||
};
|
||||
|
||||
# The bridge is created with no MAC of its own, so Linux makes it adopt
|
||||
# the lowest-numbered port's MAC - and the tap is always the first port.
|
||||
# That is how the host once ended up presenting the VM's MAC on the LAN
|
||||
# (a duplicate address, since the guest uses the same MAC). Clone the
|
||||
# uplink's MAC onto the bridge so the host keeps its LAN identity; the
|
||||
# guest keeps its own tap MAC (net.mac).
|
||||
# Bridge would inherit tap MAC (first port) and collide with guest — clone uplink MAC.
|
||||
systemd.services.hermes-bridge-mac = lib.mkIf net.bridged {
|
||||
description = "Clone the uplink MAC onto the hermenix bridge";
|
||||
after = [
|
||||
@@ -1258,7 +1184,6 @@ in
|
||||
unitConfig.ConditionPathExists = "/sys/class/net/${net.externalInterface}";
|
||||
script = ''
|
||||
set -euo pipefail
|
||||
# networkd creates the bridge netdev asynchronously; give it a moment.
|
||||
for _ in 1 2 3 4 5 6 7 8 9 10; do
|
||||
[ -e "/sys/class/net/${net.bridge}" ] && break
|
||||
sleep 0.5
|
||||
@@ -1268,17 +1193,11 @@ in
|
||||
${pkgs.iproute2}/bin/ip link set dev ${net.bridge} down
|
||||
${pkgs.iproute2}/bin/ip link set dev ${net.bridge} address "$mac"
|
||||
${pkgs.iproute2}/bin/ip link set dev ${net.bridge} up
|
||||
# Re-apply the bridge's static address config under the new MAC.
|
||||
${pkgs.systemd}/bin/networkctl reconfigure ${net.bridge} || true
|
||||
'';
|
||||
};
|
||||
|
||||
# The guest is handed its fixed LAN address (lanAddress) by a dedicated
|
||||
# DHCP server bound to the bridge, keyed to the guest's MAC. This is the
|
||||
# guest's ONLY DHCP: its broadcast is blocked from leaving the bridge
|
||||
# (see the hermenix-bridge table), so the LAN's own DHCP server can
|
||||
# never race the reservation. DNS is disabled (port=0) - the guest runs
|
||||
# static QUAD9 resolvers, and this host gets its DNS from lanDns.
|
||||
# Single-address DHCP on the bridge (MAC-keyed). Broadcasts stay on-bridge.
|
||||
systemd.services.hermes-lan-dhcp = lib.mkIf net.bridged {
|
||||
description = "Static-lease DHCP for the bridged hermenix guest";
|
||||
after = [
|
||||
@@ -1309,34 +1228,16 @@ in
|
||||
'';
|
||||
};
|
||||
|
||||
# ip_forward: use the same sysctl keys networking.nat used (mkOverride 99
|
||||
# so they win over any lower-priority defaults elsewhere in the config).
|
||||
# mkOverride 99: same keys as networking.nat, win over weaker defaults.
|
||||
boot.kernel.sysctl."net.ipv4.conf.all.forwarding" = lib.mkOverride 99 true;
|
||||
boot.kernel.sysctl."net.ipv4.conf.default.forwarding" = lib.mkOverride 99 true;
|
||||
|
||||
# Masquerade outbound guest traffic and DNAT published ports inbound.
|
||||
# We use an explicit nftables table instead of networking.nat so that the
|
||||
# rules are certain to appear in the prerouting/postrouting hooks.
|
||||
#
|
||||
# NOTE: DNAT uses 'iifname != bridge' rather than 'iifname externalInterface'.
|
||||
# Scoping to a single interface means traffic arriving on any other path
|
||||
# (Tailscale, a second NIC, WiFi, loopback) silently skips DNAT: the
|
||||
# packet stays addressed to the host's own IP, falls into the INPUT chain,
|
||||
# and gets dropped because those ports are not in allowedTCPPorts.
|
||||
# Excluding only the bridge itself prevents loops from inside the guest
|
||||
# while covering every legitimate external path.
|
||||
# Explicit nft nat (not networking.nat). DNAT iifname != bridge so extra paths still match.
|
||||
networking.nftables.tables.hermenix-nat = lib.mkIf (net.externalInterface != null) {
|
||||
family = "ip";
|
||||
content =
|
||||
if net.bridged then
|
||||
# Bridged mode: no DNAT (the guest is directly reachable on the
|
||||
# LAN) and no primary masquerade. Keep one scoped masquerade as a
|
||||
# fallback: if the guest's DHCP lease lapses it falls back to its
|
||||
# static default route via the host, and this keeps that path
|
||||
# working instead of blackholing at the uplink. With a healthy
|
||||
# lease the guest's traffic is L2-bridged and never reaches this
|
||||
# chain. Only traffic handed up from the bridge (i.e. the guest's
|
||||
# service-net traffic) is matched, never the LAN itself.
|
||||
# No DNAT when bridged. Scoped masquerade covers service-net fallback if DHCP lapses.
|
||||
''
|
||||
chain postrouting {
|
||||
type nat hook postrouting priority srcnat;
|
||||
@@ -1360,34 +1261,19 @@ in
|
||||
'';
|
||||
};
|
||||
|
||||
# =====================================================================
|
||||
# Bridged mode: bridge-family enforcement.
|
||||
#
|
||||
# With the uplink bridged, guest <-> LAN traffic is switched at L2 and
|
||||
# never traverses the ip-family hooks the NAT-mode rules rely on, so
|
||||
# bridged mode gets its own table where the tap port is unambiguous.
|
||||
# lanAccess / allowLan still gate what the guest may reach; inbound
|
||||
# from the LAN to the guest is open on every port.
|
||||
# =====================================================================
|
||||
networking.nftables.tables."hermenix-bridge" = lib.mkIf net.bridged {
|
||||
family = "bridge";
|
||||
content = ''
|
||||
chain prerouting {
|
||||
type filter hook prerouting priority filter; policy accept;
|
||||
# Tag anything entering from the guest. ip-family input rules
|
||||
# (host cache, hostAccess) match this mark.
|
||||
iifname "${net.tapId}" meta mark set ${toString bridgeMark}
|
||||
|
||||
# The guest's address is static - served by hermes-lan-dhcp on
|
||||
# this bridge - so keep its DHCP traffic off the LAN and keep the
|
||||
# LAN's DHCP answers out of the guest. Only the host-side server
|
||||
# may talk DHCP to it.
|
||||
# Guest DHCP stays on-bridge only (no race with LAN DHCP).
|
||||
iifname "${net.tapId}" oifname "${net.externalInterface}" ether type ip ip protocol udp udp sport 68 udp dport 67 drop
|
||||
iifname "${net.externalInterface}" oifname "${net.tapId}" ether type ip ip protocol udp udp sport 67 udp dport 68 drop
|
||||
}
|
||||
chain forward {
|
||||
type filter hook forward priority filter; policy accept;
|
||||
# Inbound from the LAN to the guest: any device, any port.
|
||||
oifname "${net.tapId}" accept
|
||||
''
|
||||
+ lib.concatMapStrings (s: ''
|
||||
@@ -1417,26 +1303,11 @@ in
|
||||
'';
|
||||
};
|
||||
|
||||
# =====================================================================
|
||||
# Firewall
|
||||
#
|
||||
# The guest owns its own kernel and its own config now, so nothing inside
|
||||
# it constrains it. These rules are the boundary.
|
||||
# =====================================================================
|
||||
networking.nftables.enable = true;
|
||||
networking.firewall.enable = true;
|
||||
networking.firewall.filterForward = true;
|
||||
|
||||
# The deliberate guest -> host holes: the Nix cache, plus anything listed
|
||||
# in hostAccess.
|
||||
#
|
||||
# NAT mode: the guest always arrives from its fixed service-net address,
|
||||
# so match address + bridge. Bridged mode: the guest presents either the
|
||||
# service-net address or its LAN address (served by the bridge's DHCP),
|
||||
# so the bridge table above tags guest traffic with a netfilter mark as
|
||||
# it enters the tap - the rules match that mark instead. The tap is
|
||||
# exclusive to the guest, so the mark is as specific as an address
|
||||
# match: LAN devices can never inherit these holes.
|
||||
# Guest→host holes. NAT: guestAddress. Bridged: tap mark (guest may use LAN IP).
|
||||
networking.firewall.extraInputRules =
|
||||
if net.bridged then
|
||||
optionalString cfg.hostCache.enable ''
|
||||
@@ -1453,21 +1324,14 @@ in
|
||||
iifname "${net.bridge}" ip saddr ${net.guestAddress} ${s.proto} dport ${toString s.port} accept
|
||||
'') cfg.hostAccess;
|
||||
|
||||
# Order matters here. Allowlists must precede the RFC1918 drop.
|
||||
#
|
||||
# These ip-family rules only ever see routed traffic. They are the whole
|
||||
# story in NAT mode; in bridged mode the guest's LAN traffic is
|
||||
# L2-bridged instead and is governed by the hermenix-bridge table.
|
||||
# Allowlists before RFC1918 drop. Bridged LAN is L2 (hermenix-bridge table).
|
||||
networking.firewall.extraForwardRules = ''
|
||||
iifname != "${net.bridge}" accept
|
||||
ct state established,related accept
|
||||
''
|
||||
# Guest reaching named services elsewhere on your network.
|
||||
+ lib.concatMapStrings (s: ''
|
||||
iifname "${net.bridge}" ip daddr ${s.address} ${s.proto} dport ${toString s.port} accept
|
||||
'') cfg.lanAccess
|
||||
# Inbound to guest services published on the uplink. NAT mode only; in
|
||||
# bridged mode the bridge table opens every inbound port instead.
|
||||
+ lib.optionalString (!net.bridged) (
|
||||
lib.concatMapStrings (s: ''
|
||||
oifname "${net.bridge}" ip daddr ${net.guestAddress} ${s.proto} dport ${toString s.guestPort} accept
|
||||
@@ -1493,24 +1357,14 @@ in
|
||||
''
|
||||
);
|
||||
|
||||
# =====================================================================
|
||||
# Host store as a read-only cache for the guest
|
||||
# =====================================================================
|
||||
services.nix-serve = mkIf cfg.hostCache.enable {
|
||||
enable = true;
|
||||
package = pkgs.nix-serve-ng;
|
||||
bindAddress = net.hostAddress;
|
||||
inherit (cfg.hostCache) port secretKeyFile;
|
||||
openFirewall = false; # handled above, scoped to the guest address
|
||||
openFirewall = false;
|
||||
};
|
||||
|
||||
# =====================================================================
|
||||
# Snapshots of the state disk only.
|
||||
#
|
||||
# The system disk is not snapshotted on purpose. It is large, it is where
|
||||
# all the write churn happens, and `hermenix setup` can rebuild it from the flake
|
||||
# in minutes. Snapshotting it would cost a lot and buy nothing.
|
||||
# =====================================================================
|
||||
systemd.services.hermes-vm-snapshot = mkIf cfg.snapshots.enable {
|
||||
description = "Snapshot the Hermes VM state disk";
|
||||
path = with pkgs; [
|
||||
@@ -1525,9 +1379,6 @@ in
|
||||
mkdir -p ${snapDir}
|
||||
|
||||
${optionalString cfg.disk.quiesce ''
|
||||
# Flush the guest's filesystems so the snapshot is consistent rather
|
||||
# than the equivalent of pulling the power. Takes milliseconds; the
|
||||
# VM keeps running throughout.
|
||||
${qga "freeze"}
|
||||
trap '${qga "thaw"}' EXIT
|
||||
''}
|
||||
@@ -1539,8 +1390,7 @@ in
|
||||
trap - EXIT
|
||||
''}
|
||||
|
||||
# Prune oldest first. Keep this list short: every live snapshot
|
||||
# reintroduces copy-on-write on a nodatacow image.
|
||||
# Live snapshots reintroduce CoW on nodatacow images — keep retention tight.
|
||||
ls -1d ${snapDir}/state-* 2>/dev/null | sort | head -n -${toString cfg.snapshots.keep} \
|
||||
| while read -r old; do btrfs subvolume delete "$old"; done
|
||||
'';
|
||||
@@ -1555,15 +1405,6 @@ in
|
||||
};
|
||||
};
|
||||
|
||||
# =====================================================================
|
||||
# Backups: the guest's state, not its disk.
|
||||
#
|
||||
# Pulled file-by-file over the SSH channel that already exists, rather
|
||||
# than backing up the image. That keeps the backup portable - it restores
|
||||
# onto any filesystem, or a different machine entirely - and avoids
|
||||
# loop-mounting a guest-written filesystem on the host, which would mean
|
||||
# the host kernel parsing data the agent controls.
|
||||
# =====================================================================
|
||||
systemd.services.hermes-vm-backup = mkIf cfg.backup.enable {
|
||||
description = "Back up Hermes state";
|
||||
path = with pkgs; [
|
||||
@@ -1608,17 +1449,12 @@ in
|
||||
};
|
||||
|
||||
environment.systemPackages = [
|
||||
# Nushell tab-completion definitions for hermenix. Installed to
|
||||
# vendor/autoload so nushell picks them up automatically.
|
||||
(pkgs.writeTextFile {
|
||||
name = "hermenix-completions";
|
||||
destination = "/share/nushell/vendor/autoload/hermenix.nu";
|
||||
text = builtins.readFile ../nu/completions.nu;
|
||||
})
|
||||
|
||||
# Single consolidated CLI: `hermenix vm *` controls the VM,
|
||||
# `hermenix ctl *` drives Hermes inside it. Per-host values are
|
||||
# added on top of the base package via makeWrapper.
|
||||
(pkgs.symlinkJoin {
|
||||
name = "hermenix";
|
||||
paths = [ pkgs.hermenix ];
|
||||
|
||||
+3
-23
@@ -1,12 +1,5 @@
|
||||
# hermenix ctl — drive Hermes inside the guest.
|
||||
#
|
||||
# Identity, config, secrets, gateway service, and a passthrough to the
|
||||
# hermes CLI itself. For controlling the VM around it, see vm/vm.nu.
|
||||
#
|
||||
# Depends on: lib/common.nu, lib/guest.nu (ssh-args).
|
||||
|
||||
def home-dir [] {
|
||||
$env.HERMES_HOME_DIR? | default "/var/lib/hermes-vm/.hermes"
|
||||
$env.HERMES_HOME_DIR? | default "/var/lib/hermes/.hermes"
|
||||
}
|
||||
def gateway-unit [] {
|
||||
$env.HERMES_GATEWAY_UNIT? | default "hermes-agent"
|
||||
@@ -14,8 +7,6 @@ def gateway-unit [] {
|
||||
|
||||
def ctl-gssh [cmd: string] { run-argv ((ssh-args) ++ [$cmd]) }
|
||||
|
||||
# Pull a file out of the guest, open it in $EDITOR, push it back.
|
||||
# Skips the write if nothing changed (hash comparison).
|
||||
def edit-remote [remote: string, mode: string = "0644"] {
|
||||
let tmp = (mktemp)
|
||||
let fetched = do -i { ctl-gssh $"cat ($remote)" } | complete
|
||||
@@ -38,13 +29,8 @@ def edit-remote [remote: string, mode: string = "0644"] {
|
||||
}
|
||||
}
|
||||
|
||||
# Everything not matched by a more-specific `main ctl *` goes straight to the
|
||||
# hermes CLI in the guest, with a pty for interactive subcommands like `chat`.
|
||||
#
|
||||
# A single string, not rest args: nu's parser tries to match any bare
|
||||
# --flag-shaped token against this command's own flags before it ever
|
||||
# reaches a rest parameter, so `hermenix ctl chat --model x` would fail to
|
||||
# parse. Quote the whole thing instead — e.g. `hermenix ctl "chat --model x"`.
|
||||
# Single string, not rest args: nu matches bare --flags on this command first.
|
||||
# Quote multi-word passthroughs: hermenix ctl "chat --model x"
|
||||
def "main ctl" [remainder: string = ""] {
|
||||
if $remainder == "" {
|
||||
print "usage: hermenix ctl <subcommand>"
|
||||
@@ -94,8 +80,6 @@ def "main ctl conf" [action?: string] {
|
||||
}
|
||||
}
|
||||
|
||||
# API keys. Mode 0600, never enters the Nix store, and not printed unless
|
||||
# you ask for it explicitly.
|
||||
def "main ctl secrets" [action?: string] {
|
||||
if $action == "edit" {
|
||||
edit-remote $"(home-dir)/.env" "0600"
|
||||
@@ -111,8 +95,6 @@ def "main ctl logs" [] { run-argv ((ssh-args) ++ ["-t" $"journalctl --user -u (g
|
||||
def "main ctl restart" [] { ctl-gssh $"systemctl --user restart (gateway-unit)" }
|
||||
def "main ctl units" [] { ctl-gssh "systemctl --user list-units 'hermes*' --all --no-pager" }
|
||||
|
||||
# Overwrite the agent's identity from the seed. Destructive — this is how
|
||||
# you undo whatever it has decided about itself.
|
||||
def "main ctl reseed" [] {
|
||||
warn "This overwrites SOUL.md, memories and config.yaml in the guest"
|
||||
warn "with the seed from /etc/nixos/hermes. Anything Hermes has"
|
||||
@@ -122,8 +104,6 @@ def "main ctl reseed" [] {
|
||||
ok "reseeded"
|
||||
}
|
||||
|
||||
# Copy the agent's live identity back here, so you can fold what it has
|
||||
# learned into the seed in guest/hermes/.
|
||||
def "main ctl pull" [dest: string = "./hermes-home"] {
|
||||
mkdir $dest
|
||||
(
|
||||
|
||||
@@ -1,29 +1,4 @@
|
||||
#!/usr/bin/env nu
|
||||
#
|
||||
# hermenix — host-side tooling for the Hermes Agent VM.
|
||||
#
|
||||
# No local checkout needed. `hermenix` is a flake package meant to be run
|
||||
# straight from its git+ssh URL:
|
||||
#
|
||||
# nix run <flake>#hermenix
|
||||
#
|
||||
# Or install once and just call it directly:
|
||||
#
|
||||
# nix profile install <flake>#hermenix (or via the nixos-setup overlay)
|
||||
#
|
||||
# Host-specific settings (cache signing key, SSH key) live at
|
||||
# /etc/hermenix/local.nix — see guest/local.nix.example.
|
||||
#
|
||||
# Source layout (all concatenated by flake.nix in dependency order):
|
||||
#
|
||||
# lib/common.nu output helpers, run-argv, vsock-ssh-args
|
||||
# lib/guest.nu shared guest-connection env helpers
|
||||
# host/setup.nu hermenix setup
|
||||
# host/migrate.nu hermenix migrate
|
||||
# host/remote.nu hermenix remote
|
||||
# vm/vm.nu hermenix vm *
|
||||
# ctl/ctl.nu hermenix ctl *
|
||||
# hermenix.nu this file — top-level dispatch (main)
|
||||
|
||||
def main [] {
|
||||
print "usage: hermenix <subcommand> ..."
|
||||
|
||||
+7
-67
@@ -1,23 +1,3 @@
|
||||
# hermenix migrate — carry an existing flat HERMES_HOME into the guest,
|
||||
# or selectively copy just the agent-connection files between two folder
|
||||
# deployments.
|
||||
#
|
||||
# Only ever reads from the source directory. Nothing there is deleted or
|
||||
# moved — the original stays a fully intact fallback.
|
||||
#
|
||||
# Run this AFTER setup + the host's nixos-rebuild have brought the guest up
|
||||
# at least once, and `hermenix vm ssh true` succeeds.
|
||||
#
|
||||
# Depends on: lib/common.nu, lib/guest.nu.
|
||||
|
||||
# Carry an existing flat HERMES_HOME (e.g. the nousresearch/hermes-agent
|
||||
# Docker image, HERMES_HOME bind-mounted from the host) into a hermenix
|
||||
# guest, preserving its exact layout.
|
||||
#
|
||||
# SAFETY: only ever reads from `source`. Nothing there is deleted or moved —
|
||||
# the original stays a fully intact fallback. If anything goes wrong after
|
||||
# this, `sudo nixos-rebuild switch --rollback` on the host restores the
|
||||
# previous deployment pointed at the same, untouched source data.
|
||||
def "main migrate" [
|
||||
source: string # existing flat HERMES_HOME to copy from (read-only)
|
||||
--vsock-cid: int = 77 # must match services.hermenix.vsockCid on the host
|
||||
@@ -51,22 +31,8 @@ def "main migrate" [
|
||||
gssh "true"
|
||||
|
||||
step 2 $"streaming ($source) -> guest:($dest) \(this can take a while for 8G+\)"
|
||||
# --ignore-failed-read: a source entry can be unreadable (e.g. uid-mapped
|
||||
# leftovers from a prior rootless-container deployment) without that
|
||||
# aborting an otherwise-successful migration - tar still warns about it,
|
||||
# it just doesn't fail the whole pipeline over it.
|
||||
#
|
||||
# --exclude='./.local/bin': launcher scripts in .local/bin are
|
||||
# environment-specific (Docker wrappers, old arch builds, etc.) and must
|
||||
# never be carried into the guest. hermes-install.service installs the
|
||||
# correct native binary there on first boot; an overwritten launcher from
|
||||
# the source would shadow it and fail with "docker not found on PATH".
|
||||
#
|
||||
# --exclude='./.hermes-installed': this sentinel tells hermes-install to
|
||||
# skip the native install. Migrating it from the source would prevent the
|
||||
# guest from ever installing hermes if the source's .local/bin/hermes is
|
||||
# also excluded (above). Let the install service run and place a fresh
|
||||
# binary instead.
|
||||
# ignore-failed-read: rootless-docker uid leftovers.
|
||||
# exclude .local/bin + .hermes-installed: don't shadow guest native install.
|
||||
^tar --ignore-failed-read --exclude='./.local/bin' --exclude='./.hermes-installed' -C $source -cf - . | run-argv ($ssh_base ++ [$"mkdir -p '($dest)' && tar -C '($dest)' -xf -"])
|
||||
|
||||
step 3 "rewriting stale hostnames in the guest"
|
||||
@@ -84,30 +50,13 @@ def "main migrate" [
|
||||
note "Tip: use `hermenix migrate connections <source> <dest>` to carry just"
|
||||
note "the connection config files between two folder deployments."
|
||||
print ""
|
||||
say "Next, inside the guest (hermenix vm ssh):"
|
||||
say $" export HERMES_HOME=($dest)"
|
||||
say " # hermes installs itself automatically on first boot - if that's"
|
||||
say " # already done (check: hermenix ctl units), just:"
|
||||
say " # determinate-nixd login"
|
||||
say " # sudo nix flake lock /etc/nixos"
|
||||
say " # hermes setup"
|
||||
say " # hermes gateway install"
|
||||
print ""
|
||||
say "Then verify with hermenix ctl status / logs before considering the old"
|
||||
say "deployment retired. Decrypt secrets/hermes-env.age yourself and fold"
|
||||
say $"anything relevant into ($dest)/.env \(hermenix ctl secrets edit\)."
|
||||
say "Next:"
|
||||
say $" - export HERMES_HOME=($dest) inside the guest (and set services.hermenix.hermesHome)"
|
||||
say " - finish Hermes configuration with the upstream CLI/docs"
|
||||
say " - fold secrets into the guest via: hermenix ctl secrets edit"
|
||||
say " - verify: hermenix ctl status / logs — only then retire the old deployment"
|
||||
}
|
||||
|
||||
# Copy only the agent-connection files from one flat HERMES_HOME to another.
|
||||
#
|
||||
# Use this when you want to carry the connections/integrations config across
|
||||
# deployments without overwriting identity (SOUL.md, memories, etc.).
|
||||
#
|
||||
# By default this is a pure local operation (host-to-host folder copy).
|
||||
# Pass --to-guest to stream the files into the running guest over vsock
|
||||
# instead, using the same path as `hermenix migrate`.
|
||||
#
|
||||
# SAFETY: only ever reads from `source`; the original is never modified.
|
||||
def "main migrate connections" [
|
||||
source: string # existing flat HERMES_HOME to copy from (read-only)
|
||||
dest: string = "" # destination HERMES_HOME folder; not required when --to-guest is set
|
||||
@@ -130,17 +79,13 @@ def "main migrate connections" [
|
||||
die "--rewrite-from and --rewrite-to must have the same length"
|
||||
}
|
||||
|
||||
# ---- resolve what we're actually copying --------------------------------
|
||||
# Flat files that exist in source.
|
||||
let existing_files = $files | where {|f| ($"($source)/($f)" | path exists) }
|
||||
# Subdirectories that exist in source.
|
||||
let existing_dirs = $extra_dirs | where {|d| ($"($source)/($d)" | path exists) }
|
||||
|
||||
if ($existing_files | is-empty) and ($existing_dirs | is-empty) {
|
||||
die $"none of the requested connection files/dirs were found under ($source)"
|
||||
}
|
||||
|
||||
# ---- build the sed expression for hostname rewrites --------------------
|
||||
let has_rewrites = ($rewrite_from | length) > 0
|
||||
let sed_expr = if $has_rewrites {
|
||||
let last_idx = ($rewrite_from | length) - 1
|
||||
@@ -149,7 +94,6 @@ def "main migrate connections" [
|
||||
| str join "; ")
|
||||
} else { "" }
|
||||
|
||||
# ---- announce -----------------------------------------------------------
|
||||
say $"Source \(read-only\): ($source)"
|
||||
if $to_guest {
|
||||
say $"Destination \(guest\): ($guest_user)@hermes:($guest_dest) \(over vsock, cid ($vsock_cid)\)"
|
||||
@@ -170,9 +114,7 @@ def "main migrate connections" [
|
||||
print ""
|
||||
if not (confirm-word "copy" "Type 'copy' to proceed:") { die "aborted" }
|
||||
|
||||
# =========================================================================
|
||||
if $to_guest {
|
||||
# ---- guest mode: stream over vsock ----------------------------------
|
||||
let ssh_base = (vsock-ssh-args $vsock_cid $guest_user "hermes")
|
||||
def gssh [cmd: string] { run-argv ($ssh_base ++ [$cmd]) }
|
||||
|
||||
@@ -180,7 +122,6 @@ def "main migrate connections" [
|
||||
gssh "true"
|
||||
|
||||
step 2 $"streaming connection files ($source) -> guest:($guest_dest)"
|
||||
# Pass files and dirs as positional tar arguments — no --include needed.
|
||||
let tar_targets = $existing_files | append $existing_dirs
|
||||
(
|
||||
^tar --ignore-failed-read -C $source -cf - ...$tar_targets
|
||||
@@ -198,7 +139,6 @@ def "main migrate connections" [
|
||||
}
|
||||
}
|
||||
} else {
|
||||
# ---- local mode: copy on this host ----------------------------------
|
||||
step 1 "creating destination directory"
|
||||
mkdir $dest
|
||||
|
||||
|
||||
+4
-21
@@ -1,26 +1,9 @@
|
||||
# hermenix remote — run setup or migrate on a remote host over SSH.
|
||||
#
|
||||
# No checkout involved anywhere: executes `nix run <flake>#hermenix -- ...`
|
||||
# directly on the remote, fetching straight from Gitea. The remote host needs
|
||||
# its own SSH access to Gitea already set up (same access as a git clone).
|
||||
#
|
||||
# Depends on: lib/common.nu (shquote, say, die).
|
||||
|
||||
# Run `hermenix setup` or `hermenix migrate` on a remote host over SSH,
|
||||
# instead of having to SSH in and run it locally yourself. `setup` and
|
||||
# `migrate` both act on whatever machine they run on (local disks, local
|
||||
# sudo, vsock into the local guest) — this just saves the manual SSH step.
|
||||
# cmd must be one string: nu treats bare --flags as belonging to `remote`.
|
||||
def "main remote" [
|
||||
host: string # SSH target, e.g. fwdesk.lab.internal
|
||||
cmd: string # subcommand + its own args, quoted as ONE string — e.g. "setup --clean" or "migrate /var/lib/hermes"
|
||||
--flake: string = "git+ssh://gitea@minipc1.lab.internal:22/ari/hermenix.git"
|
||||
host: string
|
||||
cmd: string # e.g. "setup --clean" or "migrate /var/lib/hermes"
|
||||
--flake: string = "git+ssh://gitea@gitea.skyg.ca/ari/hermenix.git"
|
||||
] {
|
||||
# `cmd` has to be a single string, not a rest param: nu's own CLI parser
|
||||
# treats any bare --flag-shaped token as an attempt to match a flag on
|
||||
# THIS command (main remote), even inside a rest parameter and even with
|
||||
# a `--` separator — so `hermenix remote host setup --clean` fails before
|
||||
# ever reaching this function. Splitting cmd back into words here (and
|
||||
# re-quoting each one) sidesteps that entirely.
|
||||
let words = $cmd | split row " " | where {|w| $w != "" }
|
||||
if ($words | is-empty) {
|
||||
die $'usage: hermenix remote <host> "<setup|migrate> [args...]"'
|
||||
|
||||
+9
-34
@@ -1,15 +1,4 @@
|
||||
# hermenix setup — guided host-side provisioning.
|
||||
#
|
||||
# Walks through every prerequisite check, disk creation, cache signing key,
|
||||
# local.nix, image build and image write. Safe to re-run; each step checks
|
||||
# what already exists. Never runs nixos-rebuild.
|
||||
#
|
||||
# Depends on: lib/common.nu, lib/guest.nu.
|
||||
|
||||
# `truncate -s` treats a bare number as bytes, not gigabytes — the size
|
||||
# prompts' own defaults ("80G", "20G") make gigabytes the obvious intent, so
|
||||
# normalize a bare integer answer to that rather than silently creating a
|
||||
# byte-sized disk. Anything with an existing suffix passes through as-is.
|
||||
# Bare `truncate -s N` is bytes; prompts default to gigabytes.
|
||||
def normalize-size [s: string] {
|
||||
try {
|
||||
let n = $s | into int
|
||||
@@ -19,28 +8,20 @@ def normalize-size [s: string] {
|
||||
}
|
||||
}
|
||||
|
||||
# Run a command, sudo-prefixed unless already root (set once via
|
||||
# $env.HERMENIX_NEED_SUDO at the top of `main setup`). Fails loudly, like any
|
||||
# other external command, if the command fails.
|
||||
def priv [...args: string] {
|
||||
let need_sudo = $env.HERMENIX_NEED_SUDO? | default false
|
||||
let argv = if $need_sudo { ["sudo"] ++ $args } else { $args }
|
||||
^($argv | first) ...($argv | skip 1)
|
||||
}
|
||||
|
||||
# Same as priv, but captures stdout/exit code instead of failing the script.
|
||||
def priv-out [...args: string] {
|
||||
let need_sudo = $env.HERMENIX_NEED_SUDO? | default false
|
||||
let argv = if $need_sudo { ["sudo"] ++ $args } else { $args }
|
||||
(^($argv | first) ...($argv | skip 1) | complete)
|
||||
}
|
||||
|
||||
# Guided host-side setup: disks, cache signing key, /etc/hermenix/local.nix,
|
||||
# build and write the bootstrap image. Safe to re-run — every step checks
|
||||
# what's already done and offers to skip. Never runs nixos-rebuild. Works
|
||||
# from any directory - no local checkout of this repo needed.
|
||||
def "main setup" [
|
||||
--flake: string = "git+ssh://gitea@minipc1.lab.internal:22/ari/hermenix.git"
|
||||
--flake: string = "git+ssh://gitea@gitea.skyg.ca/ari/hermenix.git"
|
||||
--clean # stop the VM and delete existing disk images + UEFI vars first, for a fully fresh start
|
||||
] {
|
||||
$env.HERMENIX_NEED_SUDO = ((^id -u | str trim) != "0")
|
||||
@@ -100,10 +81,7 @@ def "main setup" [
|
||||
do -i { priv rm "-f" $"($base)/system/system.raw" } | complete | ignore
|
||||
do -i { priv rm "-f" $"($base)/state/state.raw" } | complete | ignore
|
||||
do -i { priv rm "-f" $"($base)/OVMF_VARS.fd" } | complete | ignore
|
||||
# A fresh state disk means a fresh SSH host key on next boot - drop the
|
||||
# stale known_hosts entry now so the next connect doesn't fail on a
|
||||
# host-key mismatch. Not run through priv: this is the invoking user's
|
||||
# own ~/.ssh/known_hosts, not a root-owned path.
|
||||
# Fresh state disk => fresh SSH host key; drop stale known_hosts (user's, not root's).
|
||||
do -i { ^ssh-keygen -R hermes } | complete | ignore
|
||||
ok "cleaned existing disk images and stale known_hosts entry"
|
||||
}
|
||||
@@ -375,8 +353,7 @@ def "main setup" [
|
||||
\};
|
||||
"
|
||||
note $"detected uplink: ($uplink) — check that's right before rebuilding"
|
||||
note "network.bridged = true instead puts the guest on your LAN (own DHCP"
|
||||
note "address, every opened port reachable from the LAN) - see the README."
|
||||
note "network.bridged = true puts the guest on your LAN — see docs/NETWORK.md."
|
||||
|
||||
print $"
|
||||
Then:
|
||||
@@ -385,20 +362,18 @@ def "main setup" [
|
||||
hermenix vm console (ansi dark_gray)# watch it boot(ansi reset)
|
||||
hermenix vm ssh (ansi dark_gray)# get in(ansi reset)
|
||||
|
||||
Inside the guest, in this order:
|
||||
Inside the guest (hermenix side):
|
||||
|
||||
lsblk -f (ansi dark_gray)# CHECK LABELS match guest/hardware.nix(ansi reset)
|
||||
determinate-nixd login (ansi dark_gray)# FlakeHub Cache(ansi reset)
|
||||
sudo nix flake lock /etc/nixos
|
||||
(ansi dark_gray)# hermes installs itself automatically - give it a few minutes(ansi reset)
|
||||
hermes setup
|
||||
hermes gateway install
|
||||
(ansi dark_gray)# hermes CLI installs unattended — give it a few minutes(ansi reset)
|
||||
|
||||
Back on the host:
|
||||
Configure Hermes itself with the upstream CLI/docs (providers, gateway, …).
|
||||
From the host, once the gateway unit exists:
|
||||
|
||||
hermenix ctl units (ansi dark_gray)# confirm the gateway unit name(ansi reset)
|
||||
hermenix ctl units
|
||||
hermenix ctl status
|
||||
hermenix ctl soul
|
||||
"
|
||||
|
||||
warn "Check the disk labels before the guest's first nixos-rebuild. If they"
|
||||
|
||||
+4
-28
@@ -1,13 +1,5 @@
|
||||
# hermenix — shared helpers for the host-side nushell tools (hermenix,
|
||||
# hermes-vm, hermes-ctl).
|
||||
#
|
||||
# Not a standalone script: nu can't reliably resolve relative `use` imports
|
||||
# once a script has been copied into the Nix store as a single wrapped
|
||||
# binary, so the flake concatenates this file with each tool's own script
|
||||
# instead (see flake.nix / modules/host.nix). Keep this file free of a
|
||||
# `main` def and of anything specific to one tool.
|
||||
|
||||
# ---------------------------------------------------------------- output ---
|
||||
# Shared helpers. Concatenated into the hermenix binary by flake.nix
|
||||
# (relative `use` is unreliable once the script is in the store).
|
||||
|
||||
def say [msg: string] { print $" ($msg)" }
|
||||
def note [msg: string] { print $" (ansi dark_gray)($msg)(ansi reset)" }
|
||||
@@ -19,11 +11,7 @@ def die [msg: string] {
|
||||
}
|
||||
def step [n: int, title: string] { print $"\n(ansi attr_bold)== ($n). ($title)(ansi reset)" }
|
||||
|
||||
# Read a line of user input. On a real terminal this prompts interactively
|
||||
# via `input`. When stdin isn't a terminal (e.g. piped in from a test or
|
||||
# script), `input` would otherwise try to read from the controlling terminal
|
||||
# and hang forever waiting for a keystroke that never comes — so read the
|
||||
# piped stdin directly instead.
|
||||
# `input` hangs when stdin is piped (tests); read the pipe instead.
|
||||
def read-line [prompt: string] {
|
||||
if (is-terminal --stdin) {
|
||||
input $prompt
|
||||
@@ -53,16 +41,10 @@ def confirm-word [word: string, prompt: string] {
|
||||
(read-line $" ($prompt) ") == $word
|
||||
}
|
||||
|
||||
# Single-quote a string for safe embedding in a remote shell command line.
|
||||
def shquote [s: string] {
|
||||
"'" + ($s | str replace --all "'" "'\\''") + "'"
|
||||
}
|
||||
|
||||
# ------------------------------------------------------------------- ssh ---
|
||||
|
||||
# argv for reaching the guest over vsock — no inbound network path needed.
|
||||
# Shared by every tool that talks to the guest (hermenix migrate, hermes-vm,
|
||||
# hermes-ctl).
|
||||
def vsock-ssh-args [vsock_cid: int, guest_user: string, guest_host: string] {
|
||||
[
|
||||
"ssh"
|
||||
@@ -72,17 +54,11 @@ def vsock-ssh-args [vsock_cid: int, guest_user: string, guest_host: string] {
|
||||
]
|
||||
}
|
||||
|
||||
# Run an argv list (e.g. from vsock-ssh-args, with trailing command words
|
||||
# appended) as an external command.
|
||||
def run-argv [argv: list<string>] {
|
||||
^($argv | first) ...($argv | skip 1)
|
||||
}
|
||||
|
||||
# --------------------------------------------------------------- version ---
|
||||
|
||||
# HERMENIX_VERSION is baked in at build time (flake.nix / modules/host.nix)
|
||||
# as "YYYY.MM.DD.hh.mm-<commit-sha>" from this repo's own git metadata. "dev"
|
||||
# is only seen running an unwrapped checkout of the script directly.
|
||||
# Baked in at package build time; "dev" only for an unwrapped checkout.
|
||||
def hermenix-version [] {
|
||||
$env.HERMENIX_VERSION? | default "dev"
|
||||
}
|
||||
|
||||
@@ -1,7 +1,3 @@
|
||||
# Shared guest-connection helpers.
|
||||
# Depends on: lib/common.nu (vsock-ssh-args).
|
||||
# Used by both vm/ and ctl/.
|
||||
|
||||
def guest-user [] { "hermes" }
|
||||
def guest-host [] {
|
||||
$env.HERMES_GUEST_NAME? | default "hermes"
|
||||
@@ -10,5 +6,4 @@ def vsock-cid [] {
|
||||
($env.HERMES_VSOCK_CID? | default "77") | into int
|
||||
}
|
||||
|
||||
# Full SSH argv for the guest, shared by `vm` and `ctl` subcommands.
|
||||
def ssh-args [] { vsock-ssh-args (vsock-cid) (guest-user) (guest-host) }
|
||||
@@ -5,9 +5,9 @@ source ../lib/guest.nu
|
||||
source ../ctl/ctl.nu
|
||||
|
||||
@test
|
||||
def "home-dir defaults to the vm state disk path" [] {
|
||||
def "home-dir defaults to the guest hermes home path" [] {
|
||||
with-env { HERMES_HOME_DIR: null } {
|
||||
assert equal (home-dir) "/var/lib/hermes-vm/.hermes"
|
||||
assert equal (home-dir) "/var/lib/hermes/.hermes"
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+1
-16
@@ -1,20 +1,9 @@
|
||||
# hermenix vm — control the Hermes Agent VM itself.
|
||||
#
|
||||
# Start/stop, console, config inspection, snapshots and backups.
|
||||
# For driving Hermes the agent running inside the VM, see ctl/ctl.nu.
|
||||
#
|
||||
# Depends on: lib/common.nu, lib/guest.nu (ssh-args).
|
||||
|
||||
def disk-base-dir [] {
|
||||
$env.HERMES_DISK_BASE_DIR? | default "/var/lib/hermes-vm"
|
||||
}
|
||||
def snap-dir [] { $"(disk-base-dir)/snapshots" }
|
||||
|
||||
# Run a single already-fully-formed remote command string, or open an
|
||||
# interactive shell if none is given. A single string, not rest args: nu's
|
||||
# parser tries to match any bare --flag-shaped token against this command's
|
||||
# own flags before it ever reaches a rest parameter, so `hermenix vm ssh sudo
|
||||
# foo --bar` would fail to parse. Quote the whole remote command instead.
|
||||
# Single string, not rest args: nu matches bare --flags on this command first.
|
||||
def vm-gssh [cmd?: string] {
|
||||
let argv = if $cmd == null { ssh-args } else { (ssh-args) ++ [$cmd] }
|
||||
run-argv $argv
|
||||
@@ -45,7 +34,6 @@ def "main vm start" [] { ^systemctl start hermes-vm }
|
||||
def "main vm stop" [] { ^systemctl stop hermes-vm }
|
||||
def "main vm restart" [] { ^systemctl restart hermes-vm }
|
||||
|
||||
# What is the guest actually running right now?
|
||||
def "main vm config" [] {
|
||||
vm-gssh 'echo "== current generation =="; \
|
||||
readlink -f /run/current-system; \
|
||||
@@ -57,7 +45,6 @@ def "main vm config" [] {
|
||||
git -C /etc/nixos status --short'
|
||||
}
|
||||
|
||||
# Copy the guest's live NixOS config here for reading or diffing.
|
||||
def "main vm config-pull" [dest: string = "./hermes-etc-nixos"] {
|
||||
mkdir $dest
|
||||
(vm-gssh "tar -C /etc/nixos -cf - .") | ^tar -C $dest -xf -
|
||||
@@ -84,8 +71,6 @@ def "main vm snapshots" [] {
|
||||
| print
|
||||
}
|
||||
|
||||
# Restore the state disk from a snapshot. The system disk is left alone —
|
||||
# it holds nothing worth restoring.
|
||||
def "main vm rollback" [snapshot: string] {
|
||||
let snap = $"(snap-dir)/($snapshot)"
|
||||
if not ($snap | path exists) { die $"no such snapshot: ($snapshot)" }
|
||||
|
||||
@@ -25,9 +25,7 @@ let
|
||||
nixpkgsPath = pkgs.path;
|
||||
has = lib.strings.hasInfix;
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# Host module, evaluated with stub options for what this check reads. #
|
||||
# ------------------------------------------------------------------ #
|
||||
hostOptions = { lib, ... }: {
|
||||
config._module.check = false;
|
||||
options = {
|
||||
@@ -211,10 +209,8 @@ let
|
||||
brNetworks."11-vm-hermes".networkConfig.Bridge == "hermes-br";
|
||||
};
|
||||
|
||||
# ------------------------------------------------------------------- #
|
||||
# Guest module: must produce one NIC config that works in both modes. #
|
||||
# Evaluated against the full NixOS option set. #
|
||||
# ------------------------------------------------------------------- #
|
||||
guest = lib.evalModules {
|
||||
modules = (import "${nixpkgsPath}/nixos/modules/module-list.nix") ++ [
|
||||
{ config._module.check = false; }
|
||||
|
||||
Reference in new issue
Block a user