Initial hermex: Hermes Agent in a hardened, self-managing NixOS VM
Two-disk (system/state) btrfs-backed QEMU VM, nftables-isolated by default, adapted from a design provided as loose files. Adds a hermesHome option to the host module and migrate-docker-hermes.sh so an existing flat HERMES_HOME (e.g. from a prior Docker deployment) can be carried over intact. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
commit
12227460fd
15 files changed
+2765
No files matched your search
@@ -0,0 +1,669 @@
|
||||
# hermes-vm
|
||||
|
||||
Hermes Agent in a NixOS VM that owns its own bootloader and its own
|
||||
`/nix/store`, on two btrfs-backed disks, under an unprivileged hardened QEMU.
|
||||
|
||||
The agent can rebuild its own system and update itself, durably. It shares no
|
||||
filesystem with the host, and reaches nothing on your network unless you say
|
||||
so.
|
||||
|
||||
---
|
||||
|
||||
## Installation
|
||||
|
||||
### Guided
|
||||
|
||||
```bash
|
||||
./setup.sh
|
||||
```
|
||||
|
||||
Walks the host-side steps: checks prerequisites, creates the disks, generates
|
||||
the cache signing key, fills in `guest/common.nix` from your SSH keys, builds
|
||||
the bootstrap image and writes it to the disk. It prints the module snippet
|
||||
to paste into your own configuration at the end, with your uplink interface
|
||||
already filled in.
|
||||
|
||||
Safe to re-run — every step detects what's already done and offers to skip,
|
||||
and nothing destructive happens without an explicit confirmation. It keeps a
|
||||
`guest/common.nix.bak`.
|
||||
|
||||
It deliberately stops short of `nixos-rebuild`, since that's your config.
|
||||
|
||||
The rest of this section is what the script does, for when you'd rather do it
|
||||
by hand or something goes sideways.
|
||||
|
||||
### 1. Create the disks
|
||||
|
||||
Two of them, and the split is the whole design:
|
||||
|
||||
| | Holds | Size | Snapshotted | Backed up |
|
||||
|---|---|---|---|---|
|
||||
| **system** | NixOS, `/nix/store` | 80G | No | No |
|
||||
| **state** | identity, memories, workspace, `/etc/nixos` | 20G | Yes | Yes |
|
||||
|
||||
The system disk is large, takes all the write churn, and is entirely
|
||||
rebuildable from this flake — so there's no point protecting it. The state
|
||||
disk holds the only things that can't be reproduced, which makes snapshots
|
||||
cheap and backups quick.
|
||||
|
||||
```bash
|
||||
sudo btrfs subvolume create /var/lib/hermes-vm/system
|
||||
sudo btrfs subvolume create /var/lib/hermes-vm/state
|
||||
sudo mkdir -p /var/lib/hermes-vm/snapshots
|
||||
|
||||
# BEFORE creating any file inside them
|
||||
sudo chattr +C /var/lib/hermes-vm/system /var/lib/hermes-vm/state
|
||||
sudo btrfs property set /var/lib/hermes-vm/system state compression none
|
||||
|
||||
sudo truncate -s 80G /var/lib/hermes-vm/system/system.raw
|
||||
sudo truncate -s 20G /var/lib/hermes-vm/state/state.raw
|
||||
sudo mkfs.ext4 -L hermes-state /var/lib/hermes-vm/state/state.raw
|
||||
```
|
||||
|
||||
**The `chattr +C` ordering matters.** Btrfs is copy-on-write and so is a VM
|
||||
writing to a disk image; stacked, every guest write forks an extent and the
|
||||
image fragments badly. `+C` turns that off — but only for files created
|
||||
afterwards. Set it on empty subvolumes and confirm with `lsattr` before
|
||||
writing anything. If you get it backwards, recreate the subvolume and copy
|
||||
the image in fresh.
|
||||
|
||||
Separate subvolumes because a btrfs snapshot covers a whole subvolume. Keeping
|
||||
the system image out of the state subvolume is what makes snapshots small.
|
||||
|
||||
### 2. Generate the cache signing key
|
||||
|
||||
The host serves its store to the guest as a read-only binary cache, so the
|
||||
guest doesn't re-download from the internet what you already have.
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /var/lib/secrets
|
||||
sudo nix-store --generate-binary-cache-key hermes-cache-1 \
|
||||
/var/lib/secrets/hermes-cache.secret /var/lib/secrets/hermes-cache.pub
|
||||
sudo chown nix-serve /var/lib/secrets/hermes-cache.secret
|
||||
sudo chmod 0400 /var/lib/secrets/hermes-cache.secret
|
||||
cat /var/lib/secrets/hermes-cache.pub
|
||||
```
|
||||
|
||||
### 3. Fill in `guest/common.nix`
|
||||
|
||||
Near the top:
|
||||
|
||||
- `cachePublicKey` — paste the key you just printed. Leaving it empty is safe;
|
||||
the guest just skips the host cache.
|
||||
- `authorizedKeys` — your SSH public key.
|
||||
|
||||
### 4. Build the image and write it to the system disk
|
||||
|
||||
The two extra options stop you compiling Determinate Nix from source.
|
||||
|
||||
```bash
|
||||
nix build .#hermes-guest-image \
|
||||
--option extra-substituters https://install.determinate.systems \
|
||||
--option extra-trusted-public-keys cache.flakehub.com-3:hJuILl5sVK4iKm86JzgdXW12Y2Hwd5G07qKtHTOcDCM=
|
||||
|
||||
# notrunc keeps the disk at its full size; the image is smaller
|
||||
sudo dd if=result/nixos.img of=/var/lib/hermes-vm/system/system.raw \
|
||||
bs=4M conv=notrunc,fsync status=progress
|
||||
```
|
||||
|
||||
### 5. Add the module to your system flake
|
||||
|
||||
```nix
|
||||
{
|
||||
inputs.hermes-vm.url = "path:/etc/nixos/hermes-vm";
|
||||
|
||||
outputs = { nixpkgs, hermes-vm, ... }: {
|
||||
nixosConfigurations.myserver = nixpkgs.lib.nixosSystem {
|
||||
system = "x86_64-linux";
|
||||
modules = [ hermes-vm.nixosModules.default ./configuration.nix ];
|
||||
};
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
```nix
|
||||
services.hermesVm = {
|
||||
enable = true;
|
||||
disk.baseDir = "/var/lib/hermes-vm";
|
||||
network.externalInterface = "eno1"; # your uplink
|
||||
vcpu = 4;
|
||||
mem = 8192;
|
||||
};
|
||||
```
|
||||
|
||||
`sudo nixos-rebuild switch`, then `hermes-vm console` to watch it boot.
|
||||
|
||||
### 6. First boot, inside the guest
|
||||
|
||||
```bash
|
||||
hermes-vm ssh
|
||||
|
||||
# Confirm the disk labels match guest/hardware.nix BEFORE any rebuild
|
||||
lsblk -f
|
||||
|
||||
# FlakeHub Cache login. Device-code flow: it prints a URL you approve in a
|
||||
# browser anywhere, so the headless guest never needs one.
|
||||
determinate-nixd login
|
||||
|
||||
# Pin the seeded flake
|
||||
sudo nix flake lock /etc/nixos
|
||||
|
||||
# Install Hermes
|
||||
curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash
|
||||
hermes setup
|
||||
hermes gateway install
|
||||
```
|
||||
|
||||
The identity seed is already in place — `SOUL.md`, a starter `config.yaml`, a
|
||||
`USER.md` and a skill covering safe self-rebuilds. Back on the host, check
|
||||
the gateway came up and that the unit name matches the default:
|
||||
|
||||
```bash
|
||||
hermes-ctl units
|
||||
hermes-ctl status
|
||||
hermes-ctl soul
|
||||
```
|
||||
|
||||
> **Check the labels in step 6 before the first rebuild.**
|
||||
> `guest/hardware.nix` assumes the root partition is labelled `nixos` and the
|
||||
> ESP `ESP`. If the image generator used different ones, fix that file first.
|
||||
> Otherwise the agent's first `nixos-rebuild switch` produces a system that
|
||||
> won't boot and you're restoring a snapshot on day one.
|
||||
|
||||
From here `hermes update` self-updates in place and
|
||||
`sudo nixos-rebuild switch --flake /etc/nixos#hermes` is a durable change to
|
||||
the machine. Neither needs anything from you.
|
||||
|
||||
---
|
||||
|
||||
## SSH into the guest
|
||||
|
||||
```bash
|
||||
hermes-vm ssh # interactive shell
|
||||
hermes-vm ssh systemctl status # run a command and exit
|
||||
hermes-vm console # serial console, for when SSH is down
|
||||
```
|
||||
|
||||
This goes over **vsock**, not the network. There is no listening SSH port
|
||||
exposed anywhere, and the guest needs no inbound network path for you to
|
||||
reach it. If the guest has broken its own networking entirely, `hermes-vm ssh`
|
||||
still works — `console` is the fallback for when it's broken its SSH too.
|
||||
|
||||
Under the hood it's `ssh -o ProxyCommand='socat - VSOCK-CONNECT:77:22'`, if
|
||||
you'd rather drive it by hand.
|
||||
|
||||
---
|
||||
|
||||
## Bootstrapping Hermes's identity
|
||||
|
||||
The seed lives in `guest/hermes/`:
|
||||
|
||||
```
|
||||
guest/hermes/
|
||||
home/
|
||||
SOUL.md -> ~/.hermes/SOUL.md
|
||||
config.yaml -> ~/.hermes/config.yaml
|
||||
memories/USER.md -> ~/.hermes/memories/USER.md
|
||||
skills/nixos-self-config/SKILL.md -> ~/.hermes/skills/...
|
||||
workspace/
|
||||
AGENTS.md -> ~/workspace/AGENTS.md
|
||||
```
|
||||
|
||||
Edit these before building the image and the agent wakes up already knowing
|
||||
who it is, how you like to be spoken to, and how to rebuild its own system
|
||||
without bricking itself.
|
||||
|
||||
**Seeding happens on first boot only, and this is deliberate.** Hermes
|
||||
rewrites `SOUL.md` and its memories as it learns — that's the product. If the
|
||||
seed were re-applied on every rebuild it would silently delete everything the
|
||||
agent had worked out about you. So the copy is guarded by a marker file and
|
||||
never repeats.
|
||||
|
||||
Two consequences:
|
||||
|
||||
- Editing `guest/hermes/` after first boot changes nothing by itself. Use
|
||||
`hermes-ctl reseed` when you actually want to overwrite.
|
||||
- The seed is the *starting* identity, not the current one. To see what
|
||||
Hermes has made of itself, `hermes-ctl soul`.
|
||||
|
||||
### Machine facts vs identity
|
||||
|
||||
`/etc/hermes/ENVIRONMENT.md` is generated from `guest/common.nix` on every
|
||||
rebuild and describes the machine: what's reachable, where the store is, what
|
||||
persists. It's always accurate and there's no point editing it.
|
||||
|
||||
`SOUL.md` is identity and belongs to the agent. The seeded version points at
|
||||
`ENVIRONMENT.md` rather than repeating it, so the two can't drift.
|
||||
|
||||
### Folding what it learned back into the seed
|
||||
|
||||
```bash
|
||||
hermes-ctl pull ./hermes-home # excludes .env, sessions, state.db
|
||||
diff -ru guest/hermes/home ./hermes-home
|
||||
```
|
||||
|
||||
Worth doing occasionally. If the agent has written something genuinely good
|
||||
into its own SOUL.md, you want that in the seed before you ever rebuild from
|
||||
scratch.
|
||||
|
||||
---
|
||||
|
||||
## Managing Hermes from the host
|
||||
|
||||
`hermes-vm` drives the virtual machine. `hermes-ctl` drives the agent inside
|
||||
it. Rename the latter with `services.hermesVm.cliName = "hermes";` if you'd
|
||||
rather it felt local.
|
||||
|
||||
```bash
|
||||
hermes-ctl soul # current identity
|
||||
hermes-ctl soul edit # pull, $EDITOR, push, offer to restart
|
||||
hermes-ctl memory edit # memories/USER.md
|
||||
hermes-ctl conf edit # config.yaml
|
||||
|
||||
hermes-ctl secrets # keys, values masked
|
||||
hermes-ctl secrets edit # edit ~/.hermes/.env, written 0600
|
||||
hermes-ctl secrets show # values, when you really mean it
|
||||
|
||||
hermes-ctl status # gateway service
|
||||
hermes-ctl logs # follow it
|
||||
hermes-ctl restart
|
||||
hermes-ctl units # find the unit name if the default is wrong
|
||||
|
||||
hermes-ctl reseed # overwrite identity from the seed
|
||||
hermes-ctl pull [dir] # copy live identity to the host
|
||||
```
|
||||
|
||||
Anything unrecognised is passed straight through to the `hermes` CLI in the
|
||||
guest, with a TTY attached:
|
||||
|
||||
```bash
|
||||
hermes-ctl chat
|
||||
hermes-ctl update
|
||||
hermes-ctl sessions list
|
||||
hermes-ctl doctor
|
||||
```
|
||||
|
||||
The `edit` variants pull the file out over vsock, open your `$EDITOR`
|
||||
locally, write it back, and offer to restart the gateway. Secrets go in at
|
||||
mode 0600 and never pass through the Nix store — which is the whole reason
|
||||
there's no declarative option for them.
|
||||
|
||||
If `status` and `logs` come back empty, the gateway unit is probably named
|
||||
something other than `hermes-agent`. Run `hermes-ctl units` and set
|
||||
`services.hermesVm.gatewayUnit` to match; upstream has used more than one
|
||||
name for it.
|
||||
|
||||
## Seeing what the guest is running
|
||||
|
||||
```bash
|
||||
hermes-vm config
|
||||
```
|
||||
|
||||
Prints the current system store path, the last ten generations, the git log
|
||||
of `/etc/nixos`, and anything uncommitted. That last one matters: the agent
|
||||
editing files without committing is the usual reason the config on disk and
|
||||
the running system disagree.
|
||||
|
||||
To read or diff the config locally:
|
||||
|
||||
```bash
|
||||
hermes-vm config-pull ./guest-live
|
||||
diff -ru guest/ guest-live/
|
||||
```
|
||||
|
||||
Expect that diff to grow. The `guest/` directory in this repo is the seed
|
||||
that was written on first boot; it is deliberately not kept in sync. Once the
|
||||
agent edits its config, this repo and the guest diverge permanently, and a
|
||||
host rebuild will not fix a guest config problem.
|
||||
|
||||
---
|
||||
|
||||
## Letting the guest reach a service on your network
|
||||
|
||||
By default the guest can reach the public internet and nothing private. To
|
||||
open a single endpoint — say Ollama on another machine:
|
||||
|
||||
```nix
|
||||
services.hermesVm.lanAccess = [
|
||||
{ address = "192.168.1.50"; port = 11434; } # ollama
|
||||
{ address = "192.168.1.50"; port = 9000; proto = "udp"; }
|
||||
];
|
||||
```
|
||||
|
||||
Each entry punches one destination and port through the RFC1918 drop. The
|
||||
guest reaches it as `192.168.1.50:11434` directly. Traffic is masqueraded, so
|
||||
the other machine sees it arriving from your host's address, not the guest's.
|
||||
|
||||
There's also `allowLan = true`, which opens all private address space at
|
||||
once. It exists, but reach for `lanAccess` instead — the whole point of this
|
||||
design is that an agent which goes wrong can't go wandering.
|
||||
|
||||
### A service on the host itself
|
||||
|
||||
Different chain, different option. ComfyUI on the host, port 8188:
|
||||
|
||||
```nix
|
||||
services.hermesVm.hostAccess = [ { port = 8188; } ];
|
||||
```
|
||||
|
||||
**The service must be bound somewhere the guest can see.** If it listens
|
||||
on `127.0.0.1` the rule won't help, because the guest arrives on the bridge
|
||||
address. Bind it to `10.77.0.1` (bridge only, invisible to your LAN) or
|
||||
`0.0.0.0`:
|
||||
|
||||
```nix
|
||||
services.comfyui.listenAddress = "10.77.0.1"; # or whatever your service calls it
|
||||
```
|
||||
|
||||
From the guest that's then `http://10.77.0.1:8188`.
|
||||
|
||||
---
|
||||
|
||||
## Exposing a service the guest is running
|
||||
|
||||
Say the agent has built a web app on guest port 8080. Two halves.
|
||||
|
||||
**Guest side.** The agent opens its own firewall, in `/etc/nixos`:
|
||||
|
||||
```nix
|
||||
networking.firewall.allowedTCPPorts = [ 22 8080 ];
|
||||
```
|
||||
|
||||
**Host side.** Nothing at all, if you only want to reach it from the host —
|
||||
host-to-guest is outbound traffic and already permitted:
|
||||
|
||||
```bash
|
||||
curl http://10.77.0.2:8080
|
||||
```
|
||||
|
||||
To publish it to the wider network:
|
||||
|
||||
```nix
|
||||
services.hermesVm.publish = [
|
||||
{ hostPort = 8080; guestPort = 8080; }
|
||||
];
|
||||
```
|
||||
|
||||
That DNATs the port on your uplink to the guest and opens the matching
|
||||
forward rule. `hostPort` and `guestPort` don't have to match, so you can park
|
||||
an awkward guest port somewhere sensible.
|
||||
|
||||
A note on this one: publishing a port is the one change here that lets the
|
||||
outside world start conversations with the agent's machine. Everything else
|
||||
in this design is the guest reaching out. Put a reverse proxy with TLS and
|
||||
auth in front of anything that isn't a toy, and keep the list short.
|
||||
|
||||
---
|
||||
|
||||
## Passing hardware to the guest
|
||||
|
||||
### Before reaching for this
|
||||
|
||||
If what you want is "the agent can use the GPU", passthrough is often the
|
||||
wrong answer. It is exclusive — the host loses the card completely while the
|
||||
VM runs — and it is the single largest reduction in isolation available in
|
||||
this setup.
|
||||
|
||||
The alternative: keep Ollama or ComfyUI on the host where the GPU already
|
||||
works, and open the port with `hostAccess`. The agent gets the capability
|
||||
over HTTP and the device stays on your side of the boundary. That is usually
|
||||
both easier and better.
|
||||
|
||||
Pass the hardware through when the guest genuinely needs the device itself —
|
||||
CUDA in-process, a capture card, a radio, a display output.
|
||||
|
||||
### What passthrough costs you
|
||||
|
||||
A PCI device does DMA. The IOMMU is the only thing standing between a
|
||||
passed-through device and host memory, which is why `passthrough.iommu` is a
|
||||
hard requirement and the module refuses to build without it.
|
||||
|
||||
Two further constraints follow from how IOMMUs work:
|
||||
|
||||
- **Whole groups only.** Every device in an IOMMU group goes together. Check
|
||||
yours before planning anything:
|
||||
|
||||
```bash
|
||||
for d in /sys/kernel/iommu_groups/*/devices/*; do
|
||||
echo "group $(basename "$(dirname "$(dirname "$d")")"): $(lspci -nns "$(basename "$d")")"
|
||||
done | sort -V
|
||||
```
|
||||
|
||||
If your GPU shares a group with a controller you need, you cannot pass it
|
||||
cleanly. ACS override patches exist; they work by lying about isolation,
|
||||
and they have no place in a setup built to contain an autonomous agent.
|
||||
|
||||
- **Every function.** A GPU is at least two entries — graphics and its HDMI
|
||||
audio function.
|
||||
|
||||
### GPU
|
||||
|
||||
```bash
|
||||
lspci -Dnn | grep -i vga # addresses and vendor:device IDs
|
||||
```
|
||||
|
||||
```nix
|
||||
services.hermesVm.passthrough = {
|
||||
iommu = "amd"; # or "intel"
|
||||
pci = [
|
||||
{ address = "0000:01:00.0"; id = "10de:2204"; } # graphics
|
||||
{ address = "0000:01:00.1"; id = "10de:1aef"; } # HDMI audio
|
||||
];
|
||||
};
|
||||
```
|
||||
|
||||
Setting `id` makes vfio-pci claim the card in the initrd, before `nvidia` or
|
||||
`amdgpu` can. Do this for GPUs — unbinding a live graphics driver afterwards
|
||||
is unreliable, and the failure mode is a host with no display. Devices given
|
||||
only an `address` are bound at boot by a oneshot service instead, which is
|
||||
fine for things you swap around.
|
||||
|
||||
Rebuild and **reboot** — early binding happens in the initrd. Then check:
|
||||
|
||||
```bash
|
||||
lspci -nnk -d 10de:2204 | grep "Kernel driver" # want: vfio-pci
|
||||
```
|
||||
|
||||
On the guest side, the agent installs the drivers itself. It owns
|
||||
`/etc/nixos`, so `hardware.nvidia` and `nvidia-container-toolkit` are its
|
||||
problem, not yours — which is rather the point of the arrangement.
|
||||
|
||||
### HDMI output
|
||||
|
||||
A display hanging off a passed-through GPU works once the guest drives the
|
||||
card; nothing extra is needed here. A capture card or HDMI grabber is just
|
||||
another PCI or USB device — find it with `lspci -Dnn` or `lsusb` and use the
|
||||
matching section.
|
||||
|
||||
### USB devices and Bluetooth
|
||||
|
||||
Matched by vendor and product ID:
|
||||
|
||||
```bash
|
||||
lsusb # e.g. "Bus 001 Device 004: ID 8087:0026 Intel ..."
|
||||
```
|
||||
|
||||
```nix
|
||||
services.hermesVm.passthrough.usb = [
|
||||
{ vendorId = "8087"; productId = "0026"; } # bluetooth radio
|
||||
{ vendorId = "0403"; productId = "6001"; } # serial adapter
|
||||
];
|
||||
```
|
||||
|
||||
This adds an xHCI controller to the guest and forwards the matching devices.
|
||||
No IOMMU requirement, no VFIO, and much milder than PCI passthrough — QEMU
|
||||
mediates rather than handing over raw hardware.
|
||||
|
||||
Two things that catch people:
|
||||
|
||||
- **The host must not be using it.** For an integrated Bluetooth radio, keep
|
||||
`btusb` off it: `boot.blacklistedKernelModules = [ "btusb" ];`, and don't
|
||||
enable `hardware.bluetooth` on the host.
|
||||
- **No hotplug.** QEMU resolves these at start, so a device that's unplugged
|
||||
when the VM boots won't appear when you plug it in. `hermes-vm restart`
|
||||
after plugging. Passing a whole USB *controller* through as a PCI device
|
||||
gets you real hotplug for anything in its ports, at the cost of that
|
||||
controller being exclusive to the guest.
|
||||
|
||||
### When it doesn't start
|
||||
|
||||
`hermes-vm console` first. Then, in rough order of likelihood: the device
|
||||
didn't bind to vfio-pci (`lspci -nnk`), the IOMMU isn't on (`dmesg | grep -i
|
||||
iommu`), or something in the group is still held by the host. The module
|
||||
lifts `LimitMEMLOCK` for you when PCI passthrough is configured, since VFIO
|
||||
pins the whole guest RAM allocation and the default limit otherwise produces
|
||||
a memorably unhelpful error.
|
||||
|
||||
## Recovery
|
||||
|
||||
Four layers, cheapest first. The first two live inside the guest and cover
|
||||
almost every real mistake.
|
||||
|
||||
```bash
|
||||
# The agent's own config history
|
||||
hermes-ctl soul # what it thinks it is now
|
||||
hermes-vm ssh git -C /etc/nixos log --oneline
|
||||
|
||||
# Previous NixOS generations, also in the guest's boot menu
|
||||
hermes-vm ssh sudo nixos-rebuild switch --rollback
|
||||
|
||||
# Snapshot of the state disk
|
||||
hermes-vm snapshots
|
||||
hermes-vm rollback state-20260914-1400
|
||||
|
||||
# Restore from backup
|
||||
hermes-vm backups
|
||||
```
|
||||
|
||||
### Snapshots
|
||||
|
||||
Hourly by default, keeping 48. Only the state disk — the system disk isn't
|
||||
snapshotted because `setup.sh` rebuilds it from the flake in minutes.
|
||||
|
||||
The host fsfreezes the guest through qemu-guest-agent around each one, so the
|
||||
snapshot captures a consistent filesystem rather than the equivalent of
|
||||
pulling the power. Takes milliseconds; the VM keeps running.
|
||||
|
||||
`hermes-vm rollback` stops the VM, moves the current state subvolume aside as
|
||||
`state.old-<timestamp>`, and restores the snapshot in its place. Nothing is
|
||||
deleted, so a mistaken rollback is itself reversible. Clean the `state.old-*`
|
||||
directories up yourself once you're happy.
|
||||
|
||||
**Keep retention tighter than you would on ZFS.** The state image is
|
||||
nodatacow, and every live snapshot reintroduces copy-on-write for the blocks
|
||||
it still references. Performance sags while snapshots exist and recovers as
|
||||
they're pruned.
|
||||
|
||||
### Backups
|
||||
|
||||
```nix
|
||||
services.hermesVm.backup = {
|
||||
enable = true;
|
||||
repository = "sftp:backup@nas:/hermes";
|
||||
passwordFile = "/var/lib/secrets/hermes-restic";
|
||||
};
|
||||
```
|
||||
|
||||
This backs up the guest's *state*, not its disk. Files are pulled over the
|
||||
SSH channel that already exists and streamed into restic.
|
||||
|
||||
Two reasons it works that way. The backup stays portable — it restores onto
|
||||
any filesystem, or a different machine entirely, and doesn't care that you're
|
||||
on btrfs today. And it avoids loop-mounting a guest-written filesystem on the
|
||||
host, which would mean the host kernel parsing data the agent controls. That
|
||||
would undo a fair bit of what the rest of this design is for.
|
||||
|
||||
If the repository is on your LAN rather than the internet, note this runs on
|
||||
the *host*, so it needs no `lanAccess` entry — the guest never touches it.
|
||||
|
||||
### When the guest won't boot at all
|
||||
|
||||
The system disk is disposable by design. Re-run `setup.sh`, write a fresh
|
||||
image to the system disk, and leave the state disk alone — Hermes comes back
|
||||
with its identity, memories, workspace and NixOS config intact. That is the
|
||||
intended repair, not a last resort.
|
||||
|
||||
---
|
||||
|
||||
## Nix flavour
|
||||
|
||||
The guest runs **Determinate Nix**, nixpkgs pinned to FlakeHub
|
||||
(`f/NixOS/nixpkgs/0.1`, rolling; use `0` for current stable).
|
||||
|
||||
- `/etc/nix/nix.conf` belongs to Determinate and must not be edited. Custom
|
||||
settings go in `/etc/nix/nix.custom.conf`, written from `guest/common.nix`.
|
||||
There is deliberately no `nix.settings` block.
|
||||
- No `nix.gc` either — `determinate-nixd` runs the collector.
|
||||
- Lazy trees are on by default, which makes the agent's flake evaluations
|
||||
much cheaper. That matters more than it sounds when something re-evaluates
|
||||
nixpkgs on a schedule.
|
||||
- Neither flake uses `inputs.nixpkgs.follows` for `determinate`. Determinate
|
||||
warn it causes FlakeHub Cache misses.
|
||||
|
||||
Substituter order in the guest: FlakeHub Cache, your host cache,
|
||||
`cache.nixos.org`. If FlakeHub Cache goes quiet, the login has expired — run
|
||||
`determinate-nixd login` again.
|
||||
|
||||
The host stays on stock Nix. Nothing here requires changing that.
|
||||
|
||||
---
|
||||
|
||||
## What the guest can and cannot reach
|
||||
|
||||
| | Default |
|
||||
|---|---|
|
||||
| Public internet | Yes — it needs this for model APIs and messaging |
|
||||
| Host Nix cache | Yes, one port, `hostCache.enable = false` closes it |
|
||||
| Anything else on the host | No, unless in `hostAccess` |
|
||||
| Your LAN and other VMs | No, unless in `lanAccess` |
|
||||
| Inbound from the network | No, unless in `publish` |
|
||||
| Host hardware | None, unless in `passthrough` |
|
||||
|
||||
The guest owns its kernel and its config, so nothing inside it is a security
|
||||
boundary. The nftables rules in `modules/host.nix` are. QEMU runs as an
|
||||
unprivileged `hermes-vm` user with its own seccomp sandbox and systemd
|
||||
hardening, with a thin device model: virtio-blk, virtio-net, vsock, RNG,
|
||||
serial. No virtiofs. `vhost-net` is off deliberately — it would move packet
|
||||
handling into the host kernel in exchange for throughput one agent doesn't
|
||||
need.
|
||||
|
||||
The host cache does let the guest read every store path on the host. Store
|
||||
paths are world-readable anyway, so this isn't new exposure, but don't bake
|
||||
secrets into your host's store.
|
||||
|
||||
---
|
||||
|
||||
## Known rough edges
|
||||
|
||||
- The Determinate NixOS module's option names for custom Nix settings aren't
|
||||
published — only the nix-darwin `determinateNix.customSettings` equivalent
|
||||
is. So `guest/common.nix` writes `/etc/nix/nix.custom.conf` via
|
||||
`environment.etc`. If a future module version claims that file, you'll get
|
||||
a conflict; switch to its option then.
|
||||
- `determinate-nixd login` is manual. There's no way to bake the credential
|
||||
into the image without parking a token somewhere it shouldn't be.
|
||||
- `MemoryDenyWriteExecute = true` is correct under KVM but is the first thing
|
||||
to flip if QEMU refuses to start.
|
||||
- `pkgs.OVMF.firmware` / `pkgs.OVMF.variables` are current attribute names;
|
||||
older nixpkgs put these under `pkgs.OVMF.fd` in an `FV/` subdirectory.
|
||||
- PCI passthrough needs a reboot to take effect, because early vfio binding
|
||||
happens in the initrd.
|
||||
- Snapshots need btrfs. Everything else works on any filesystem; without it
|
||||
you rely on backups alone for recovery.
|
||||
- `state.old-*` directories left by `hermes-vm rollback` are not cleaned up
|
||||
automatically, on purpose. Delete them yourself once you're sure.
|
||||
- First boot builds a lot. Give it time.
|
||||
|
||||
---
|
||||
|
||||
## Migrating an existing (e.g. Docker-based) Hermes deployment
|
||||
|
||||
If you already have a running Hermes with real data (`HERMES_HOME` pointing
|
||||
at some directory full of `SOUL.md`, `memories/`, `sessions/`, `state.db`,
|
||||
etc.), see `migrate-docker-hermes.sh`. It copies — never moves or deletes —
|
||||
the source data, so the original stays a safe fallback until you've verified
|
||||
the new VM.
|
||||
|
||||
The module's `services.hermesVm.hermesHome` option lets `hermes-ctl` (and
|
||||
your own `HERMES_HOME` export inside the guest) point at wherever that
|
||||
existing flat layout lands, instead of the default seeded `~/.hermes`.
|
||||
Reference in new issue
Block a user