Two-disk (system/state) btrfs-backed QEMU VM, nftables-isolated by default, adapted from a design provided as loose files. Adds a hermesHome option to the host module and migrate-docker-hermes.sh so an existing flat HERMES_HOME (e.g. from a prior Docker deployment) can be carried over intact. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
670 lines
23 KiB
Markdown
670 lines
23 KiB
Markdown
# hermes-vm
|
|
|
|
Hermes Agent in a NixOS VM that owns its own bootloader and its own
|
|
`/nix/store`, on two btrfs-backed disks, under an unprivileged hardened QEMU.
|
|
|
|
The agent can rebuild its own system and update itself, durably. It shares no
|
|
filesystem with the host, and reaches nothing on your network unless you say
|
|
so.
|
|
|
|
---
|
|
|
|
## Installation
|
|
|
|
### Guided
|
|
|
|
```bash
|
|
./setup.sh
|
|
```
|
|
|
|
Walks the host-side steps: checks prerequisites, creates the disks, generates
|
|
the cache signing key, fills in `guest/common.nix` from your SSH keys, builds
|
|
the bootstrap image and writes it to the disk. It prints the module snippet
|
|
to paste into your own configuration at the end, with your uplink interface
|
|
already filled in.
|
|
|
|
Safe to re-run — every step detects what's already done and offers to skip,
|
|
and nothing destructive happens without an explicit confirmation. It keeps a
|
|
`guest/common.nix.bak`.
|
|
|
|
It deliberately stops short of `nixos-rebuild`, since that's your config.
|
|
|
|
The rest of this section is what the script does, for when you'd rather do it
|
|
by hand or something goes sideways.
|
|
|
|
### 1. Create the disks
|
|
|
|
Two of them, and the split is the whole design:
|
|
|
|
| | Holds | Size | Snapshotted | Backed up |
|
|
|---|---|---|---|---|
|
|
| **system** | NixOS, `/nix/store` | 80G | No | No |
|
|
| **state** | identity, memories, workspace, `/etc/nixos` | 20G | Yes | Yes |
|
|
|
|
The system disk is large, takes all the write churn, and is entirely
|
|
rebuildable from this flake — so there's no point protecting it. The state
|
|
disk holds the only things that can't be reproduced, which makes snapshots
|
|
cheap and backups quick.
|
|
|
|
```bash
|
|
sudo btrfs subvolume create /var/lib/hermes-vm/system
|
|
sudo btrfs subvolume create /var/lib/hermes-vm/state
|
|
sudo mkdir -p /var/lib/hermes-vm/snapshots
|
|
|
|
# BEFORE creating any file inside them
|
|
sudo chattr +C /var/lib/hermes-vm/system /var/lib/hermes-vm/state
|
|
sudo btrfs property set /var/lib/hermes-vm/system state compression none
|
|
|
|
sudo truncate -s 80G /var/lib/hermes-vm/system/system.raw
|
|
sudo truncate -s 20G /var/lib/hermes-vm/state/state.raw
|
|
sudo mkfs.ext4 -L hermes-state /var/lib/hermes-vm/state/state.raw
|
|
```
|
|
|
|
**The `chattr +C` ordering matters.** Btrfs is copy-on-write and so is a VM
|
|
writing to a disk image; stacked, every guest write forks an extent and the
|
|
image fragments badly. `+C` turns that off — but only for files created
|
|
afterwards. Set it on empty subvolumes and confirm with `lsattr` before
|
|
writing anything. If you get it backwards, recreate the subvolume and copy
|
|
the image in fresh.
|
|
|
|
Separate subvolumes because a btrfs snapshot covers a whole subvolume. Keeping
|
|
the system image out of the state subvolume is what makes snapshots small.
|
|
|
|
### 2. Generate the cache signing key
|
|
|
|
The host serves its store to the guest as a read-only binary cache, so the
|
|
guest doesn't re-download from the internet what you already have.
|
|
|
|
```bash
|
|
sudo mkdir -p /var/lib/secrets
|
|
sudo nix-store --generate-binary-cache-key hermes-cache-1 \
|
|
/var/lib/secrets/hermes-cache.secret /var/lib/secrets/hermes-cache.pub
|
|
sudo chown nix-serve /var/lib/secrets/hermes-cache.secret
|
|
sudo chmod 0400 /var/lib/secrets/hermes-cache.secret
|
|
cat /var/lib/secrets/hermes-cache.pub
|
|
```
|
|
|
|
### 3. Fill in `guest/common.nix`
|
|
|
|
Near the top:
|
|
|
|
- `cachePublicKey` — paste the key you just printed. Leaving it empty is safe;
|
|
the guest just skips the host cache.
|
|
- `authorizedKeys` — your SSH public key.
|
|
|
|
### 4. Build the image and write it to the system disk
|
|
|
|
The two extra options stop you compiling Determinate Nix from source.
|
|
|
|
```bash
|
|
nix build .#hermes-guest-image \
|
|
--option extra-substituters https://install.determinate.systems \
|
|
--option extra-trusted-public-keys cache.flakehub.com-3:hJuILl5sVK4iKm86JzgdXW12Y2Hwd5G07qKtHTOcDCM=
|
|
|
|
# notrunc keeps the disk at its full size; the image is smaller
|
|
sudo dd if=result/nixos.img of=/var/lib/hermes-vm/system/system.raw \
|
|
bs=4M conv=notrunc,fsync status=progress
|
|
```
|
|
|
|
### 5. Add the module to your system flake
|
|
|
|
```nix
|
|
{
|
|
inputs.hermes-vm.url = "path:/etc/nixos/hermes-vm";
|
|
|
|
outputs = { nixpkgs, hermes-vm, ... }: {
|
|
nixosConfigurations.myserver = nixpkgs.lib.nixosSystem {
|
|
system = "x86_64-linux";
|
|
modules = [ hermes-vm.nixosModules.default ./configuration.nix ];
|
|
};
|
|
};
|
|
}
|
|
```
|
|
|
|
```nix
|
|
services.hermesVm = {
|
|
enable = true;
|
|
disk.baseDir = "/var/lib/hermes-vm";
|
|
network.externalInterface = "eno1"; # your uplink
|
|
vcpu = 4;
|
|
mem = 8192;
|
|
};
|
|
```
|
|
|
|
`sudo nixos-rebuild switch`, then `hermes-vm console` to watch it boot.
|
|
|
|
### 6. First boot, inside the guest
|
|
|
|
```bash
|
|
hermes-vm ssh
|
|
|
|
# Confirm the disk labels match guest/hardware.nix BEFORE any rebuild
|
|
lsblk -f
|
|
|
|
# FlakeHub Cache login. Device-code flow: it prints a URL you approve in a
|
|
# browser anywhere, so the headless guest never needs one.
|
|
determinate-nixd login
|
|
|
|
# Pin the seeded flake
|
|
sudo nix flake lock /etc/nixos
|
|
|
|
# Install Hermes
|
|
curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash
|
|
hermes setup
|
|
hermes gateway install
|
|
```
|
|
|
|
The identity seed is already in place — `SOUL.md`, a starter `config.yaml`, a
|
|
`USER.md` and a skill covering safe self-rebuilds. Back on the host, check
|
|
the gateway came up and that the unit name matches the default:
|
|
|
|
```bash
|
|
hermes-ctl units
|
|
hermes-ctl status
|
|
hermes-ctl soul
|
|
```
|
|
|
|
> **Check the labels in step 6 before the first rebuild.**
|
|
> `guest/hardware.nix` assumes the root partition is labelled `nixos` and the
|
|
> ESP `ESP`. If the image generator used different ones, fix that file first.
|
|
> Otherwise the agent's first `nixos-rebuild switch` produces a system that
|
|
> won't boot and you're restoring a snapshot on day one.
|
|
|
|
From here `hermes update` self-updates in place and
|
|
`sudo nixos-rebuild switch --flake /etc/nixos#hermes` is a durable change to
|
|
the machine. Neither needs anything from you.
|
|
|
|
---
|
|
|
|
## SSH into the guest
|
|
|
|
```bash
|
|
hermes-vm ssh # interactive shell
|
|
hermes-vm ssh systemctl status # run a command and exit
|
|
hermes-vm console # serial console, for when SSH is down
|
|
```
|
|
|
|
This goes over **vsock**, not the network. There is no listening SSH port
|
|
exposed anywhere, and the guest needs no inbound network path for you to
|
|
reach it. If the guest has broken its own networking entirely, `hermes-vm ssh`
|
|
still works — `console` is the fallback for when it's broken its SSH too.
|
|
|
|
Under the hood it's `ssh -o ProxyCommand='socat - VSOCK-CONNECT:77:22'`, if
|
|
you'd rather drive it by hand.
|
|
|
|
---
|
|
|
|
## Bootstrapping Hermes's identity
|
|
|
|
The seed lives in `guest/hermes/`:
|
|
|
|
```
|
|
guest/hermes/
|
|
home/
|
|
SOUL.md -> ~/.hermes/SOUL.md
|
|
config.yaml -> ~/.hermes/config.yaml
|
|
memories/USER.md -> ~/.hermes/memories/USER.md
|
|
skills/nixos-self-config/SKILL.md -> ~/.hermes/skills/...
|
|
workspace/
|
|
AGENTS.md -> ~/workspace/AGENTS.md
|
|
```
|
|
|
|
Edit these before building the image and the agent wakes up already knowing
|
|
who it is, how you like to be spoken to, and how to rebuild its own system
|
|
without bricking itself.
|
|
|
|
**Seeding happens on first boot only, and this is deliberate.** Hermes
|
|
rewrites `SOUL.md` and its memories as it learns — that's the product. If the
|
|
seed were re-applied on every rebuild it would silently delete everything the
|
|
agent had worked out about you. So the copy is guarded by a marker file and
|
|
never repeats.
|
|
|
|
Two consequences:
|
|
|
|
- Editing `guest/hermes/` after first boot changes nothing by itself. Use
|
|
`hermes-ctl reseed` when you actually want to overwrite.
|
|
- The seed is the *starting* identity, not the current one. To see what
|
|
Hermes has made of itself, `hermes-ctl soul`.
|
|
|
|
### Machine facts vs identity
|
|
|
|
`/etc/hermes/ENVIRONMENT.md` is generated from `guest/common.nix` on every
|
|
rebuild and describes the machine: what's reachable, where the store is, what
|
|
persists. It's always accurate and there's no point editing it.
|
|
|
|
`SOUL.md` is identity and belongs to the agent. The seeded version points at
|
|
`ENVIRONMENT.md` rather than repeating it, so the two can't drift.
|
|
|
|
### Folding what it learned back into the seed
|
|
|
|
```bash
|
|
hermes-ctl pull ./hermes-home # excludes .env, sessions, state.db
|
|
diff -ru guest/hermes/home ./hermes-home
|
|
```
|
|
|
|
Worth doing occasionally. If the agent has written something genuinely good
|
|
into its own SOUL.md, you want that in the seed before you ever rebuild from
|
|
scratch.
|
|
|
|
---
|
|
|
|
## Managing Hermes from the host
|
|
|
|
`hermes-vm` drives the virtual machine. `hermes-ctl` drives the agent inside
|
|
it. Rename the latter with `services.hermesVm.cliName = "hermes";` if you'd
|
|
rather it felt local.
|
|
|
|
```bash
|
|
hermes-ctl soul # current identity
|
|
hermes-ctl soul edit # pull, $EDITOR, push, offer to restart
|
|
hermes-ctl memory edit # memories/USER.md
|
|
hermes-ctl conf edit # config.yaml
|
|
|
|
hermes-ctl secrets # keys, values masked
|
|
hermes-ctl secrets edit # edit ~/.hermes/.env, written 0600
|
|
hermes-ctl secrets show # values, when you really mean it
|
|
|
|
hermes-ctl status # gateway service
|
|
hermes-ctl logs # follow it
|
|
hermes-ctl restart
|
|
hermes-ctl units # find the unit name if the default is wrong
|
|
|
|
hermes-ctl reseed # overwrite identity from the seed
|
|
hermes-ctl pull [dir] # copy live identity to the host
|
|
```
|
|
|
|
Anything unrecognised is passed straight through to the `hermes` CLI in the
|
|
guest, with a TTY attached:
|
|
|
|
```bash
|
|
hermes-ctl chat
|
|
hermes-ctl update
|
|
hermes-ctl sessions list
|
|
hermes-ctl doctor
|
|
```
|
|
|
|
The `edit` variants pull the file out over vsock, open your `$EDITOR`
|
|
locally, write it back, and offer to restart the gateway. Secrets go in at
|
|
mode 0600 and never pass through the Nix store — which is the whole reason
|
|
there's no declarative option for them.
|
|
|
|
If `status` and `logs` come back empty, the gateway unit is probably named
|
|
something other than `hermes-agent`. Run `hermes-ctl units` and set
|
|
`services.hermesVm.gatewayUnit` to match; upstream has used more than one
|
|
name for it.
|
|
|
|
## Seeing what the guest is running
|
|
|
|
```bash
|
|
hermes-vm config
|
|
```
|
|
|
|
Prints the current system store path, the last ten generations, the git log
|
|
of `/etc/nixos`, and anything uncommitted. That last one matters: the agent
|
|
editing files without committing is the usual reason the config on disk and
|
|
the running system disagree.
|
|
|
|
To read or diff the config locally:
|
|
|
|
```bash
|
|
hermes-vm config-pull ./guest-live
|
|
diff -ru guest/ guest-live/
|
|
```
|
|
|
|
Expect that diff to grow. The `guest/` directory in this repo is the seed
|
|
that was written on first boot; it is deliberately not kept in sync. Once the
|
|
agent edits its config, this repo and the guest diverge permanently, and a
|
|
host rebuild will not fix a guest config problem.
|
|
|
|
---
|
|
|
|
## Letting the guest reach a service on your network
|
|
|
|
By default the guest can reach the public internet and nothing private. To
|
|
open a single endpoint — say Ollama on another machine:
|
|
|
|
```nix
|
|
services.hermesVm.lanAccess = [
|
|
{ address = "192.168.1.50"; port = 11434; } # ollama
|
|
{ address = "192.168.1.50"; port = 9000; proto = "udp"; }
|
|
];
|
|
```
|
|
|
|
Each entry punches one destination and port through the RFC1918 drop. The
|
|
guest reaches it as `192.168.1.50:11434` directly. Traffic is masqueraded, so
|
|
the other machine sees it arriving from your host's address, not the guest's.
|
|
|
|
There's also `allowLan = true`, which opens all private address space at
|
|
once. It exists, but reach for `lanAccess` instead — the whole point of this
|
|
design is that an agent which goes wrong can't go wandering.
|
|
|
|
### A service on the host itself
|
|
|
|
Different chain, different option. ComfyUI on the host, port 8188:
|
|
|
|
```nix
|
|
services.hermesVm.hostAccess = [ { port = 8188; } ];
|
|
```
|
|
|
|
**The service must be bound somewhere the guest can see.** If it listens
|
|
on `127.0.0.1` the rule won't help, because the guest arrives on the bridge
|
|
address. Bind it to `10.77.0.1` (bridge only, invisible to your LAN) or
|
|
`0.0.0.0`:
|
|
|
|
```nix
|
|
services.comfyui.listenAddress = "10.77.0.1"; # or whatever your service calls it
|
|
```
|
|
|
|
From the guest that's then `http://10.77.0.1:8188`.
|
|
|
|
---
|
|
|
|
## Exposing a service the guest is running
|
|
|
|
Say the agent has built a web app on guest port 8080. Two halves.
|
|
|
|
**Guest side.** The agent opens its own firewall, in `/etc/nixos`:
|
|
|
|
```nix
|
|
networking.firewall.allowedTCPPorts = [ 22 8080 ];
|
|
```
|
|
|
|
**Host side.** Nothing at all, if you only want to reach it from the host —
|
|
host-to-guest is outbound traffic and already permitted:
|
|
|
|
```bash
|
|
curl http://10.77.0.2:8080
|
|
```
|
|
|
|
To publish it to the wider network:
|
|
|
|
```nix
|
|
services.hermesVm.publish = [
|
|
{ hostPort = 8080; guestPort = 8080; }
|
|
];
|
|
```
|
|
|
|
That DNATs the port on your uplink to the guest and opens the matching
|
|
forward rule. `hostPort` and `guestPort` don't have to match, so you can park
|
|
an awkward guest port somewhere sensible.
|
|
|
|
A note on this one: publishing a port is the one change here that lets the
|
|
outside world start conversations with the agent's machine. Everything else
|
|
in this design is the guest reaching out. Put a reverse proxy with TLS and
|
|
auth in front of anything that isn't a toy, and keep the list short.
|
|
|
|
---
|
|
|
|
## Passing hardware to the guest
|
|
|
|
### Before reaching for this
|
|
|
|
If what you want is "the agent can use the GPU", passthrough is often the
|
|
wrong answer. It is exclusive — the host loses the card completely while the
|
|
VM runs — and it is the single largest reduction in isolation available in
|
|
this setup.
|
|
|
|
The alternative: keep Ollama or ComfyUI on the host where the GPU already
|
|
works, and open the port with `hostAccess`. The agent gets the capability
|
|
over HTTP and the device stays on your side of the boundary. That is usually
|
|
both easier and better.
|
|
|
|
Pass the hardware through when the guest genuinely needs the device itself —
|
|
CUDA in-process, a capture card, a radio, a display output.
|
|
|
|
### What passthrough costs you
|
|
|
|
A PCI device does DMA. The IOMMU is the only thing standing between a
|
|
passed-through device and host memory, which is why `passthrough.iommu` is a
|
|
hard requirement and the module refuses to build without it.
|
|
|
|
Two further constraints follow from how IOMMUs work:
|
|
|
|
- **Whole groups only.** Every device in an IOMMU group goes together. Check
|
|
yours before planning anything:
|
|
|
|
```bash
|
|
for d in /sys/kernel/iommu_groups/*/devices/*; do
|
|
echo "group $(basename "$(dirname "$(dirname "$d")")"): $(lspci -nns "$(basename "$d")")"
|
|
done | sort -V
|
|
```
|
|
|
|
If your GPU shares a group with a controller you need, you cannot pass it
|
|
cleanly. ACS override patches exist; they work by lying about isolation,
|
|
and they have no place in a setup built to contain an autonomous agent.
|
|
|
|
- **Every function.** A GPU is at least two entries — graphics and its HDMI
|
|
audio function.
|
|
|
|
### GPU
|
|
|
|
```bash
|
|
lspci -Dnn | grep -i vga # addresses and vendor:device IDs
|
|
```
|
|
|
|
```nix
|
|
services.hermesVm.passthrough = {
|
|
iommu = "amd"; # or "intel"
|
|
pci = [
|
|
{ address = "0000:01:00.0"; id = "10de:2204"; } # graphics
|
|
{ address = "0000:01:00.1"; id = "10de:1aef"; } # HDMI audio
|
|
];
|
|
};
|
|
```
|
|
|
|
Setting `id` makes vfio-pci claim the card in the initrd, before `nvidia` or
|
|
`amdgpu` can. Do this for GPUs — unbinding a live graphics driver afterwards
|
|
is unreliable, and the failure mode is a host with no display. Devices given
|
|
only an `address` are bound at boot by a oneshot service instead, which is
|
|
fine for things you swap around.
|
|
|
|
Rebuild and **reboot** — early binding happens in the initrd. Then check:
|
|
|
|
```bash
|
|
lspci -nnk -d 10de:2204 | grep "Kernel driver" # want: vfio-pci
|
|
```
|
|
|
|
On the guest side, the agent installs the drivers itself. It owns
|
|
`/etc/nixos`, so `hardware.nvidia` and `nvidia-container-toolkit` are its
|
|
problem, not yours — which is rather the point of the arrangement.
|
|
|
|
### HDMI output
|
|
|
|
A display hanging off a passed-through GPU works once the guest drives the
|
|
card; nothing extra is needed here. A capture card or HDMI grabber is just
|
|
another PCI or USB device — find it with `lspci -Dnn` or `lsusb` and use the
|
|
matching section.
|
|
|
|
### USB devices and Bluetooth
|
|
|
|
Matched by vendor and product ID:
|
|
|
|
```bash
|
|
lsusb # e.g. "Bus 001 Device 004: ID 8087:0026 Intel ..."
|
|
```
|
|
|
|
```nix
|
|
services.hermesVm.passthrough.usb = [
|
|
{ vendorId = "8087"; productId = "0026"; } # bluetooth radio
|
|
{ vendorId = "0403"; productId = "6001"; } # serial adapter
|
|
];
|
|
```
|
|
|
|
This adds an xHCI controller to the guest and forwards the matching devices.
|
|
No IOMMU requirement, no VFIO, and much milder than PCI passthrough — QEMU
|
|
mediates rather than handing over raw hardware.
|
|
|
|
Two things that catch people:
|
|
|
|
- **The host must not be using it.** For an integrated Bluetooth radio, keep
|
|
`btusb` off it: `boot.blacklistedKernelModules = [ "btusb" ];`, and don't
|
|
enable `hardware.bluetooth` on the host.
|
|
- **No hotplug.** QEMU resolves these at start, so a device that's unplugged
|
|
when the VM boots won't appear when you plug it in. `hermes-vm restart`
|
|
after plugging. Passing a whole USB *controller* through as a PCI device
|
|
gets you real hotplug for anything in its ports, at the cost of that
|
|
controller being exclusive to the guest.
|
|
|
|
### When it doesn't start
|
|
|
|
`hermes-vm console` first. Then, in rough order of likelihood: the device
|
|
didn't bind to vfio-pci (`lspci -nnk`), the IOMMU isn't on (`dmesg | grep -i
|
|
iommu`), or something in the group is still held by the host. The module
|
|
lifts `LimitMEMLOCK` for you when PCI passthrough is configured, since VFIO
|
|
pins the whole guest RAM allocation and the default limit otherwise produces
|
|
a memorably unhelpful error.
|
|
|
|
## Recovery
|
|
|
|
Four layers, cheapest first. The first two live inside the guest and cover
|
|
almost every real mistake.
|
|
|
|
```bash
|
|
# The agent's own config history
|
|
hermes-ctl soul # what it thinks it is now
|
|
hermes-vm ssh git -C /etc/nixos log --oneline
|
|
|
|
# Previous NixOS generations, also in the guest's boot menu
|
|
hermes-vm ssh sudo nixos-rebuild switch --rollback
|
|
|
|
# Snapshot of the state disk
|
|
hermes-vm snapshots
|
|
hermes-vm rollback state-20260914-1400
|
|
|
|
# Restore from backup
|
|
hermes-vm backups
|
|
```
|
|
|
|
### Snapshots
|
|
|
|
Hourly by default, keeping 48. Only the state disk — the system disk isn't
|
|
snapshotted because `setup.sh` rebuilds it from the flake in minutes.
|
|
|
|
The host fsfreezes the guest through qemu-guest-agent around each one, so the
|
|
snapshot captures a consistent filesystem rather than the equivalent of
|
|
pulling the power. Takes milliseconds; the VM keeps running.
|
|
|
|
`hermes-vm rollback` stops the VM, moves the current state subvolume aside as
|
|
`state.old-<timestamp>`, and restores the snapshot in its place. Nothing is
|
|
deleted, so a mistaken rollback is itself reversible. Clean the `state.old-*`
|
|
directories up yourself once you're happy.
|
|
|
|
**Keep retention tighter than you would on ZFS.** The state image is
|
|
nodatacow, and every live snapshot reintroduces copy-on-write for the blocks
|
|
it still references. Performance sags while snapshots exist and recovers as
|
|
they're pruned.
|
|
|
|
### Backups
|
|
|
|
```nix
|
|
services.hermesVm.backup = {
|
|
enable = true;
|
|
repository = "sftp:backup@nas:/hermes";
|
|
passwordFile = "/var/lib/secrets/hermes-restic";
|
|
};
|
|
```
|
|
|
|
This backs up the guest's *state*, not its disk. Files are pulled over the
|
|
SSH channel that already exists and streamed into restic.
|
|
|
|
Two reasons it works that way. The backup stays portable — it restores onto
|
|
any filesystem, or a different machine entirely, and doesn't care that you're
|
|
on btrfs today. And it avoids loop-mounting a guest-written filesystem on the
|
|
host, which would mean the host kernel parsing data the agent controls. That
|
|
would undo a fair bit of what the rest of this design is for.
|
|
|
|
If the repository is on your LAN rather than the internet, note this runs on
|
|
the *host*, so it needs no `lanAccess` entry — the guest never touches it.
|
|
|
|
### When the guest won't boot at all
|
|
|
|
The system disk is disposable by design. Re-run `setup.sh`, write a fresh
|
|
image to the system disk, and leave the state disk alone — Hermes comes back
|
|
with its identity, memories, workspace and NixOS config intact. That is the
|
|
intended repair, not a last resort.
|
|
|
|
---
|
|
|
|
## Nix flavour
|
|
|
|
The guest runs **Determinate Nix**, nixpkgs pinned to FlakeHub
|
|
(`f/NixOS/nixpkgs/0.1`, rolling; use `0` for current stable).
|
|
|
|
- `/etc/nix/nix.conf` belongs to Determinate and must not be edited. Custom
|
|
settings go in `/etc/nix/nix.custom.conf`, written from `guest/common.nix`.
|
|
There is deliberately no `nix.settings` block.
|
|
- No `nix.gc` either — `determinate-nixd` runs the collector.
|
|
- Lazy trees are on by default, which makes the agent's flake evaluations
|
|
much cheaper. That matters more than it sounds when something re-evaluates
|
|
nixpkgs on a schedule.
|
|
- Neither flake uses `inputs.nixpkgs.follows` for `determinate`. Determinate
|
|
warn it causes FlakeHub Cache misses.
|
|
|
|
Substituter order in the guest: FlakeHub Cache, your host cache,
|
|
`cache.nixos.org`. If FlakeHub Cache goes quiet, the login has expired — run
|
|
`determinate-nixd login` again.
|
|
|
|
The host stays on stock Nix. Nothing here requires changing that.
|
|
|
|
---
|
|
|
|
## What the guest can and cannot reach
|
|
|
|
| | Default |
|
|
|---|---|
|
|
| Public internet | Yes — it needs this for model APIs and messaging |
|
|
| Host Nix cache | Yes, one port, `hostCache.enable = false` closes it |
|
|
| Anything else on the host | No, unless in `hostAccess` |
|
|
| Your LAN and other VMs | No, unless in `lanAccess` |
|
|
| Inbound from the network | No, unless in `publish` |
|
|
| Host hardware | None, unless in `passthrough` |
|
|
|
|
The guest owns its kernel and its config, so nothing inside it is a security
|
|
boundary. The nftables rules in `modules/host.nix` are. QEMU runs as an
|
|
unprivileged `hermes-vm` user with its own seccomp sandbox and systemd
|
|
hardening, with a thin device model: virtio-blk, virtio-net, vsock, RNG,
|
|
serial. No virtiofs. `vhost-net` is off deliberately — it would move packet
|
|
handling into the host kernel in exchange for throughput one agent doesn't
|
|
need.
|
|
|
|
The host cache does let the guest read every store path on the host. Store
|
|
paths are world-readable anyway, so this isn't new exposure, but don't bake
|
|
secrets into your host's store.
|
|
|
|
---
|
|
|
|
## Known rough edges
|
|
|
|
- The Determinate NixOS module's option names for custom Nix settings aren't
|
|
published — only the nix-darwin `determinateNix.customSettings` equivalent
|
|
is. So `guest/common.nix` writes `/etc/nix/nix.custom.conf` via
|
|
`environment.etc`. If a future module version claims that file, you'll get
|
|
a conflict; switch to its option then.
|
|
- `determinate-nixd login` is manual. There's no way to bake the credential
|
|
into the image without parking a token somewhere it shouldn't be.
|
|
- `MemoryDenyWriteExecute = true` is correct under KVM but is the first thing
|
|
to flip if QEMU refuses to start.
|
|
- `pkgs.OVMF.firmware` / `pkgs.OVMF.variables` are current attribute names;
|
|
older nixpkgs put these under `pkgs.OVMF.fd` in an `FV/` subdirectory.
|
|
- PCI passthrough needs a reboot to take effect, because early vfio binding
|
|
happens in the initrd.
|
|
- Snapshots need btrfs. Everything else works on any filesystem; without it
|
|
you rely on backups alone for recovery.
|
|
- `state.old-*` directories left by `hermes-vm rollback` are not cleaned up
|
|
automatically, on purpose. Delete them yourself once you're sure.
|
|
- First boot builds a lot. Give it time.
|
|
|
|
---
|
|
|
|
## Migrating an existing (e.g. Docker-based) Hermes deployment
|
|
|
|
If you already have a running Hermes with real data (`HERMES_HOME` pointing
|
|
at some directory full of `SOUL.md`, `memories/`, `sessions/`, `state.db`,
|
|
etc.), see `migrate-docker-hermes.sh`. It copies — never moves or deletes —
|
|
the source data, so the original stays a safe fallback until you've verified
|
|
the new VM.
|
|
|
|
The module's `services.hermesVm.hermesHome` option lets `hermes-ctl` (and
|
|
your own `HERMES_HOME` export inside the guest) point at wherever that
|
|
existing flat layout lands, instead of the default seeded `~/.hermes`.
|