4.3 KiB
Hardware passthrough & virtual devices
Before reaching for passthrough
If the goal is "the agent can use the GPU", passthrough is often wrong. It is exclusive (host loses the card while the VM runs) and is the largest isolation reduction in this setup.
Prefer: keep Ollama/ComfyUI on the host, open the port with hostAccess
(NETWORK.md). Pass hardware through
when the guest genuinely needs the device — CUDA in-process, capture card,
radio, display output.
What PCI passthrough costs
A PCI device does DMA. The IOMMU is the only thing between it and host memory,
so passthrough.iommu is required; the module refuses to build without it.
-
Whole IOMMU groups only. Check before planning:
for d in /sys/kernel/iommu_groups/*/devices/*; do echo "group $(basename "$(dirname "$(dirname "$d")")"): $(lspci -nns "$(basename "$d")")" done | sort -VACS override patches lie about isolation — not appropriate here.
-
Every function. A GPU is at least graphics + HDMI audio.
GPU
lspci -Dnn | grep -i vga
services.hermenix.passthrough = {
iommu = "amd"; # or "intel"
pci = [
{ address = "0000:01:00.0"; id = "10de:2204"; } # graphics
{ address = "0000:01:00.1"; id = "10de:1aef"; } # HDMI audio
];
};
Setting id makes vfio-pci claim the card in the initrd before nvidia /
amdgpu. Do this for GPUs. Address-only devices bind at boot via a oneshot
instead.
Rebuild and reboot, then:
lspci -nnk -d 10de:2204 | grep "Kernel driver" # want: vfio-pci
Guest-side drivers are the agent's problem (/etc/nixos).
USB / Bluetooth
lsusb
services.hermenix.passthrough.usb = [
{ vendorId = "8087"; productId = "0026"; }
{ vendorId = "0403"; productId = "6001"; }
];
QEMU mediates (no IOMMU/VFIO). Host must not be using the device (btusb
blacklist, no hardware.bluetooth for an integrated radio). No hotplug —
hermenix vm restart after plugging. Whole USB controller as PCI gives real
hotplug at the cost of exclusivity.
When the VM won't start
hermenix vm console first. Then: device not on vfio-pci (lspci -nnk),
IOMMU off (dmesg | grep -i iommu), or something in the group still held by
the host. The module raises LimitMEMLOCK when PCI passthrough is configured.
Virtual devices
services.hermenix.devices attaches optional QEMU devices (no host hardware):
| Option | Default | Attaches |
|---|---|---|
devices.rng.enable |
true |
virtio entropy |
devices.vsock.enable |
true |
host↔guest vsock (ctl, backups) |
devices.video.enable |
false |
virtual display / render node |
devices.video.model |
"virtio" |
virtio (render node) or legacy VGA |
devices.balloon.enable |
false |
virtio memory balloon |
devices.watchdog.enable |
false |
i6300esb/ib700 |
devices.input.enable |
false |
virtio keyboard/tablet |
devices.audio.enable |
false |
intel-hda/virtio-sound (see caveat) |
devices.tpm.enable |
false |
TPM 2.0 via swtpm |
devices.cdroms |
[] |
read-only CD images |
devices.extraDisks |
[] |
extra raw disks |
devices.usbController |
"auto" |
qemu-xhci iff passthrough.usb set |
extraQemuArgs appends raw QEMU args (unvalidated). Prefer typed options.
Render node
services.hermenix.devices.video.enable = true;
Gives /dev/dri/card0 and /dev/dri/renderD128 (virtio model only).
Console stays serial.
TPM
devices.tpm.enable = true starts swtpm; state under
/var/lib/hermes-vm-tpm by default.
Audio caveat
hermes-vm.service is hardened (ProtectHome, PrivateTmp, dedicated user)
and cannot reach PipeWire without loosening that. Leave audio off unless you
mean to relax the unit.