# Hardware passthrough & virtual devices ## Before reaching for passthrough If the goal is "the agent can use the GPU", passthrough is often wrong. It is exclusive (host loses the card while the VM runs) and is the largest isolation reduction in this setup. Prefer: keep Ollama/ComfyUI on the host, open the port with `hostAccess` ([NETWORK.md](NETWORK.md#guest--a-service-on-the-host)). Pass hardware through when the guest genuinely needs the device — CUDA in-process, capture card, radio, display output. ## What PCI passthrough costs A PCI device does DMA. The IOMMU is the only thing between it and host memory, so `passthrough.iommu` is required; the module refuses to build without it. - **Whole IOMMU groups only.** Check before planning: ```bash for d in /sys/kernel/iommu_groups/*/devices/*; do echo "group $(basename "$(dirname "$(dirname "$d")")"): $(lspci -nns "$(basename "$d")")" done | sort -V ``` ACS override patches lie about isolation — not appropriate here. - **Every function.** A GPU is at least graphics + HDMI audio. ## GPU ```bash lspci -Dnn | grep -i vga ``` ```nix services.hermenix.passthrough = { iommu = "amd"; # or "intel" pci = [ { address = "0000:01:00.0"; id = "10de:2204"; } # graphics { address = "0000:01:00.1"; id = "10de:1aef"; } # HDMI audio ]; }; ``` Setting `id` makes vfio-pci claim the card in the initrd before `nvidia` / `amdgpu`. Do this for GPUs. Address-only devices bind at boot via a oneshot instead. Rebuild and **reboot**, then: ```bash lspci -nnk -d 10de:2204 | grep "Kernel driver" # want: vfio-pci ``` Guest-side drivers are the agent's problem (`/etc/nixos`). ## USB / Bluetooth ```bash lsusb ``` ```nix services.hermenix.passthrough.usb = [ { vendorId = "8087"; productId = "0026"; } { vendorId = "0403"; productId = "6001"; } ]; ``` QEMU mediates (no IOMMU/VFIO). Host must not be using the device (`btusb` blacklist, no `hardware.bluetooth` for an integrated radio). No hotplug — `hermenix vm restart` after plugging. Whole USB controller as PCI gives real hotplug at the cost of exclusivity. ## When the VM won't start `hermenix vm console` first. Then: device not on vfio-pci (`lspci -nnk`), IOMMU off (`dmesg | grep -i iommu`), or something in the group still held by the host. The module raises `LimitMEMLOCK` when PCI passthrough is configured. --- ## Virtual devices `services.hermenix.devices` attaches optional QEMU devices (no host hardware): | Option | Default | Attaches | | ------------------------- | ---------- | ----------------------------------------------------- | | `devices.rng.enable` | `true` | virtio entropy | | `devices.vsock.enable` | `true` | host↔guest vsock (`ctl`, backups) | | `devices.video.enable` | `false` | virtual display / render node | | `devices.video.model` | `"virtio"` | `virtio` (render node) or legacy VGA | | `devices.balloon.enable` | `false` | virtio memory balloon | | `devices.watchdog.enable` | `false` | `i6300esb`/`ib700` | | `devices.input.enable` | `false` | virtio keyboard/tablet | | `devices.audio.enable` | `false` | `intel-hda`/`virtio-sound` (see caveat) | | `devices.tpm.enable` | `false` | TPM 2.0 via swtpm | | `devices.cdroms` | `[]` | read-only CD images | | `devices.extraDisks` | `[]` | extra raw disks | | `devices.usbController` | `"auto"` | `qemu-xhci` iff `passthrough.usb` set | `extraQemuArgs` appends raw QEMU args (unvalidated). Prefer typed options. ### Render node ```nix services.hermenix.devices.video.enable = true; ``` Gives `/dev/dri/card0` and `/dev/dri/renderD128` (`virtio` model only). Console stays serial. ### TPM `devices.tpm.enable = true` starts swtpm; state under `/var/lib/hermes-vm-tpm` by default. ### Audio caveat `hermes-vm.service` is hardened (`ProtectHome`, `PrivateTmp`, dedicated user) and cannot reach PipeWire without loosening that. Leave audio off unless you mean to relax the unit.