doc clean
This commit is contained in:
1 parent
c2a7f94821
commit
30427e3975
34 files changed
+1119
-1758
No files matched your search
@@ -0,0 +1,124 @@
|
||||
# Hardware passthrough & virtual devices
|
||||
|
||||
## Before reaching for passthrough
|
||||
|
||||
If the goal is "the agent can use the GPU", passthrough is often wrong. It is
|
||||
exclusive (host loses the card while the VM runs) and is the largest isolation
|
||||
reduction in this setup.
|
||||
|
||||
Prefer: keep Ollama/ComfyUI on the host, open the port with `hostAccess`
|
||||
([NETWORK.md](NETWORK.md#guest--a-service-on-the-host)). Pass hardware through
|
||||
when the guest genuinely needs the device — CUDA in-process, capture card,
|
||||
radio, display output.
|
||||
|
||||
## What PCI passthrough costs
|
||||
|
||||
A PCI device does DMA. The IOMMU is the only thing between it and host memory,
|
||||
so `passthrough.iommu` is required; the module refuses to build without it.
|
||||
|
||||
- **Whole IOMMU groups only.** Check before planning:
|
||||
|
||||
```bash
|
||||
for d in /sys/kernel/iommu_groups/*/devices/*; do
|
||||
echo "group $(basename "$(dirname "$(dirname "$d")")"): $(lspci -nns "$(basename "$d")")"
|
||||
done | sort -V
|
||||
```
|
||||
|
||||
ACS override patches lie about isolation — not appropriate here.
|
||||
|
||||
- **Every function.** A GPU is at least graphics + HDMI audio.
|
||||
|
||||
## GPU
|
||||
|
||||
```bash
|
||||
lspci -Dnn | grep -i vga
|
||||
```
|
||||
|
||||
```nix
|
||||
services.hermenix.passthrough = {
|
||||
iommu = "amd"; # or "intel"
|
||||
pci = [
|
||||
{ address = "0000:01:00.0"; id = "10de:2204"; } # graphics
|
||||
{ address = "0000:01:00.1"; id = "10de:1aef"; } # HDMI audio
|
||||
];
|
||||
};
|
||||
```
|
||||
|
||||
Setting `id` makes vfio-pci claim the card in the initrd before `nvidia` /
|
||||
`amdgpu`. Do this for GPUs. Address-only devices bind at boot via a oneshot
|
||||
instead.
|
||||
|
||||
Rebuild and **reboot**, then:
|
||||
|
||||
```bash
|
||||
lspci -nnk -d 10de:2204 | grep "Kernel driver" # want: vfio-pci
|
||||
```
|
||||
|
||||
Guest-side drivers are the agent's problem (`/etc/nixos`).
|
||||
|
||||
## USB / Bluetooth
|
||||
|
||||
```bash
|
||||
lsusb
|
||||
```
|
||||
|
||||
```nix
|
||||
services.hermenix.passthrough.usb = [
|
||||
{ vendorId = "8087"; productId = "0026"; }
|
||||
{ vendorId = "0403"; productId = "6001"; }
|
||||
];
|
||||
```
|
||||
|
||||
QEMU mediates (no IOMMU/VFIO). Host must not be using the device (`btusb`
|
||||
blacklist, no `hardware.bluetooth` for an integrated radio). No hotplug —
|
||||
`hermenix vm restart` after plugging. Whole USB controller as PCI gives real
|
||||
hotplug at the cost of exclusivity.
|
||||
|
||||
## When the VM won't start
|
||||
|
||||
`hermenix vm console` first. Then: device not on vfio-pci (`lspci -nnk`),
|
||||
IOMMU off (`dmesg | grep -i iommu`), or something in the group still held by
|
||||
the host. The module raises `LimitMEMLOCK` when PCI passthrough is configured.
|
||||
|
||||
---
|
||||
|
||||
## Virtual devices
|
||||
|
||||
`services.hermenix.devices` attaches optional QEMU devices (no host hardware):
|
||||
|
||||
| Option | Default | Attaches |
|
||||
| ------------------------- | ---------- | ----------------------------------------------------- |
|
||||
| `devices.rng.enable` | `true` | virtio entropy |
|
||||
| `devices.vsock.enable` | `true` | host↔guest vsock (`ctl`, backups) |
|
||||
| `devices.video.enable` | `false` | virtual display / render node |
|
||||
| `devices.video.model` | `"virtio"` | `virtio` (render node) or legacy VGA |
|
||||
| `devices.balloon.enable` | `false` | virtio memory balloon |
|
||||
| `devices.watchdog.enable` | `false` | `i6300esb`/`ib700` |
|
||||
| `devices.input.enable` | `false` | virtio keyboard/tablet |
|
||||
| `devices.audio.enable` | `false` | `intel-hda`/`virtio-sound` (see caveat) |
|
||||
| `devices.tpm.enable` | `false` | TPM 2.0 via swtpm |
|
||||
| `devices.cdroms` | `[]` | read-only CD images |
|
||||
| `devices.extraDisks` | `[]` | extra raw disks |
|
||||
| `devices.usbController` | `"auto"` | `qemu-xhci` iff `passthrough.usb` set |
|
||||
|
||||
`extraQemuArgs` appends raw QEMU args (unvalidated). Prefer typed options.
|
||||
|
||||
### Render node
|
||||
|
||||
```nix
|
||||
services.hermenix.devices.video.enable = true;
|
||||
```
|
||||
|
||||
Gives `/dev/dri/card0` and `/dev/dri/renderD128` (`virtio` model only).
|
||||
Console stays serial.
|
||||
|
||||
### TPM
|
||||
|
||||
`devices.tpm.enable = true` starts swtpm; state under
|
||||
`/var/lib/hermes-vm-tpm` by default.
|
||||
|
||||
### Audio caveat
|
||||
|
||||
`hermes-vm.service` is hardened (`ProtectHome`, `PrivateTmp`, dedicated user)
|
||||
and cannot reach PipeWire without loosening that. Leave audio off unless you
|
||||
mean to relax the unit.
|
||||
Reference in new issue
Block a user