Files
nixos-setup/modules
hermenix 26856003d4 llama-cpp: add CUDA support and bigbox1 RTX 3060 service config
- Extend pkgs/llama-cpp to support CUDA backend (backend="cuda") for NVIDIA GPUs
- Add llama-cpp-cuda variant to modules/core/default.nix overlay
- Create hosts/hl-bigbox1/services/llama-cpp.nix with RTX 3060 tuning:
  * gpuLayers=30 (conserve 12GB VRAM for context)
  * parallel=2 (handle concurrent requests on smaller VRAM)
  * Same Qwen3.8-27B + DFlash2 drafter as fwdesk
- Wire service into bigbox1 services/default.nix
- Disabled by default (weights download manual)
2026-11-11 16:02:41 +00:00
..
2026-10-16 15:07:11 -04:00
2026-10-16 15:07:11 -04:00
2026-11-05 15:21:01 -05:00
2026-09-29 11:23:25 -04:00
2026-10-26 16:18:48 -04:00
…
2026-11-05 15:21:01 -05:00
2026-04-13 16:39:18 -04:00
2026-10-13 19:52:54 -04:00