Files
nixos-setup/modules/core
hermenix 26856003d4 llama-cpp: add CUDA support and bigbox1 RTX 3060 service config
- Extend pkgs/llama-cpp to support CUDA backend (backend="cuda") for NVIDIA GPUs
- Add llama-cpp-cuda variant to modules/core/default.nix overlay
- Create hosts/hl-bigbox1/services/llama-cpp.nix with RTX 3060 tuning:
  * gpuLayers=30 (conserve 12GB VRAM for context)
  * parallel=2 (handle concurrent requests on smaller VRAM)
  * Same Qwen3.8-27B + DFlash2 drafter as fwdesk
- Wire service into bigbox1 services/default.nix
- Disabled by default (weights download manual)
2026-11-11 16:02:41 +00:00
..
2026-10-21 09:34:30 -04:00
2026-09-21 14:38:16 -04:00
2024-08-18 21:39:19 -04:00
2026-10-21 09:34:30 -04:00
2026-06-01 13:32:35 -04:00