v0.8.0 is the first upstream release with DFlash2 drafters; nixpkgs has 0.6.0 (unstable) and an older tree (stable), so the engine is built here. HIP build for Strix Halo. The NixOS module exposes llama-server behind skyg.nixos.server.services.llama-cpp; fwdesk wires it to Qwen3.8-27B + its DFlash2 drafter, disabled until the weights are on disk.
60 lines
3.4 KiB
Markdown
60 lines
3.4 KiB
Markdown
# modules/nixos/server/services/
|
|
|
|
Self-hosted application services. `default.nix` imports each one; every service is an
|
|
independent `skyg.nixos.server.services.<name>.enable` flag. Most run as OCI containers via
|
|
`virtualisation.oci-containers` (so they honour the Docker/Podman runtime chosen in
|
|
`common/containers/`), often backed by NFS volumes.
|
|
|
|
## Files
|
|
|
|
```
|
|
services/
|
|
├── default.nix # imports the services below
|
|
├── ai-services.nix # skyg.nixos.server.services.ai — Wyoming (whisper/piper/wake)
|
|
├── audiobookshelf.nix # skyg.nixos.server.services.audiobookshelf — audiobook/podcast server
|
|
├── jellyfin.nix # skyg.nixos.server.services.jellyfin — media server (uid/gid 7777)
|
|
├── nbot-agent.nix # skyg.nixos.server.services.nbot-agent — nbot's agent system + gh, for nbot's `system`
|
|
├── signal-cli.nix # skyg.nixos.server.services.signal-cli — signal-cli HTTP daemon (Hermes Signal bridge)
|
|
├── colibri.nix # skyg.nixos.server.services.colibri — coli serve HTTP API (GLM-5.2/OLMoE local inference)
|
|
├── llama-cpp.nix # skyg.nixos.server.services.llama-cpp — llama-server (GGUF, DFlash2 speculative decoding)
|
|
└── comfyui/ # skyg.nixos.server.services.comfyui — ComfyUI with ROCm/CUDA GPU support
|
|
```
|
|
|
|
## Notable details
|
|
|
|
- **`ai-services.nix`** openwakeword as containers and
|
|
enables NixOS-native Wyoming faster-whisper + piper. Opens 11434/10200/10300/10400.
|
|
- **`nbot-agent.nix`** adds `gh` to nbot's agent system from this repo, leaving the nbot
|
|
repo unchanged. `gh` reads `GH_TOKEN` from `agent_env`, and git over HTTPS to github.com
|
|
uses it through `gh`. Every container gets that token (chosen on purpose, 2026-11-03),
|
|
so anyone who can prompt an agent can use it or ask it to print the token.
|
|
- **`colibri.nix`** runs `coli serve` (colibrì's OpenAI-compatible API) as a native systemd
|
|
service. `package` picks the backend (`pkgs.colibri`
|
|
cpu / `pkgs.colibri-rocm`); `environmentFile` supplies `COLI_API_KEY` via agenix instead of
|
|
a CLI flag.
|
|
- **`llama-cpp.nix`** runs `llama-server` as a native systemd service. `draftModel` plus
|
|
`specType = "draft-dflash"` turns on DFlash2 speculative decoding; `cacheReuse` turns on
|
|
prefix reuse for repeated prompts (agents). Model files are read from disk, never packaged —
|
|
the unit refuses to start with a clear message when one is missing. Enabled on fwdesk
|
|
(off by default: ~19 GB of weights, first build is the ROCm package).
|
|
|
|
## Option Namespace
|
|
|
|
```
|
|
skyg.nixos.server.services.ai.enable
|
|
skyg.nixos.server.services.audiobookshelf.{enable,package,configDir,dataDir}
|
|
skyg.nixos.server.services.jellyfin.enable
|
|
skyg.nixos.server.services.nbot-agent.system
|
|
skyg.nixos.server.services.signal-cli.{enable,host,port,configDir,account,environmentFile,openFirewall}
|
|
skyg.nixos.server.services.colibri.{enable,package,model,host,port,environmentFile,openFirewall,...}
|
|
skyg.nixos.server.services.llama-cpp.{enable,package,model,draftModel,specType,specDraftNMax,cacheReuse,ctx,host,port,openFirewall,...}
|
|
skyg.nixos.server.services.comfyui.{enable,mode,port,openFirewall}
|
|
```
|
|
|
|
## Conventions
|
|
|
|
- One file per service; gate everything behind `enable` (default `false`).
|
|
- Expose an `openFirewall` option instead of opening ports unconditionally.
|
|
- Secrets (tokens, credentials) come from agenix — never hardcode them.
|
|
- See `GUIDELINES.md` in `modules/nixos/server/` for the full service-authoring pattern.
|