llama-cpp: llama-server with DFlash2 speculative decoding for gfx1151

v0.8.0 is the first upstream release with DFlash2 drafters; nixpkgs has 0.6.0 (unstable) and an older tree (stable), so the engine is built here. HIP build for Strix Halo. The NixOS module exposes llama-server behind skyg.nixos.server.services.llama-cpp; fwdesk wires it to Qwen3.8-27B + its DFlash2 drafter, disabled until the weights are on disk.
This commit is contained in:
hermenix committed 2026-11-11 16:01:32 +00:00
1 parent 62c41a6f8e
commit ce92096042
9 files changed
+392

No files matched your search

+1
View File
@@ -12,6 +12,7 @@ Packages here are exposed via an overlay defined in `modules/core/default.nix`.
| `buzz-desktop/` | Block Buzz desktop (Type-2 AppImage from `block/buzz`). Wrapped with `appimageTools.wrapType2` plus extra FHS libs (`elfutils`, `zstd`, WebKit/tray, GStreamer good/bad) so it runs on NixOS. Exposed as `pkgs.buzz-desktop` (`buzz-desktop` binary + desktop entry / icons). |
| `buzz-cli/` | Buzz CLI (`buzz`, `crates/buzz-cli` of `block/buzz`), built with `rustPlatform.buildRustPackage` from the same tag as `buzz-desktop`. Hermes's Buzz platform shells out to it. Bump with `update.sh` (sets the source hash, resets `cargoHash` to `lib.fakeHash`; build once and paste the `got:` hash). Robots call it with `pkgs.callPackage` (their nixpkgs has no host overlay). |
| `delta-editor/` | Delta Editor (Zed Industries' AI-native editor, `delta.dev`) prebuilt tarball. No stable URL exists — the release API returns a short-lived signed R2 link — so `src` is a fixed-output derivation that resolves the API at build time. Interpreter/RPATH patched (`--force-rpath`, `dontPatchELF`), dlopen'd graphics libs (`libglvnd`, `vulkan-loader`, `wayland`) added via wrapper, `XKB_CONFIG_ROOT` set for the bundled libxkbcommon. Exposed as `pkgs.delta-editor` (`delta-editor` binary + desktop entry / icons); named `-editor` to avoid clashing with the `delta` git diff pager. |
| `llama-cpp/` | llama.cpp pinned to v0.8.0, the first release with DFlash2 drafters (`--spec-type draft-dflash`). nixpkgs only carries v0.6.0 in the unstable input and an older tree in the stable one, so the server is built here. HIP build for Strix Halo (`CMAKE_HIP_ARCHITECTURES=gfx1151`, ROCm clang++, `autoPatchelfHook`). Backend-parameterized: `pkgs.llama-cpp-halo` (rocm, default) and `pkgs.llama-cpp-halo-cpu` (same tree, no GPU backend). Web UI is off — it downloads prebuilt assets at build time, which the sandbox cannot do. Model weights are not packaged. |
## Adding a New Package