15 KiB
Plan: Package antirez/ds4 (DwarfStar) as a Nix package
This document is a self-contained implementation plan for adding the
antirez/ds4 inference engine ("DwarfStar")
to this repo's custom package set under pkgs/ds4. It is written so another
agent (or future me) can pick it up cold.
Goal
A single, backend-parameterized derivation at pkgs/ds4/default.nix that
builds ds4 from source for a chosen GPU backend, wired into the repo's overlay.
Backends in scope:
- ROCm (
make strix-halo, AMD Strix Halo /gfx1151). - CUDA (
make cuda-generic, ormake cuda CUDA_ARCH=sm_NNfor an explicit arch). - CPU (
make cpu) as the safe default and cheapest smoke build.
(Which backend is installed where is intentionally deferred to the final Host wiring & deployment section.)
Decisions already made by the repo owner:
- Parameterize the package by target/backend (one
default.nix, selected via abackendargument). - Build per host — i.e. compile natively with
-march=native/-mcpu=native(the upstream Makefile default). We are not pinning a portable baseline-march. This means the resulting store paths are CPU-specific; that's acceptable here. (Implication: don't expect these variants to be cache-shared across differing CPUs, and the CUDA/ROCm variants generally must be built on a machine with the toolchain anyway.) - Track
main— pin the currentmainHEAD commit (below). No upstream releases/tags exist.
Upstream facts (verified from the repo on 2026-06-30)
- License: MIT.
- Pinned commit (
mainHEAD at planning time):80ebbc396aee40eedc1d829222f3362d10fa4c6c - Build system: hand-written
Makefile, noinstalltarget. Languages: C (C99), CUDA, Objective-C (Metal, macOS only), Metal shaders, some C++. - Binaries produced (all backends):
ds4,ds4-server,ds4-bench,ds4-eval,ds4-agent. - Metal is macOS-only and irrelevant to our NixOS Linux hosts.
Makefile behavior that matters for packaging
From the upstream Makefile (Linux branch, i.e. uname -s != Darwin):
CC ?= cc
NATIVE_CPU_FLAG ?= -march=native # Linux; -mcpu=native on Darwin
CFLAGS ?= -O3 -ffast-math -g $(NATIVE_CPU_FLAG) -Wall -Wextra -std=c99
CFLAGS += -D_GNU_SOURCE -fno-finite-math-only
LDLIBS ?= -lm -pthread
# CUDA
CUDA_HOME ?= /usr/local/cuda
NVCC ?= $(CUDA_HOME)/bin/nvcc
CUDA_ARCH ?= # empty unless overridden
NVCCFLAGS ?= -O3 -g -lineinfo --use_fast_math $(NVCC_ARCH_FLAGS) \
-Xcompiler $(NATIVE_CPU_FLAG) -Xcompiler -pthread
CUDA_LDLIBS ?= -lm -Xcompiler -pthread \
-L$(CUDA_HOME)/targets/sbsa-linux/lib -L$(CUDA_HOME)/lib64 \
-lcudart -lcublas
# ROCm
HIPCC ?= $(shell command -v hipcc 2>/dev/null || echo /opt/rocm/bin/hipcc)
ROCM_ARCH ?= gfx1151
ROCM_CFLAGS ?= -O3 -ffast-math -g -fno-finite-math-only -pthread \
-D__HIP_PLATFORM_AMD__ -Wno-unused-command-line-argument \
--offload-arch=$(ROCM_ARCH)
ROCM_LDLIBS ?= -lm -pthread -lhipblas -lhipblaslt
Relevant phony targets (Linux):
make cuda-generic→$(MAKE) -B ds4 ds4-server ds4-bench ds4-eval ds4-agent CUDA_ARCH=nativemake cuda-spark→ same butCUDA_ARCH=(empty; DGX Spark/GB10)make cuda CUDA_ARCH=sm_NN→ explicit arch (errors ifCUDA_ARCHempty)make strix-halo(aliasmake rocm) → rebuilds the 5 binaries withCORE_OBJS="ds4.o ds4_distributed.o ds4_ssd.o ds4_rocm.o",CFLAGS="… -DDS4_ROCM_BUILD",DS4_LINK="$(HIPCC) $(ROCM_CFLAGS)",DS4_LINK_LIBS="$(ROCM_LDLIBS)".make cpu→ CPU-only reference build (-DDS4_NO_GPU), no GPU libs. Useful as a cheap CI/portable smoke build.make test→ buildsds4_test,ds4_agent_test,ds4-eval, a dot-product test, and runs them. The CUDA path links tests withnvcc.
Key gotchas:
- No
installtarget → we must write our owninstallPhasecopying the 5 binaries into$out/bin. -march=nativeis on by default. Per decision #2 we keep it (build per host). For the optionalcpusmoke variant built on the remote builder, the resulting binary is CPU-specific to the builder — fine for a smoke test, but do not distribute acpuvariant to heterogeneous machines without overridingNATIVE_CPU_FLAG.- CUDA library paths are hardcoded to
/usr/local/cuda/...and ansbsa-linux(aarch64) target dir. Under Nix we must pointCUDA_HOME/NVCCat the nixpkgs CUDA toolkit and likely overrideCUDA_LDLIBSso the-Lpaths resolve inside the store. Expect to setNVCC,CUDA_HOME, and a correctedCUDA_LDLIBSviamakeFlags. - ROCm finds
hipccviacommand -v hipcc, so havingrocmPackages.clr(provideshipcc) onPATH(nativeBuildInputs) is enough.ROCM_ARCHdefaults togfx1151(exactly the Strix Halo target). The backend uses rocWMMA; on Ubuntu upstream notes the distro packages missrocwmma/internal/headers (seeSTRIXHALO.md). Under Nix,rocmPackages.rocwmmashould provide the complete header tree — verify during implementation (ROCm phase). -ffast-math/-fno-finite-math-onlyare upstream defaults; leave them. Standardfortify/stripNix hardening should be fine but watch for warnings; can disable hardening flags if the build complains.- Runtime: on Linux the binaries are self-contained (Metal
.metalruntime files are macOS-only). GGUF model weights anddownload_model.share a runtime concern, NOT packaged. The agent/server accept--chdirand-m <path>for model selection.
Repo conventions (how packages are added here)
See pkgs/ABOUTME.md. The pattern is:
- Create
pkgs/<name>/default.nix(standard derivation;callPackage-style function taking its inputs as arguments). - Register in the overlay in
modules/core/default.nix:nixpkgs.overlays = [ (final: _prev: { grok-cli = final.callPackage ../../pkgs/grok-cli { }; # add ds4 entries here }) # ... ]; - Reference as
pkgs.<name>anywhere in the flake / host modules. - Optionally expose as a flake output under
packagesinflake.nix(see the existinggrok-cli = lib.pkgs.${system}.grok-cli;line).
pkgs/grok-cli/default.nix is the existing reference example (but note it is a
prebuilt binary using stdenvNoCC + fetchurl; ds4 is build-from-source and
must use stdenv.mkDerivation + fetchFromGitHub).
Implementation phases
Phase 1 — Source pinning
- Write the
fetchFromGitHubsrcwith:owner = "antirez"; repo = "ds4";rev = "80ebbc396aee40eedc1d829222f3362d10fa4c6c";(currentmainHEAD; bump as desired since we trackmain).hash— start withlib.fakeHash, run a build, copy the real SRI hash from the error. (Equivalent to thenix-prefetch/fakeHashflow noted inpkgs/ABOUTME.md.)
Phase 2 — Core derivation skeleton (pkgs/ds4/default.nix)
Parameterized function. Sketch (to be refined during implementation):
{ lib
, stdenv
, fetchFromGitHub
, rocmPackages
, cudaPackages
# backend selects the build target / toolchain
, backend ? "cpu" # one of "cpu" | "rocm" | "cuda"
, cudaArch ? null # e.g. "sm_89"; null => cuda-generic (CUDA_ARCH=native)
, rocmArch ? "gfx1151"
}:
assert lib.elem backend [ "cpu" "rocm" "cuda" ];
stdenv.mkDerivation (finalAttrs: {
pname = "ds4";
version = "unstable-2026-06-17"; # date of pinned commit
src = fetchFromGitHub { /* Phase 1 */ };
enableParallelBuilding = true;
# nativeBuildInputs / buildInputs / makeFlags / buildPhase vary by backend
# (ROCm / CUDA / CPU phases).
installPhase = ''
runHook preInstall
install -Dm755 -t "$out/bin" ds4 ds4-server ds4-bench ds4-eval ds4-agent
runHook postInstall
'';
meta = {
description = "DeepSeek V4 Flash/PRO local inference engine (DwarfStar)";
homepage = "https://github.com/antirez/ds4";
license = lib.licenses.mit;
sourceProvenance = [ lib.sourceTypes.fromSource ];
platforms = lib.platforms.linux; # Metal/macOS out of scope
mainProgram = "ds4";
};
})
- Implement the skeleton with a
let buildTarget = { cpu = "cpu"; rocm = "strix-halo"; cuda = if cudaArch == null then "cuda-generic" else "cuda"; }style mapping, and a per-backendmakeFlags/buildFlags. - Keep
-march=native(do not overrideNATIVE_CPU_FLAG) per decision #2.
Phase 3 — ROCm backend
nativeBuildInputs:rocmPackages.clr(provideshipcc).buildInputs:rocmPackageslibs needed byds4_rocm.cuand link flags (-lhipblas -lhipblaslt): at leastclr,hipblas,hipblaslt,rocblas,rocwmma, plus header-onlyhipcub/rocprimas required.- Build:
make strix-halo(or pass the equivalentmakeFlags).ROCM_ARCHdefaults togfx1151; allow override viarocmArcharg. - Verify rocWMMA headers: ensure
rocwmma/internal/*resolves from the nixpkgsrocwmmapackage (upstreamSTRIXHALO.mdwarns the Ubuntu pkgs are incomplete). If missing, add the include path or the right package. - May need
ROCM_PATH/include/lib flags sohipccfinds the store-located ROCm libs; adjustROCM_CFLAGS/ROCM_LDLIBSviamakeFlagsif needed.
Phase 4 — CUDA backend
nativeBuildInputs:cudaPackages.cuda_nvcc.buildInputs:cudaPackages.cuda_cudart,cudaPackages.libcublas.- Override the hardcoded CUDA paths via
makeFlags:CUDA_HOME = .../NVCC = "${cudaPackages.cuda_nvcc}/bin/nvcc",- a corrected
CUDA_LDLIBSthat points-Lat the nixpkgs cudart/cublaslibdirs instead of/usr/local/cuda/...andsbsa-linux(that path is aarch64; an x86_64 target should uselib64/store libs).
- Build target:
cuda-generic(CUDA_ARCH=native) by default, orcudawith an explicitcudaArch(preferred for reproducibility / when the build host differs from the GPU host). - CUDA is unfree → the package's closure is unfree. Keep
meta.license = lib.licenses.mitfor the source itself, but be awareallowUnfreeis required to build/realize this variant.
Phase 5 — CPU backend (default)
make cpu; no GPU inputs. Cheapest build, good for a portable smoke test and as the safe defaultbackend = "cpu".- Add
doInstallCheckrunning$out/bin/ds4 --help(only meaningful for the CPU variant; GPU variants may require a device at runtime, so guard or skip the check for them).
Phase 6 — Overlay wiring (modules/core/default.nix)
Add to the existing nixpkgs.overlays first entry:
(final: _prev: {
grok-cli = final.callPackage ../../pkgs/grok-cli { };
ds4 = final.callPackage ../../pkgs/ds4 { }; # cpu default
ds4-rocm = final.callPackage ../../pkgs/ds4 { backend = "rocm"; };
ds4-cuda = final.callPackage ../../pkgs/ds4 { backend = "cuda"; };
})
- Confirm
rocmPackages/cudaPackagesresolve viacallPackage(they are top-level attrs in nixpkgs, so they will).
Phase 7 — Flake output (optional)
- In
flake.nixpackagesoutput, alongsidegrok-cli, optionally expose:ds4 = lib.pkgs.${system}.ds4;(CPU variant — buildable on any system). The GPU variants are awkward as generic flake outputs (need toolchains), so leave them overlay-only unless wanted.
Phase 8 — Validation
Order by cheapness / portability:
nix build .#ds4(or via a throwawaynix-build -E) — CPU variant, builds on the default builder. Resolve thesrchash here (Phase 1).nix buildthe ROCm variant (build on a machine with the ROCm toolchain). Smoke:ds4 --help.nix buildthe CUDA variant (build on a machine with the CUDA toolchain). Smoke:ds4 --help.nixpkgs-fmtthe new/edited Nix files (repo uses it as a pre-commit hook and devshell formatter).- Update
pkgs/ABOUTME.mdcontents table with the newds4entries.
Phase 9 — Docs / cleanup
- Add a short header comment in
pkgs/ds4/default.nix(likegrok-cli) explaining backends, the-march=native"build per host" choice, themain-tracking pin, and how to bump (rev+ refreshhash). - Delete or keep this
plan.md(suggest keeping until implementation lands, then optionally remove).
Risks / unknowns to resolve during implementation
- CUDA path overrides: the Makefile's
CUDA_LDLIBShardcodestargets/sbsa-linux/lib(aarch64) +/usr/local/cuda. On x86_64 nixpkgs this must be replaced. Highest-risk part of the CUDA backend. - rocWMMA internal headers under nixpkgs (ROCm phase) — verify availability.
- Hardening flags:
-ffast-math+ Nix default hardening may produce warnings; if the build breaks, selectively disable (hardeningDisable). - Build-per-host vs. binary cache: with
-march=native, store paths are CPU-specific; ensure builds happen on (or for) the right host.
Host wiring & deployment (do this last)
All host-specific assignment and system enablement lives here, after the package itself builds and is wired into the overlay.
Hosts are defined in flake.nix under nixosConfigurations. System-level
packages for a host are typically added in the host's module (or a shared
module it imports); locate where each host declares environment.systemPackages
/ module package lists and add the appropriate pkgs.ds4-* there.
hl-fwdesk — ROCm
- Module:
hosts/hl-fwdesk.nix(usesframework-desktop-amd-ai-max-300-series); the box is AMD Ryzen AI Max 300 (Strix Halo,gfx1151) — exactly the ROCm/make strix-halotarget. - Add
pkgs.ds4-rocmto the host's package list. - Confirm ROCm is enabled on the host (drivers,
hardware.graphics/ ROCm runtime) before expecting the GPU path to work. - System enablement required to run it (not part of the package), per
STRIXHALO.md:/dev/kfd+ DRM render-node access (user inrender/videogroups).- GTT memory kernel params so the full 128 GB is GPU-visible:
amd_iommu=off amdgpu.gttsize=126976 ttm.pages_limit=32505856 ttm.page_pool_size=32505856(inboot.kernelParams).
hl-bigbox1 — CUDA
- Module:
hosts/hl-bigbox1.nix. Host has an NVIDIA GPU. - Add
pkgs.ds4-cudato the host's package list. - Confirm the exact NVIDIA GPU/arch to decide between
cuda-generic(CUDA_ARCH=native) and a pinnedCUDA_ARCH=sm_NN(e.g.sm_86/sm_89/sm_90). Pinning is preferred when the build host differs from the GPU host. - Ensure
nixpkgs.config.allowUnfree(and CUDA driver/hardware.graphicssupport) is set for this host — the CUDA closure is unfree. - The configured remote builder
root@bigbox1.lab.internalis x86_64 and may double as the native CUDA build host (convenient, avoids cross-CPU-march=nativemismatches).