ds4 glm support
This commit is contained in:
1 parent
4cc2123eff
commit
d090ebaad8
6 files changed
+40
-16
No files matched your search
+13
-1
@@ -4,6 +4,7 @@
|
||||
, makeWrapper
|
||||
, curl
|
||||
, cacert
|
||||
, python3
|
||||
, rocmPackages
|
||||
, cudaPackages
|
||||
# backend selects the build target / toolchain.
|
||||
@@ -36,6 +37,17 @@
|
||||
assert lib.elem backend [ "cpu" "rocm" "cuda" ];
|
||||
|
||||
let
|
||||
# `hf` CLI used by ds4-download-model for the large/sharded GGUF files
|
||||
# (MXFP4, PRO, GLM). huggingface-hub provides the `hf` binary; hf-xet is the
|
||||
# optional Xet backend it uses to speed up big transfers. Default python3 is
|
||||
# 3.14, so this resolves to python314Packages.{huggingface-hub,hf-xet}.
|
||||
pythonEnv = python3.withPackages (
|
||||
ps: with ps; [
|
||||
huggingface-hub
|
||||
hf-xet
|
||||
]
|
||||
);
|
||||
|
||||
# ROCm libraries the gfx kernels and link flags (-lhipblas -lhipblaslt) need.
|
||||
rocmInputs = [
|
||||
rocmPackages.clr # provides hipcc + HIP runtime
|
||||
@@ -163,7 +175,7 @@ stdenv.mkDerivation (finalAttrs: {
|
||||
# environment doesn't already set one.
|
||||
postFixup = ''
|
||||
wrapProgram "$out/bin/ds4-download-model" \
|
||||
--prefix PATH : ${lib.makeBinPath [ curl ]} \
|
||||
--prefix PATH : ${lib.makeBinPath [ curl pythonEnv ]} \
|
||||
--set-default SSL_CERT_FILE ${cacert}/etc/ssl/certs/ca-bundle.crt
|
||||
'';
|
||||
|
||||
|
||||
@@ -58,8 +58,9 @@ ds4-download-model --help # full target list
|
||||
|
||||
Pick by machine memory: `ds4f-q2` (96/128 GB), `ds4f-q4` (>=256 GB),
|
||||
`pro-*` (512 GB / distributed), `glm-*` (per-target sizes above). The small
|
||||
Flash quants download via `curl`; the MXFP4, PRO, and GLM files need the `hf`
|
||||
CLI (not bundled — `nix shell nixpkgs#huggingface-hub`).
|
||||
Flash quants download via `curl`; the MXFP4, PRO, and GLM files use the `hf`
|
||||
CLI (bundled in the package via `huggingface-hub` + `hf-xet`), which
|
||||
`ds4-download-model` puts on its PATH automatically.
|
||||
|
||||
## 2. Run it
|
||||
|
||||
|
||||
Reference in new issue
Block a user