bigbox2 runs nix-serve, which only serves its own store: every
'nix copy --to http://bigbox2.lab.internal:5000' from the post-build hook
ended in HTTP 404 after compressing the whole closure, so nothing ever
landed and the next build tried the same paths again. A post-build-hook
also blocks the build until it returns, which held up every build on
fwdesk (nbot's VM checks spent most of a cold CI run there).
- cacheReceive (on bigbox2): nix.sshServe with write over ssh-ng, for
fwdesk's ssh host key; nix-ssh is trusted, and nix-serve signs the
pushed paths when it serves them.
- cachePush: the hook only queues $OUT_PATHS. nix-cache-push.service
drains the queue with nix copy to ssh-ng://nix-ssh@bigbox2.lab.internal
as the host key, with bigbox2's host key pinned; a path unit starts it,
and a failed push is retried every 10 minutes by a timer.
Co-Authored-By: nBot
The env-reload unit is a oneshot whose ExecStart was a blocking
systemctl restart of the group unit. When the stack's cold start failed
transiently (container-services-buzz on minipc1, where buzz-relay-1 came
up unhealthy and compose exited 1), the group unit recovered on its own a
few seconds later through its own Restart=on-failure, but the reload unit
had already exited 1 with no restart policy and stayed failed until the
next env file write.
The trigger unit should queue the restart, not own the health of the whole
stack: pass --no-block. Real failures still surface on the group unit,
which carries Restart=on-failure and the start limit and is what the fleet
scan reports.