goose knows about nix
This commit is contained in:
1 parent
25fcf85855
commit
25f648d606
8 files changed
+35
-11
No files matched your search
+4
-2
@@ -31,11 +31,13 @@ flowchart LR
|
||||
|
||||
**Lifecycle threads:** `manager.EnsureProgress` reports `StartEvent`s (placing, booting, registered) only when it really starts a container. The bot (`bot/lifecycle.go`) turns them into replies in a dedicated top-level message for that container, pins it, and unpins the previous run's message. The message's channel and timestamp are kept on the owner (`lifecycle_channel`, `lifecycle_ts`). The user's thread only gets a link. Stops (`stop`, the button, the idle reaper) post to the same thread, and `manager.WaitStopped` confirms when the machine is gone.
|
||||
|
||||
**Files:** attachments on a message (`file_share`, or `files` on an `app_mention`) are downloaded by the bot and written into the home with the node's `write_file` call, then their paths are added to the prompt. The other way, goose gets the `slackerbot` MCP server (`slackerbot tools`, run inside the container as `slackerbot-tools`) through `mcpServers` in `session/new` and `session/load`. Its `send_file` tool only checks the path and size and answers "queued". The bot watches the ACP `tool_call` / `tool_call_update` messages, and when a `send_file` call completes it reads the file with `read_file` and uploads it to the thread with `files.uploadV2`. Bytes go node → master → Slack, so slaves never need a Slack token. Both calls work in 4 MB chunks, open files with `os.Root` so symlinks and `..` can't leave the home, and are capped by `max_send_file_mb`. An older slave answers "unknown call", which surfaces as `node.ErrTooOld` and a polite reply.
|
||||
**Files:** attachments on a message (`file_share`, or `files` on an `app_mention`) are downloaded by the bot and written into the home with the node's `write_file` call, then their paths are added to the prompt. The other way, goose gets the `slackerbot` MCP server (`slackerbot tools`, run inside the container as `slackerbot-tools`) through `mcpServers` in `session/new` and `session/load`. Its `initialize` reply carries `instructions` (how to use Nix in the container), which goose adds to its system prompt. Its `send_file` tool only checks the path and size and answers "queued". The bot watches the ACP `tool_call` / `tool_call_update` messages, and when a `send_file` call completes it reads the file with `read_file` and uploads it to the thread with `files.uploadV2`. Bytes go node → master → Slack, so slaves never need a Slack token. Both calls work in 4 MB chunks, open files with `os.Root` so symlinks and `..` can't leave the home, and are capped by `max_send_file_mb`. An older slave answers "unknown call", which surfaces as `node.ErrTooOld` and a polite reply.
|
||||
|
||||
**Commands as tools:** the same MCP server also lists one tool per Slack command (`tools.Commands`: `status`, `scale`, `keep`, `storage`, `move`, `listen`, `grant`, …), each with an optional `args` string that's appended to the verb, exactly as a user would type it. The MCP server can't reach the bot, so the call goes round: the bot spots the `tool_call` over ACP (`tools.CommandCall`), runs the command through the same code as Slack (`bot.runCommand`, `bot.move`) as the user whose message started the turn, and writes the reply with `write_file` to `/root/.slackerbot/replies/<hash of tool and args>-<random>.json`. The server waits up to a minute for a file with that prefix and claims it by deleting it. Commands that change something are also posted in the thread. `stop` waits for the turn to end (`agentCommands.Finish`), and `move` reports as soon as it starts, since its final pass waits for the reply to finish anyway.
|
||||
|
||||
**Containers** are `systemd-nspawn` machines started through `systemd-run`, each in its own `slackerbot-<machine>.service` unit, with CPU and memory limits. They boot the NixOS system built by `nix build .#agent` (`nix/container.nix`), which runs `goose serve` on port 1984. `/nix/store` is shared read-only. `/root` is bind-mounted from a btrfs subvolume on one of the node's drives, and is the only thing that persists. `ro_binds` and `rw_binds` add more mounts, and their missing source folders are created first. The agent's env file (API keys, the per-container goose secret) is written by the host and bind-mounted in read-only.
|
||||
**Containers** are `systemd-nspawn` machines started through `systemd-run`, each in its own `slackerbot-<machine>.service` unit, with CPU and memory limits. They boot the NixOS system built by `nix build .#agent` (`nix/container.nix`), which runs `goose serve` on port 1984. Containers use user namespacing (`--private-users=pick`), so their root is an unprivileged uid on the host. `/root` is bind-mounted from a btrfs subvolume on one of the node's drives, and is the only thing that persists. `ro_binds` and `rw_binds` add more mounts, and their missing source folders are created first. The agent's env file (API keys, the per-container goose secret) is written by the host and bind-mounted in read-only. The home, the env file and `/nix/store` (read-only) are mounted with `idmap`, so files stay owned by host root on disk and show up as container root inside.
|
||||
|
||||
**Nix in containers:** the host's `/nix/var/nix/daemon-socket` is bind-mounted in, and `container.nix` turns off the container's own `nix-daemon` and enables flakes. `nix build`/`nix develop` inside a container go to the host daemon, which sees an unprivileged uid and so treats it as an untrusted user: builds are sandboxed and it can't add substituters or import unsigned paths. Outputs land in the shared store, outside the home quota, and aren't GC roots on the host. The MCP server's `initialize` `instructions` tell goose to use a flake and `devShells.default` for each project.
|
||||
|
||||
**System updates:** each node reports running containers whose `slackerbot-<machine>.service` command ends with a different system's `init` than the node's current `system` (`outdated` in `RunningMachine`, read over D-Bus; unknown counts as current). After a reply, the bot asks once per container (`bot/update.go`, remembered in memory) whether to restart. **Restart now** calls `manager.Restart` (power off, wait, then the normal start), unless a reply is in progress.
|
||||
|
||||
|
||||
+2
-2
@@ -2,9 +2,9 @@
|
||||
|
||||
## Requirements
|
||||
|
||||
- Linux with systemd (`systemd-nspawn`, `systemd-machined`, `systemd-networkd`), running as root.
|
||||
- Linux with systemd (`systemd-nspawn`, `systemd-machined`, `systemd-networkd`), running as root. Containers run with user namespacing and ID-mapped mounts, so you need systemd 250 or newer and a kernel that supports ID-mapped mounts for the filesystems holding `/nix/store`, `state_dir` and the drives (btrfs, ext4 and xfs on any recent kernel).
|
||||
- One or more **btrfs** folders for home folders. Quotas are switched on automatically.
|
||||
- Nix with flakes, to build the bot and the container system. NixOS is easiest, but not required.
|
||||
- Nix with flakes, to build the bot and the container system. NixOS is easiest, but not required. It has to be a multi-user install with `nix-daemon` running: containers build through `/nix/var/nix/daemon-socket`, and won't start without it. The daemon sees containers as untrusted users (they run as unprivileged host uids), so `trusted-users` needs no changes, but `allowed-users` must still allow everyone (the default `*`). Their builds go into the host's store, so run `nix-collect-garbage` or `nix.gc` now and then.
|
||||
- An API key for the model goose should use (e.g. Anthropic).
|
||||
|
||||
## 1. Slack app
|
||||
|
||||
+1
-1
@@ -15,7 +15,7 @@ In DMs the bot reads every message. You can also chat with it in Slack's assista
|
||||
|
||||
## Talking to the agent
|
||||
|
||||
1. Send a message. If the container is stopped, the bot starts it, and if it doesn't exist yet, the bot creates it. Your home folder (`/root`) is kept between starts.
|
||||
1. Send a message. If the container is stopped, the bot starts it, and if it doesn't exist yet, the bot creates it. Your home folder (`/root`) is kept between starts. The container comes with few tools, but it has Nix with flakes, and the agent is told to give each project a `flake.nix` with a dev shell rather than installing things globally. Anything outside `/root` is gone after a restart.
|
||||
2. A new top-level message starts a **new session**, and the reply goes in a thread.
|
||||
3. **Reply in that thread to continue the same session**, even days later. `sessions` lists past ones.
|
||||
4. The answer streams in as the agent works. Tool calls show up as 🔧 lines. Your message gets 👀 while the agent works, then ✅ when it's done or ⚠️ if something went wrong.
|
||||
|
||||
+2
-1
@@ -12,7 +12,8 @@ boot_timeout = "2m"
|
||||
max_send_file_mb = 100
|
||||
|
||||
# Extra mounts as "host:container". "{machine}" becomes the container name, e.g. c-c07abc123.
|
||||
# Missing host folders are created.
|
||||
# Missing host folders are created. Containers use user namespacing, so host files show up owned by
|
||||
# nobody. Add ":idmap" ("host:container:idmap") to make host root's files container root's, e.g. for rw_binds.
|
||||
ro_binds = []
|
||||
rw_binds = []
|
||||
|
||||
|
||||
@@ -17,6 +17,11 @@
|
||||
(writeShellScriptBin "slackerbot-tools" ''exec ${slackerbot}/bin/slackerbot "$@"'')
|
||||
];
|
||||
|
||||
# Builds go through the host's nix-daemon, whose socket the bot bind-mounts in.
|
||||
nix.settings.experimental-features = [ "nix-command" "flakes" ];
|
||||
systemd.sockets.nix-daemon.enable = false;
|
||||
systemd.services.nix-daemon.enable = false;
|
||||
|
||||
systemd.services.goose = {
|
||||
wantedBy = [ "multi-user.target" ];
|
||||
after = [ "network-online.target" ];
|
||||
|
||||
@@ -34,7 +34,8 @@ const (
|
||||
systemdMgr = "org.freedesktop.systemd1.Manager"
|
||||
sigPoweroff = 38 // SIGRTMIN+4, what `machinectl poweroff` sends
|
||||
|
||||
agentEnvInside = "/var/lib/slackerbot/agent.env"
|
||||
agentEnvInside = "/var/lib/slackerbot/agent.env"
|
||||
nixDaemonSocket = "/nix/var/nix/daemon-socket"
|
||||
)
|
||||
|
||||
type Local struct {
|
||||
@@ -212,6 +213,9 @@ func (l *Local) Start(ctx context.Context, spec node.StartSpec) error {
|
||||
f.Close()
|
||||
}
|
||||
|
||||
if _, err := os.Stat(nixDaemonSocket); err != nil {
|
||||
return fmt.Errorf("containers build through the host's nix-daemon, but %s is missing: %w", nixDaemonSocket, err)
|
||||
}
|
||||
envFile, err := l.writeEnvFile(spec)
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -234,9 +238,12 @@ func (l *Local) Start(ctx context.Context, spec node.StartSpec) error {
|
||||
"--directory=" + root,
|
||||
"--keep-unit",
|
||||
"--network-veth", // own network, so every container can use port 1984
|
||||
"--bind-ro=/nix/store",
|
||||
"--bind-ro=" + envFile + ":" + agentEnvInside,
|
||||
"--bind=" + home + ":/root",
|
||||
// Container root is an unprivileged host uid, so the host nix-daemon treats it as untrusted.
|
||||
"--private-users=pick",
|
||||
"--bind-ro=/nix/store:/nix/store:idmap",
|
||||
"--bind-ro=" + nixDaemonSocket,
|
||||
"--bind-ro=" + envFile + ":" + agentEnvInside + ":idmap",
|
||||
"--bind=" + home + ":/root:idmap",
|
||||
}
|
||||
binds := func(list []string, flag string) error {
|
||||
for _, b := range list {
|
||||
|
||||
@@ -24,6 +24,14 @@ const sendFileHelp = "Send a file to the user in Slack: images, videos, PDFs, ar
|
||||
"It's uploaded into the conversation when your turn ends, so mention it in your reply. " +
|
||||
"At most a few files per turn."
|
||||
|
||||
const instructions = `This container runs NixOS with only a few tools installed. Nix is available with flakes, and builds run through the host. Use it for every toolchain and dependency:
|
||||
- For a new project, write a flake.nix with a devShells.default that provides everything the project needs (and packages.default if it builds something), then run commands with ` + "`nix develop -c <command>`" + `.
|
||||
- In an existing project without a flake.nix, add one before installing anything.
|
||||
- Flakes only see files tracked by git, so ` + "`git init`" + ` and ` + "`git add`" + ` new files, including flake.nix and flake.lock, before building.
|
||||
- For a one-off tool, use ` + "`nix shell nixpkgs#<package> -c <command>`" + `.
|
||||
- Don't install things globally (nix-env, nix profile, pip install --user, npm install -g): only the home folder survives a restart.
|
||||
- The host may garbage-collect build outputs, so rebuild instead of relying on old result links.`
|
||||
|
||||
const commandNote = " It's the same as the Slack command `%s`, and runs with the permissions of the person whose message you're answering."
|
||||
|
||||
const (
|
||||
@@ -168,6 +176,7 @@ func (s *Server) handle(m msg) (any, *rpcError) {
|
||||
"protocolVersion": p.ProtocolVersion,
|
||||
"capabilities": map[string]any{"tools": map[string]any{}},
|
||||
"serverInfo": map[string]string{"name": ServerName, "version": "1"},
|
||||
"instructions": instructions,
|
||||
}, nil
|
||||
case "ping":
|
||||
return map[string]any{}, nil
|
||||
|
||||
@@ -29,7 +29,7 @@ func TestSendFileTool(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
lines := strings.Split(strings.TrimSpace(out.String()), "\n")
|
||||
if len(lines) != 5 || !strings.Contains(lines[0], `"protocolVersion":"2025-06-18"`) {
|
||||
if len(lines) != 5 || !strings.Contains(lines[0], `"protocolVersion":"2025-06-18"`) || !strings.Contains(lines[0], "devShells.default") {
|
||||
t.Fatalf("replies: %v", lines)
|
||||
}
|
||||
want := []struct {
|
||||
|
||||
Reference in new issue
Block a user