17 KiB
Architecture
System
flowchart LR
slack([Slack]) <-->|Socket Mode| bot
subgraph master["Master host"]
bot[bot] --> mgr[manager]
bot <-->|ACP over WebSocket| acp[acp client]
mgr --> local1[local node]
mgr --> state[(state.json)]
local1 -->|systemd-run + machined D-Bus| c1["nspawn: c-…<br/>goose serve :1984"]
local1 -->|btrfs subvolume + quota| d1[(drives)]
end
subgraph slave["Slave host (0..n)"]
local2[local node] --> c2["nspawn: u-…<br/>goose serve :1984"]
local2 --> d2[(drives)]
end
acp -.->|direct| c1
mgr <==>|"TLS WebSocket (slave dials out)<br/>calls · reports every 5s · agent relay · btrfs send/receive"| local2
acp -.->|relayed over the link| c2
One binary, two modes. -mode master runs the Slack bot, the manager and a local node. -mode slave runs only a local node, and keeps one outgoing TLS WebSocket to the master, so slaves can sit behind NAT.
A message's path: Slack event (app_mention, a DM, or a plain reply in a channel thread set to all-messages) → bot → manager.Ensure (find the container, or place and start it) → manager.DialAgent (direct on the master, relayed for slaves) → acp session → updates stream back through bot's streamer, which edits the Slack message. While the reply runs, its newest message also has a status line (what the agent is doing, and when it last sent an update) and a Cancel button whose value is the placeholder's ts, the key in Bot.running. The final edit removes both.
In-flight turns: once the placeholder is posted, bot.converse records the turn (thread, message and placeholder ts, sender, text, attachments) in state_dir/in-flight-<channel>.json, one file per channel keyed by thread, and marks it prompted just before session/prompt. The record is removed when the turn ends, unless it ended because the bot is shutting down: then the reply ends with a restarting note, and Bot.Run waits up to 5s for those last edits. On start, Bot.Run resumes every recorded turn through the same path. If the agent already got the prompt, it gets a short "carry on" note in the same session instead of the message again, because goose keeps the partial turn and resending could repeat what it already did. If it didn't (or the session is gone), it gets the original message and attachments. Turns older than an hour, or already resumed 3 times, are dropped with a note asking the user to send them again. A broken file is skipped and left on disk. Super users can read the files with in-flight (Slack, answered ephemerally) or the in_flight tool. running (tool running) lists a channel's recorded turns, and cancel all (tool cancel_all) calls the Bot.running cancel func of each one whose prompt has started, skipping the calling agent's own turn.
Lifecycle threads: manager.EnsureProgress reports StartEvents (placing, booting, registered) only when it really starts a container. The bot (bot/lifecycle.go) turns them into replies in a dedicated top-level message for that container, pins it, and unpins the previous run's message. The message's channel and timestamp are kept on the owner (lifecycle_channel, lifecycle_ts). The user's thread only gets a link. Stops (stop, the button, the idle reaper) post to the same thread, and manager.WaitStopped confirms when the machine is gone.
Files: attachments on a message (file_share, or files on an app_mention) are downloaded by the bot and written into the home with the node's write_file call, then their paths are added to the prompt. The other way, goose gets the slackerbot MCP server (slackerbot tools, run inside the container as slackerbot-tools) through mcpServers in session/new and session/load. Its initialize reply carries instructions (how to use Nix in the container), which goose adds to its system prompt. Its send_file tool only checks the path and size and answers "queued". The bot watches the ACP tool_call / tool_call_update messages, and when a send_file call completes it reads the file with read_file and uploads it to the thread with files.uploadV2. Bytes go node → master → Slack, so slaves never need a Slack token. Both calls work in 4 MB chunks, open files with os.Root so symlinks and .. can't leave the home, and are capped by max_send_file_mb. An older slave answers "unknown call", which surfaces as node.ErrTooOld and a polite reply.
Commands as tools: the same MCP server also lists one tool per Slack command (tools.Commands: status, scale, keep, storage, move, listen, grant, …), each with an optional args string that's appended to the verb, exactly as a user would type it. The MCP server can't reach the bot, so the call goes round: the bot spots the tool_call over ACP (tools.CommandCall), runs the command through the same code as Slack (bot.runCommand, bot.move) as the user whose message started the turn, and writes the reply with write_file to /root/.slackerbot/replies/<hash of tool and args>-<random>.json. The server waits up to a minute for a file with that prefix and claims it by deleting it. Commands that change something are also posted in the thread. stop waits for the turn to end (agentCommands.Finish), and move reports as soon as it starts, since its final pass waits for the reply to finish anyway.
Containers are systemd-nspawn machines started through systemd-run, each in its own slackerbot-<machine>.service unit, with CPU and memory limits. They boot the NixOS system built by nix build .#agent (nix/container.nix), which runs goose serve on port 1984. Containers use user namespacing (--private-users=pick), so their root is an unprivileged uid on the host. /root is bind-mounted from a btrfs subvolume on one of the node's drives, and is the only thing that persists. ro_binds and rw_binds add more mounts, and their missing source folders are created first. The agent's env file (API keys, the per-container goose secret) is written by the host and bind-mounted in read-only. The home, the env file and /nix/store (read-only) are mounted with idmap, so files stay owned by host root on disk and show up as container root inside.
Nix in containers: the host's /nix/var/nix/daemon-socket is bind-mounted in, and container.nix turns off the container's own nix-daemon and enables flakes. nix build/nix develop inside a container go to the host daemon, which sees an unprivileged uid and so treats it as an untrusted user: builds are sandboxed and it can't add substituters or import unsigned paths. Outputs land in the shared store, outside the home quota, and aren't GC roots on the host. The MCP server's initialize instructions tell goose to use a flake and devShells.default for each project.
System updates: each node reports running containers whose slackerbot-<machine>.service command ends with a different system's init than the node's current system (outdated in RunningMachine, read over D-Bus; unknown counts as current). After a reply, the bot asks once per container (bot/update.go, remembered in memory) whether to restart. Restart now calls manager.Restart (power off, wait, then the normal start), unless a reply is in progress.
State lives only on the master, in state_dir/state.json: owners (container, size, node, drive, storage limits, idle limit, lifecycle message), sessions (Slack thread → ACP session id, plus the thread's listen mode), each channel's default listen mode, and permissions. Turns still in progress go in state_dir/in-flight-<channel>.json (see In-flight turns). Nodes report what's running and how much disk each home uses. Nothing else is stored.
Placement and capacity
- Each node's usable compute = total RAM/CPU −
host_reserve. Each running container counts its full limit. - Each drive's usable space = free + what homes use −
reserve. Each home counts its guarantee or its real usage, whichever is bigger. The caps added up may exceed the space (overbooking is allowed, and the bot flags it). - New containers go where
max(cpu load, ram load, storage load)after placement stays lowest (balance.ChoosePlacement), on the drive with the most headroom. - Containers stay on their node until moved. When a node is too full to start a container, the bot moves it to another node automatically.
Cluster link
- The slave connects to
wss://master/clusterwithX-Cluster-SecretandX-Node-Name, and pins the master's self-signed certificate by fingerprint, either from the config or trusted on first use. - Frames are JSON:
call/replyfor node operations,reportfor node info every 5s, anddata/closefor byte streams. Agent connections are tunnelled as streams. - Each
reportcarries the slave's build version. The bot compares it with the master's and flags any difference at the top ofhelp,nodesandstatus. A slave that reports no version (an older build) counts as a mismatch. - A slave that hasn't reported for 20s, or whose link has dropped, counts as offline. The master doesn't trust reports for a short while after it asks a slave to start or stop something, so stale reports don't undo the change.
Migration
manager.Migrate: pick a target with room, btrfs send a read-only snapshot in 4 MB chunks through the master into btrfs receive on the target, and keep a running container up during this pass. Then the bot waits for any reply in progress to finish, stops the container, sends an incremental snapshot, makes a writable snapshot the new home, switches the owner record, deletes the old home, and starts the container on the target. If anything fails before the switch, the bot rolls back: the half-copied home is removed and the container restarts where it was.
Code
All Go lives in slackerbot/ (module slackerbot). Packages under internal/ each do one job, and only import downward:
flowchart TD
main[main] --> bot & manager & local & state & config & tools
bot --> manager & acp & commands & balance & tools
tools --> acp
manager --> balance & state & config & storage
local --> config & storage
acp --> node
balance --> node & storage
commands --> units
config --> units
manager --> node
local --> node
node --> model
state --> model
commands --> model
config --> model
balance --> model
units
storage
util
util (KeyedMutex, RandomHex) is used by bot, manager and local, and left out of the diagram to keep it readable.
| Package | Role |
|---|---|
main.go |
Flags, wiring, master vs slave |
model |
Shared types: Size, Owner, Session, permission rules, machine names |
units |
Parses and prints sizes (5G, 1.5 TB) |
commands |
Parses Slack command text: scale, keep, storage, grant, and which permission a command needs |
config |
Loads and validates config.toml |
storage |
Drive math (DriveReport, headroom, overbooking) and btrfs subvolume/quota commands |
state |
state.json and in-flight-<channel>.json persistence |
node |
Node interface, wire types, offline/full errors, the WebSocket connection used to reach goose |
local |
The Node on this machine: nspawn via systemd-run, machined and systemd over D-Bus, capacity detection, btrfs send/receive |
balance |
Builds a view of each node and picks where a container goes |
manager |
Manager (owners, placement, start/stop/resize/storage), the cluster link (Remote, ServeCluster, RunSlave) and migration. These three share a package because they're tightly coupled |
tools |
The in-container MCP server (send_file and the Slack commands), how goose starts it, spotting its calls in ACP updates, and the reply files commands are answered with |
acp |
JSON-RPC client for goose's ACP endpoint. Tool permission requests are auto-approved |
bot |
Slack events and buttons, replies, permission checks, grant/revoke, move announcements, idle reaper, streaming replies, thread listen modes, stop reactions, the assistant pane, commands the agent calls as tools |
util |
KeyedMutex, RandomHex |
testutil, testutil/clustertest |
Fake nodes, a fake goose, and a real master + slave harness for tests |
nix/container.nix |
NixOS system for each container |
nix/host.nix |
NixOS module: the service, the ve-* network, firewall |
There are two builds. Nix (flake.nix) builds the deployable package, the container system and the host module. Bazel (MODULE.bazel, with BUILD.bazel files generated by Gazelle) builds and tests the Go code and packages the static binary as a tarball. Both take their dependencies from slackerbot/go.mod.
Known limits
- The agent's tool calls are auto-approved, since the container is the sandbox (
acp.handleRequest). - Anyone in a channel controls that channel's container and everything the agent can reach from it.
- The agent runs commands with the sender's permissions, so whatever steers it (a file, a web page) can scale, move or re-grant as far as that person may. The thread shows every change it makes.
- Containers run as root without user namespaces, and container root can get around btrfs quotas. That's fine for fair sharing among trusted users, but not safe for hostile ones.
- Restarting the master makes slaves look offline for a few seconds. Restarting the bot doesn't stop containers, and replies in progress are picked back up, but Slack doesn't replay messages sent while the bot was down.
state.jsonis fine for hundreds of users. Swap thestatepackage for SQLite if you need more.- Home folders are never deleted automatically (see INSTALLATION.md).