restart fix + slack commands

This commit is contained in:
ari committed 2026-11-01 15:10:39 -05:00
1 parent e90c9f324b
commit 25fcf85855
22 files changed
+1077 -119

No files matched your search

+26 -21
View File
@@ -33,7 +33,11 @@ flowchart LR
**Files:** attachments on a message (`file_share`, or `files` on an `app_mention`) are downloaded by the bot and written into the home with the node's `write_file` call, then their paths are added to the prompt. The other way, goose gets the `slackerbot` MCP server (`slackerbot tools`, run inside the container as `slackerbot-tools`) through `mcpServers` in `session/new` and `session/load`. Its `send_file` tool only checks the path and size and answers "queued". The bot watches the ACP `tool_call` / `tool_call_update` messages, and when a `send_file` call completes it reads the file with `read_file` and uploads it to the thread with `files.uploadV2`. Bytes go node → master → Slack, so slaves never need a Slack token. Both calls work in 4 MB chunks, open files with `os.Root` so symlinks and `..` can't leave the home, and are capped by `max_send_file_mb`. An older slave answers "unknown call", which surfaces as `node.ErrTooOld` and a polite reply.
**Containers** are `systemd-nspawn` machines started through `systemd-run`, each in its own `slackerbot-<machine>.service` unit, with CPU and memory limits. They boot the NixOS system built by `nix build .#agent` (`nix/container.nix`), which runs `goose serve` on port 1984. `/nix/store` is shared read-only. `/root` is bind-mounted from a btrfs subvolume on one of the node's drives, and is the only thing that persists. The agent's env file (API keys, the per-container goose secret) is written by the host and bind-mounted in read-only.
**Commands as tools:** the same MCP server also lists one tool per Slack command (`tools.Commands`: `status`, `scale`, `keep`, `storage`, `move`, `listen`, `grant`, …), each with an optional `args` string that's appended to the verb, exactly as a user would type it. The MCP server can't reach the bot, so the call goes round: the bot spots the `tool_call` over ACP (`tools.CommandCall`), runs the command through the same code as Slack (`bot.runCommand`, `bot.move`) as the user whose message started the turn, and writes the reply with `write_file` to `/root/.slackerbot/replies/<hash of tool and args>-<random>.json`. The server waits up to a minute for a file with that prefix and claims it by deleting it. Commands that change something are also posted in the thread. `stop` waits for the turn to end (`agentCommands.Finish`), and `move` reports as soon as it starts, since its final pass waits for the reply to finish anyway.
**Containers** are `systemd-nspawn` machines started through `systemd-run`, each in its own `slackerbot-<machine>.service` unit, with CPU and memory limits. They boot the NixOS system built by `nix build .#agent` (`nix/container.nix`), which runs `goose serve` on port 1984. `/nix/store` is shared read-only. `/root` is bind-mounted from a btrfs subvolume on one of the node's drives, and is the only thing that persists. `ro_binds` and `rw_binds` add more mounts, and their missing source folders are created first. The agent's env file (API keys, the per-container goose secret) is written by the host and bind-mounted in read-only.
**System updates:** each node reports running containers whose `slackerbot-<machine>.service` command ends with a different system's `init` than the node's current `system` (`outdated` in `RunningMachine`, read over D-Bus; unknown counts as current). After a reply, the bot asks once per container (`bot/update.go`, remembered in memory) whether to restart. **Restart now** calls `manager.Restart` (power off, wait, then the normal start), unless a reply is in progress.
**State** lives only on the master, in `state_dir/state.json`: owners (container, size, node, drive, storage limits, idle limit, lifecycle message), sessions (Slack thread → ACP session id, plus the thread's listen mode), each channel's default listen mode, and permissions. Nodes report what's running and how much disk each home uses. Nothing else is stored.
@@ -84,26 +88,26 @@ flowchart TD
`util` (`KeyedMutex`, `RandomHex`) is used by `bot`, `manager` and `local`, and left out of the diagram to keep it readable.
| Package | Role |
| ---------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `main.go` | Flags, wiring, master vs slave |
| `model` | Shared types: `Size`, `Owner`, `Session`, permission rules, machine names |
| `units` | Parses and prints sizes (`5G`, `1.5 TB`) |
| `commands` | Parses Slack command text: `scale`, `keep`, `storage`, `grant`, and which permission a command needs |
| `config` | Loads and validates `config.toml` |
| `storage` | Drive math (`DriveReport`, headroom, overbooking) and btrfs subvolume/quota commands |
| `state` | `state.json` persistence |
| `node` | `Node` interface, wire types, offline/full errors, the WebSocket connection used to reach goose |
| `local` | The `Node` on this machine: nspawn via `systemd-run`, machined and systemd over D-Bus, capacity detection, btrfs send/receive |
| `balance` | Builds a view of each node and picks where a container goes |
| `manager` | `Manager` (owners, placement, start/stop/resize/storage), the cluster link (`Remote`, `ServeCluster`, `RunSlave`) and migration. These three share a package because they're tightly coupled |
| `tools` | The in-container MCP server (`send_file`), how goose starts it, and spotting its calls in ACP updates |
| `acp` | JSON-RPC client for goose's ACP endpoint. Tool permission requests are auto-approved |
| `bot` | Slack events and buttons, replies, permission checks, `grant`/`revoke`, move announcements, idle reaper, streaming replies, thread listen modes, stop reactions, the assistant pane |
| `util` | `KeyedMutex`, `RandomHex` |
| `testutil`, `testutil/clustertest` | Fake nodes, and a real master + slave harness for tests |
| `nix/container.nix` | NixOS system for each container |
| `nix/host.nix` | NixOS module: the service, the `ve-*` network, firewall |
| Package | Role |
| ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `main.go` | Flags, wiring, master vs slave |
| `model` | Shared types: `Size`, `Owner`, `Session`, permission rules, machine names |
| `units` | Parses and prints sizes (`5G`, `1.5 TB`) |
| `commands` | Parses Slack command text: `scale`, `keep`, `storage`, `grant`, and which permission a command needs |
| `config` | Loads and validates `config.toml` |
| `storage` | Drive math (`DriveReport`, headroom, overbooking) and btrfs subvolume/quota commands |
| `state` | `state.json` persistence |
| `node` | `Node` interface, wire types, offline/full errors, the WebSocket connection used to reach goose |
| `local` | The `Node` on this machine: nspawn via `systemd-run`, machined and systemd over D-Bus, capacity detection, btrfs send/receive |
| `balance` | Builds a view of each node and picks where a container goes |
| `manager` | `Manager` (owners, placement, start/stop/resize/storage), the cluster link (`Remote`, `ServeCluster`, `RunSlave`) and migration. These three share a package because they're tightly coupled |
| `tools` | The in-container MCP server (`send_file` and the Slack commands), how goose starts it, spotting its calls in ACP updates, and the reply files commands are answered with |
| `acp` | JSON-RPC client for goose's ACP endpoint. Tool permission requests are auto-approved |
| `bot` | Slack events and buttons, replies, permission checks, `grant`/`revoke`, move announcements, idle reaper, streaming replies, thread listen modes, stop reactions, the assistant pane, commands the agent calls as tools |
| `util` | `KeyedMutex`, `RandomHex` |
| `testutil`, `testutil/clustertest` | Fake nodes, and a real master + slave harness for tests |
| `nix/container.nix` | NixOS system for each container |
| `nix/host.nix` | NixOS module: the service, the `ve-*` network, firewall |
There are two builds. Nix (`flake.nix`) builds the deployable package, the container system and the host module. Bazel (`MODULE.bazel`, with `BUILD.bazel` files generated by Gazelle) builds and tests the Go code and packages the static binary as a tarball. Both take their dependencies from `slackerbot/go.mod`.
@@ -111,6 +115,7 @@ There are two builds. Nix (`flake.nix`) builds the deployable package, the conta
- The agent's tool calls are auto-approved, since the container is the sandbox (`acp.handleRequest`).
- Anyone in a channel controls that channel's container and everything the agent can reach from it.
- The agent runs commands with the sender's permissions, so whatever steers it (a file, a web page) can scale, move or re-grant as far as that person may. The thread shows every change it makes.
- Containers run as root without user namespaces, and container root can get around btrfs quotas. That's fine for fair sharing among trusted users, but not safe for hostile ones.
- Restarting the master makes slaves look offline for a few seconds. Restarting the bot doesn't stop containers.
- `state.json` is fine for hundreds of users. Swap the `state` package for SQLite if you need more.