better startup flow

This commit is contained in:
ari committed 2026-11-01 14:20:13 -05:00
1 parent d7f5dba677
commit 650bcff83a
8 files changed
+345 -68

No files matched your search

+22 -20
View File
@@ -29,9 +29,11 @@ flowchart LR
**A message's path:** Slack event (`app_mention`, a DM, or a plain reply in a channel thread set to `all-messages`) → `bot` → `manager.Ensure` (find the container, or place and start it) → `manager.DialAgent` (direct on the master, relayed for slaves) → `acp` session → updates stream back through `bot`'s streamer, which edits the Slack message.
**Lifecycle threads:** `manager.EnsureProgress` reports `StartEvent`s (placing, booting, registered) only when it really starts a container. The bot (`bot/lifecycle.go`) turns them into replies in a dedicated top-level message for that container, pins it, and unpins the previous run's message. The message's channel and timestamp are kept on the owner (`lifecycle_channel`, `lifecycle_ts`). The user's thread only gets a link. Stops (`stop`, the button, the idle reaper) post to the same thread, and `manager.WaitStopped` confirms when the machine is gone.
**Containers** are `systemd-nspawn` machines started through `systemd-run`, each in its own `slackerbot-<machine>.service` unit, with CPU and memory limits. They boot the NixOS system built by `nix build .#agent` (`nix/container.nix`), which runs `goose serve` on port 1984. `/nix/store` is shared read-only. `/root` is bind-mounted from a btrfs subvolume on one of the node's drives, and is the only thing that persists. The agent's env file (API keys, the per-container goose secret) is written by the host and bind-mounted in read-only.
**State** lives only on the master, in `state_dir/state.json`: owners (container, size, node, drive, storage limits, idle limit), sessions (Slack thread → ACP session id, plus the thread's listen mode), each channel's default listen mode, and permissions. Nodes report what's running and how much disk each home uses. Nothing else is stored.
**State** lives only on the master, in `state_dir/state.json`: owners (container, size, node, drive, storage limits, idle limit, lifecycle message), sessions (Slack thread → ACP session id, plus the thread's listen mode), each channel's default listen mode, and permissions. Nodes report what's running and how much disk each home uses. Nothing else is stored.
## Placement and capacity
@@ -78,25 +80,25 @@ flowchart TD
`util` (`KeyedMutex`, `RandomHex`) is used by `bot`, `manager` and `local`, and left out of the diagram to keep it readable.
| Package | Role |
|---|---|
| `main.go` | Flags, wiring, master vs slave |
| `model` | Shared types: `Size`, `Owner`, `Session`, permission rules, machine names |
| `units` | Parses and prints sizes (`5G`, `1.5 TB`) |
| `commands` | Parses Slack command text: `scale`, `keep`, `storage`, `grant`, and which permission a command needs |
| `config` | Loads and validates `config.toml` |
| `storage` | Drive math (`DriveReport`, headroom, overbooking) and btrfs subvolume/quota commands |
| `state` | `state.json` persistence |
| `node` | `Node` interface, wire types, offline/full errors, the WebSocket connection used to reach goose |
| `local` | The `Node` on this machine: nspawn via `systemd-run`, machined and systemd over D-Bus, capacity detection, btrfs send/receive |
| `balance` | Builds a view of each node and picks where a container goes |
| `manager` | `Manager` (owners, placement, start/stop/resize/storage), the cluster link (`Remote`, `ServeCluster`, `RunSlave`) and migration. These three share a package because they're tightly coupled |
| `acp` | JSON-RPC client for goose's ACP endpoint. Tool permission requests are auto-approved |
| `bot` | Slack events and buttons, replies, permission checks, `grant`/`revoke`, move announcements, idle reaper, streaming replies, thread listen modes, stop reactions, the assistant pane |
| `util` | `KeyedMutex`, `RandomHex` |
| `testutil`, `testutil/clustertest` | Fake nodes, and a real master + slave harness for tests |
| `nix/container.nix` | NixOS system for each container |
| `nix/host.nix` | NixOS module: the service, the `ve-*` network, firewall |
| Package | Role |
| ---------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `main.go` | Flags, wiring, master vs slave |
| `model` | Shared types: `Size`, `Owner`, `Session`, permission rules, machine names |
| `units` | Parses and prints sizes (`5G`, `1.5 TB`) |
| `commands` | Parses Slack command text: `scale`, `keep`, `storage`, `grant`, and which permission a command needs |
| `config` | Loads and validates `config.toml` |
| `storage` | Drive math (`DriveReport`, headroom, overbooking) and btrfs subvolume/quota commands |
| `state` | `state.json` persistence |
| `node` | `Node` interface, wire types, offline/full errors, the WebSocket connection used to reach goose |
| `local` | The `Node` on this machine: nspawn via `systemd-run`, machined and systemd over D-Bus, capacity detection, btrfs send/receive |
| `balance` | Builds a view of each node and picks where a container goes |
| `manager` | `Manager` (owners, placement, start/stop/resize/storage), the cluster link (`Remote`, `ServeCluster`, `RunSlave`) and migration. These three share a package because they're tightly coupled |
| `acp` | JSON-RPC client for goose's ACP endpoint. Tool permission requests are auto-approved |
| `bot` | Slack events and buttons, replies, permission checks, `grant`/`revoke`, move announcements, idle reaper, streaming replies, thread listen modes, stop reactions, the assistant pane |
| `util` | `KeyedMutex`, `RandomHex` |
| `testutil`, `testutil/clustertest` | Fake nodes, and a real master + slave harness for tests |
| `nix/container.nix` | NixOS system for each container |
| `nix/host.nix` | NixOS module: the service, the `ve-*` network, firewall |
## Known limits