buz support
This commit is contained in:
1 parent
5d727c26cd
commit
beb83f21b3
13 files changed
+397
-42
No files matched your search
@@ -17,7 +17,7 @@ common/
|
||||
├── ssh-notify.nix # skyg.nixos.common.ssh-notify — forward notify-send from SSH sessions to a desktop client
|
||||
├── qemu.nix # skyg.core.qemu — QEMU/quickemu + SPICE guest tooling
|
||||
├── binary-cache.nix # skyg.nixos.common.cachePush — push finished builds to the bigbox2 cache
|
||||
├── containers/ # Docker vs Podman runtime selection
|
||||
├── containers/ # Docker vs Podman runtime selection + host-created container networks
|
||||
├── hardware/ # GPU, fan control, audio, laptop power, Coral TPU udev
|
||||
└── pritunl/ # Pritunl VPN client service
|
||||
```
|
||||
@@ -53,6 +53,6 @@ skyg.nixos.common.pritunl.* → pritunl/
|
||||
|
||||
## Conventions
|
||||
|
||||
- This tree is for behaviour that is reasonable on *any* Linux host. Anything role-specific
|
||||
- This tree is for behaviour that is reasonable on _any_ Linux host. Anything role-specific
|
||||
belongs in `server/` or `desktop/`.
|
||||
- Hostname-independent defaults can be set directly; anything optional gets an `enable` gate.
|
||||
@@ -8,7 +8,8 @@ Container runtime configuration and compose-style service groups.
|
||||
containers/
|
||||
├── default.nix # Shared runtime options (runtime, enableOnBoot, registries, metrics)
|
||||
├── docker.nix # Applied when runtime == "docker"
|
||||
├── podman.nix # Applied when runtime == "podman" (dockerCompat + socket)
|
||||
├── podman.nix # Applied when runtime == "podman" (dockerCompat + socket + registries)
|
||||
├── networks.nix # Host-created container networks (skyg.nixos.common.containers.networks)
|
||||
├── container-services/ # Compose-style service groups (modular implementation)
|
||||
│ ├── default.nix
|
||||
│ ├── options.nix
|
||||
@@ -31,7 +32,8 @@ consistently (compose tooling, user group membership, OCI backend, metrics).
|
||||
(default `minipc1.lab.internal:5001`), exposes metrics on `metricsPort` (9323), adds the
|
||||
user to the `docker` group, and installs `docker-compose`.
|
||||
- **Podman** enables `dockerCompat` + docker socket so `docker`/`oci-containers` keep working,
|
||||
and installs `podman-compose` + `podman-tui`.
|
||||
allows plain-HTTP pulls from `localDockerRegistries` via
|
||||
`virtualisation.containers.registries.settings`, and installs `podman-compose` + `podman-tui`.
|
||||
- `openMetricsPort` opens the metrics port in the firewall (Docker only).
|
||||
|
||||
### Option namespace
|
||||
@@ -43,8 +45,39 @@ skyg.nixos.common.containers.enableOnBoot
|
||||
skyg.nixos.common.containers.localDockerRegistries
|
||||
skyg.nixos.common.containers.metricsPort
|
||||
skyg.nixos.common.containers.openMetricsPort
|
||||
skyg.nixos.common.containers.networks.<name>.* → networks.nix (host-created networks)
|
||||
```
|
||||
|
||||
## Host-created networks (`skyg.nixos.common.containers.networks`)
|
||||
|
||||
Creates container networks once on the host (one systemd oneshot each, ordered before
|
||||
`container-networks.target`), independent of any compose project. This is how a _single_
|
||||
network — e.g. one macvlan network — is shared by several `container-services` groups
|
||||
instead of being re-declared (and re-created) per stack.
|
||||
|
||||
```nix
|
||||
skyg.nixos.common.containers.networks.lab = {
|
||||
driver = "macvlan";
|
||||
driverOpts.parent = "eno1";
|
||||
subnet = "10.0.0.0/16";
|
||||
gateway = "10.0.0.1";
|
||||
ipRange = "10.0.101.16/28";
|
||||
};
|
||||
```
|
||||
|
||||
Each network exposes a computed `compose` attribute (an `external: true` entry) for use in a
|
||||
group's `networks` option:
|
||||
|
||||
```nix
|
||||
skyg.nixos.common.container-services.my-stack = {
|
||||
networks.lan = config.skyg.nixos.common.containers.networks.lab.compose;
|
||||
services.app.networks.lan = { ipv4_address = "10.0.101.3"; };
|
||||
};
|
||||
```
|
||||
|
||||
Fields: `enable`, `name` (default = attr name), `driver`, `driverOpts`, `subnet`, `gateway`,
|
||||
`ipRange`, `internal`. Works with both docker and podman.
|
||||
|
||||
## Container service groups (`skyg.nixos.common.container-services`)
|
||||
|
||||
A compose-like abstraction that:
|
||||
|
||||
@@ -4,14 +4,14 @@ Custom compose-style container service groups for systemd.
|
||||
|
||||
## Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `default.nix` | Main module entry point; imports options and wires everything |
|
||||
| `options.nix` | All `skyg.nixos.common.container-services.*` option declarations |
|
||||
| `lib.nix` | Shared helpers (runtime dispatch, path sanitization, env file collection) |
|
||||
| File | Purpose |
|
||||
| ------------- | ----------------------------------------------------------------------------------- |
|
||||
| `default.nix` | Main module entry point; imports options and wires everything |
|
||||
| `options.nix` | All `skyg.nixos.common.container-services.*` option declarations |
|
||||
| `lib.nix` | Shared helpers (runtime dispatch, path sanitization, env file collection) |
|
||||
| `compose.nix` | YAML composition (mkComposeService, mkComposeAttrs, mkComposeFile, mkOverridesFile) |
|
||||
| `files.nix` | File mounting logic (getAllFiles, mkFileVolumesForService, mkFilesService) |
|
||||
| `systemd.nix` | Systemd unit builders (mkSystemdService, mkPathUnit, mkEnvReloadService) |
|
||||
| `files.nix` | File mounting logic (getAllFiles, mkFileVolumesForService, mkFilesService) |
|
||||
| `systemd.nix` | Systemd unit builders (mkSystemdService, mkPathUnit, mkEnvReloadService) |
|
||||
|
||||
## How it works
|
||||
|
||||
@@ -42,6 +42,8 @@ If adding a new per-service option:
|
||||
## Architecture
|
||||
|
||||
- Each group gets **one** systemd unit (`container-services-<group>.service`) that manages the whole stack
|
||||
- Group units are ordered after `container-networks.target`, so any network declared in
|
||||
`skyg.nixos.common.containers.networks` exists before a stack starts
|
||||
- Custom files are collected and written by a separate unit (`container-services-<group>-files.service`)
|
||||
- Env file changes trigger a path unit that restarts the stack (`container-services-<group>-env-reload.*`)
|
||||
- Compose files are rendered to the store and staged to `/var/lib/container-services/<group>/compose.yml`
|
||||
@@ -14,9 +14,10 @@
|
||||
{
|
||||
description = "Container service group '${groupName}'";
|
||||
wantedBy = [ "multi-user.target" ];
|
||||
wants = [ "network-online.target" "remote-fs.target" ];
|
||||
wants = [ "network-online.target" "remote-fs.target" "container-networks.target" ];
|
||||
after = [
|
||||
runtimeService
|
||||
"container-networks.target"
|
||||
"network-online.target"
|
||||
"agenix.service"
|
||||
"remote-fs.target"
|
||||
|
||||
@@ -4,6 +4,7 @@
|
||||
imports = [
|
||||
./docker.nix
|
||||
./podman.nix
|
||||
./networks.nix
|
||||
./container-services
|
||||
];
|
||||
options = {
|
||||
|
||||
@@ -0,0 +1,128 @@
|
||||
{ config, lib, pkgs, ... }:
|
||||
|
||||
let
|
||||
cfg = config.skyg.nixos.common.containers;
|
||||
|
||||
isDocker = cfg.runtime == "docker";
|
||||
runtimeBin = if isDocker then "${pkgs.docker}/bin/docker" else "${pkgs.podman}/bin/podman";
|
||||
runtimeService = if isDocker then "docker.service" else "podman.service";
|
||||
|
||||
enabledNetworks = lib.filterAttrs (_: net: net.enable) cfg.networks;
|
||||
|
||||
createArgs = net:
|
||||
lib.concatStringsSep " " (
|
||||
[ "network" "create" "--driver" net.driver ]
|
||||
++ lib.mapAttrsToList (k: v: "--opt ${k}=${v}") net.driverOpts
|
||||
++ lib.optional (net.subnet != null) "--subnet ${net.subnet}"
|
||||
++ lib.optional (net.gateway != null) "--gateway ${net.gateway}"
|
||||
++ lib.optional (net.ipRange != null) "--ip-range ${net.ipRange}"
|
||||
++ lib.optional net.internal "--internal"
|
||||
++ [ net.name ]
|
||||
);
|
||||
in
|
||||
{
|
||||
options.skyg.nixos.common.containers.networks = lib.mkOption {
|
||||
default = { };
|
||||
description = ''
|
||||
Container networks created on the host at boot, one systemd oneshot per
|
||||
network, independent of any compose project. This gives a single network
|
||||
that several container-services groups can share.
|
||||
|
||||
Reference one from a container-services group through its computed
|
||||
`compose` attribute:
|
||||
|
||||
networks.lan = config.skyg.nixos.common.containers.networks.lab.compose;
|
||||
|
||||
Works with both docker and podman (see runtime).
|
||||
'';
|
||||
type = lib.types.attrsOf (lib.types.submodule ({ name, config, ... }: {
|
||||
options = {
|
||||
enable = lib.mkEnableOption "creating this container network" // { default = true; };
|
||||
|
||||
name = lib.mkOption {
|
||||
type = lib.types.str;
|
||||
default = name;
|
||||
description = "Runtime name of the network.";
|
||||
};
|
||||
|
||||
driver = lib.mkOption {
|
||||
type = lib.types.str;
|
||||
default = "bridge";
|
||||
example = "macvlan";
|
||||
description = "Network driver.";
|
||||
};
|
||||
|
||||
driverOpts = lib.mkOption {
|
||||
type = lib.types.attrsOf lib.types.str;
|
||||
default = { };
|
||||
example = { parent = "eno1"; };
|
||||
description = "Driver options (--opt key=value), e.g. the macvlan parent interface.";
|
||||
};
|
||||
|
||||
subnet = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default = null;
|
||||
description = "IPv4 subnet (--subnet).";
|
||||
};
|
||||
|
||||
gateway = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default = null;
|
||||
description = "IPv4 gateway (--gateway).";
|
||||
};
|
||||
|
||||
ipRange = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default = null;
|
||||
description = "IPv4 range for dynamic address allocation (--ip-range).";
|
||||
};
|
||||
|
||||
internal = lib.mkOption {
|
||||
type = lib.types.bool;
|
||||
default = false;
|
||||
description = "Restrict the network to internal traffic (--internal).";
|
||||
};
|
||||
|
||||
compose = lib.mkOption {
|
||||
type = lib.types.attrsOf lib.types.anything;
|
||||
description = ''
|
||||
Computed compose `networks:` entry (external) for referencing this
|
||||
network from a container-services group.
|
||||
'';
|
||||
};
|
||||
};
|
||||
|
||||
config.compose = {
|
||||
name = config.name;
|
||||
external = true;
|
||||
};
|
||||
}));
|
||||
};
|
||||
|
||||
config = {
|
||||
# Always defined so container-services units can order themselves after it.
|
||||
systemd.targets.container-networks = {
|
||||
description = "Container networks";
|
||||
wantedBy = [ "multi-user.target" ];
|
||||
};
|
||||
|
||||
systemd.services = lib.mkIf cfg.enable (lib.mapAttrs'
|
||||
(n: net: lib.nameValuePair "container-network-${n}" {
|
||||
description = "Create container network '${net.name}'";
|
||||
after = [ runtimeService "network-online.target" ];
|
||||
requires = [ runtimeService ];
|
||||
wantedBy = [ "container-networks.target" ];
|
||||
before = [ "container-networks.target" ];
|
||||
serviceConfig = {
|
||||
Type = "oneshot";
|
||||
RemainAfterExit = true;
|
||||
};
|
||||
# Idempotent: reuse the network if it already exists.
|
||||
script = ''
|
||||
${runtimeBin} network inspect ${net.name} >/dev/null 2>&1 \
|
||||
|| ${runtimeBin} ${createArgs net}
|
||||
'';
|
||||
})
|
||||
enabledNetworks);
|
||||
};
|
||||
}
|
||||
@@ -13,6 +13,10 @@ in
|
||||
dockerSocket.enable = true;
|
||||
autoPrune.enable = true;
|
||||
};
|
||||
# Allow pulling from local registries that serve plain HTTP
|
||||
containers.registries.settings = lib.optionalAttrs (cfg.localDockerRegistries != [ ]) {
|
||||
registries.insecure.registries = cfg.localDockerRegistries;
|
||||
};
|
||||
};
|
||||
environment.systemPackages = with pkgs; [ podman-compose podman-tui ];
|
||||
};
|
||||
|
||||
@@ -5,14 +5,14 @@ staged on disk, and managed by a systemd oneshot unit.
|
||||
|
||||
## Where things live
|
||||
|
||||
| What | Path |
|
||||
| ---- | ---- |
|
||||
| Staged compose file | `/var/lib/container-services/<group>/compose.yml` |
|
||||
| What | Path |
|
||||
| -------------------------------------------------------------------------------- | ----------------------------------------------------------- |
|
||||
| Staged compose file | `/var/lib/container-services/<group>/compose.yml` |
|
||||
| Staged overrides file (composeFile groups with networks/volumes/extraConfig set) | `/var/lib/container-services/<group>/compose.overrides.yml` |
|
||||
| Nix store source | `/nix/store/…-compose.yml` (copied on each activation) |
|
||||
| Systemd unit | `container-services-<group>.service` |
|
||||
| Container logs | `docker compose -p <group> logs -f <service>` |
|
||||
| Unit journal | `journalctl -u container-services-<group> -f` |
|
||||
| Nix store source | `/nix/store/…-compose.yml` (copied on each activation) |
|
||||
| Systemd unit | `container-services-<group>.service` |
|
||||
| Container logs | `docker compose -p <group> logs -f <service>` |
|
||||
| Unit journal | `journalctl -u container-services-<group> -f` |
|
||||
|
||||
## Quick reference
|
||||
|
||||
@@ -51,6 +51,7 @@ journalctl -u container-services-<group> -n 50
|
||||
```
|
||||
|
||||
Common causes:
|
||||
|
||||
- **Docker not running** — `systemctl status docker`
|
||||
- **Secret path missing** — agenix may not have decrypted yet; check `systemctl status agenix`
|
||||
- **Port already in use** — another service holds the host port; `ss -tlnp | grep <port>`
|
||||
@@ -123,6 +124,7 @@ services.app = {
|
||||
```
|
||||
|
||||
Files are:
|
||||
|
||||
- Written to `/var/lib/container-services/<group>/files/` on the host
|
||||
- Mounted read-only into the container at the specified paths
|
||||
- Updated automatically when you rebuild
|
||||
@@ -155,17 +157,19 @@ skyg.nixos.common.container-services.my-stack = {
|
||||
```
|
||||
|
||||
When `composeFile` is set, the module:
|
||||
|
||||
- Copies that file to `/var/lib/container-services/<group>/compose.yml` at
|
||||
startup, just like a rendered one. It must define `services` itself —
|
||||
ignores this group's `services`/`files` blocks.
|
||||
- If this group also sets `networks`/`volumes`/`extraConfig`, those are
|
||||
rendered to `/var/lib/container-services/<group>/compose.overrides.yml` and
|
||||
passed to compose **before** the composeFile (`-f compose.overrides.yml -f
|
||||
compose.yml`), so compose's native multi-file merge applies —
|
||||
compose.yml`), so compose's native multi-file merge applies —
|
||||
composeFile's own definitions win on any overlapping keys.
|
||||
- Requires exactly one of `composeFile` or `services` to be set (assertion).
|
||||
|
||||
Notes:
|
||||
|
||||
- `env_file:` entries in the external file resolve **relative to**
|
||||
`/var/lib/container-services/<group>/`. Use absolute paths (e.g. another
|
||||
agenix secret path) or inline `environment:` instead.
|
||||
@@ -173,6 +177,44 @@ Notes:
|
||||
|
||||
---
|
||||
|
||||
## Shared Networks (created on the host)
|
||||
|
||||
Compose networks declared inside a group are scoped to that group's compose project, so two
|
||||
groups would each get their own copy. To share **one** network across several stacks (e.g. a
|
||||
single macvlan network), declare it once with `skyg.nixos.common.containers.networks` and
|
||||
reference it as `external` via its computed `compose` attribute:
|
||||
|
||||
```nix
|
||||
# once, e.g. in the host file
|
||||
skyg.nixos.common.containers.networks.lab = {
|
||||
driver = "macvlan";
|
||||
driverOpts.parent = "eno1";
|
||||
subnet = "10.0.0.0/16";
|
||||
gateway = "10.0.0.1";
|
||||
ipRange = "10.0.101.16/28";
|
||||
};
|
||||
|
||||
skyg.nixos.common.container-services.my-stack = {
|
||||
enable = true;
|
||||
networks = {
|
||||
internal = { driver = "bridge"; };
|
||||
lan = config.skyg.nixos.common.containers.networks.lab.compose;
|
||||
};
|
||||
services.app = {
|
||||
image = "example/app:latest";
|
||||
networks = {
|
||||
internal = { };
|
||||
lan = { ipv4_address = "10.0.101.3"; };
|
||||
};
|
||||
};
|
||||
};
|
||||
```
|
||||
|
||||
The network is created by a host systemd unit (`container-network-<name>.service`) before any
|
||||
group starts; groups only reference it (`external: true`) and never create it.
|
||||
|
||||
---
|
||||
|
||||
## Limitations
|
||||
|
||||
- **One unit per group.** No per-container systemd units. Use `docker compose logs` for
|
||||
|
||||
Reference in new issue
Block a user