clean up and docs
This commit is contained in:
1 parent
d3963f8cc3
commit
6a7d35442f
3 files changed
+352
-5
No files matched your search
Vendored
+2
-1
@@ -4,5 +4,6 @@
|
|||||||
},
|
},
|
||||||
"cSpell.words": [
|
"cSpell.words": [
|
||||||
"authelia"
|
"authelia"
|
||||||
]
|
],
|
||||||
|
"nixEnvSelector.suggestion": false
|
||||||
}
|
}
|
||||||
@@ -1,11 +1,38 @@
|
|||||||
# (My) Homelab
|
# (My) Homelab
|
||||||
|
|
||||||
|
A GitOps-driven Kubernetes (k3s) homelab, provisioned with **Nix**, **Terraform**, **Helm**, and **Argo CD**.
|
||||||
|
|
||||||
|
Argo CD watches this repo and syncs the applications defined under `charts/`, while Terraform bootstraps the cluster plugins and manages Cloudflare DNS.
|
||||||
|
|
||||||
|
## Documentation
|
||||||
|
|
||||||
|
See **[`docs/DOCUMENTATION.md`](docs/DOCUMENTATION.md)** for the full guide covering:
|
||||||
|
|
||||||
|
- Architecture overview (Terraform + Argo CD + GitOps flow)
|
||||||
|
- Getting started / prerequisites
|
||||||
|
- Terraform configuration & variables
|
||||||
|
- The `bin/dev` helper CLI
|
||||||
|
- Every Helm chart (`infrastructure`, `apps`, `ingress`, `argoapps`)
|
||||||
|
- Secrets management & External Secrets
|
||||||
|
- OS configuration (Raspberry Pi, OpenWrt routers)
|
||||||
|
|
||||||
|
### Quick status
|
||||||
|
|
||||||
|
| Area | Tooling |
|
||||||
|
|------|---------|
|
||||||
|
| Dev environment | Nix (`flake.nix`) |
|
||||||
|
| Provisioning | Terraform (`tf-*.tf`) |
|
||||||
|
| Cluster plugins | Helm via `tf-modules/k8s-helm-charts` |
|
||||||
|
| GitOps | Argo CD (`charts/infrastructure`) |
|
||||||
|
| DNS | Cloudflare (`tf-dns.tf`) |
|
||||||
|
|
||||||
## Repository Structure
|
## Repository Structure
|
||||||
|
|
||||||
- `bin/` – helper scripts
|
- `bin/` – helper scripts (CLI at `bin/dev`)
|
||||||
- `certs/` – TLS certificates
|
- `certs/` – TLS certificates
|
||||||
- `charts/` – Helm charts
|
- `charts/` – Helm charts (`apps/`, `infrastructure/`)
|
||||||
- `os/` – OS‑specific configuration
|
- `docs/` – documentation
|
||||||
|
- `os/` – OS-specific configuration
|
||||||
- `tf-modules/` – reusable Terraform modules
|
- `tf-modules/` – reusable Terraform modules
|
||||||
- `tf-*.tf` – Terraform configuration files
|
- `tf-*.tf` – Terraform configuration files
|
||||||
|
|
||||||
@@ -18,8 +45,11 @@ This repo uses Nix + Terraform to provision a homelab infrastructure.
|
|||||||
3. Review the plan: `terraform plan`
|
3. Review the plan: `terraform plan`
|
||||||
4. Apply the configuration: `terraform apply`
|
4. Apply the configuration: `terraform apply`
|
||||||
|
|
||||||
|
> Requires a k3s cluster configured as the `homelab` kubectl context, plus Cloudflare API tokens.
|
||||||
|
|
||||||
## Usage
|
## Usage
|
||||||
|
|
||||||
- Edit `var.auto.tfvars` for environment‑specific values.
|
- Edit `var.auto.tfvars` for environment-specific values.
|
||||||
- Run `./bin/` scripts to deploy services.
|
- Run `./bin/` scripts to deploy services.
|
||||||
- Use `terraform state` commands to inspect resources.
|
- Use `terraform state` commands to inspect resources.
|
||||||
|
- Argo CD auto-syncs applications from `charts/apps/*`.
|
||||||
@@ -0,0 +1,316 @@
|
|||||||
|
# HomeLab
|
||||||
|
|
||||||
|
> GitOps-driven Kubernetes (k3s) homelab infrastructure, provisioned with **Nix**, **Terraform**, **Helm**, and **Argo CD**.
|
||||||
|
|
||||||
|
This repository provisions and manages the author's home cluster. It is built around three layers:
|
||||||
|
|
||||||
|
1. **Bootstrap** – a Nix dev shell provides all tooling (`terraform`, `helm`, `kubectl`, `k3d`, `argocd`, etc.).
|
||||||
|
2. **Provisioning** – Terraform installs the cluster "plugins" (Argo CD, MetalLB, cert-manager, Traefik, External Secrets, Tailscale operator) directly on the cluster and configures DNS on Cloudflare.
|
||||||
|
3. **GitOps** – Argo CD watches this repo and syncs the applications in `charts/apps/*` and `charts/infrastructure/*`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Repository Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
.
|
||||||
|
├── bin/ # Helper CLI scripts (dev)
|
||||||
|
│ └── extra/common.sh # Shared helpers + CLI dispatch
|
||||||
|
├── certs/ # TLS certificates (currently empty)
|
||||||
|
├── charts/ # Helm charts
|
||||||
|
│ ├── apps/ # Application charts (deployed via Argo CD ApplicationSet)
|
||||||
|
│ │ ├── argoapps/ # Extra Argo applications (compose apps, NFS storage)
|
||||||
|
│ │ ├── authelia/ # Authentication / SSO
|
||||||
|
│ │ ├── certs/ # Certificate management (cert-manager issuers)
|
||||||
|
│ │ ├── dummy/ # Example/demo chart
|
||||||
|
│ │ ├── ingress/ # Reverse proxies, services & ingress rules
|
||||||
|
│ │ └── torrents/ # Torrenting stack
|
||||||
|
│ └── infrastructure/ # Cluster-level manifests (Argo CD, MetalLB, secrets)
|
||||||
|
├── os/ # OS-specific configuration
|
||||||
|
│ ├── raspberrypi2-raspbian # ARM device config (compose) that can't run Nix
|
||||||
|
│ └── routers/ # OpenWrt router config (mesh & themes)
|
||||||
|
├── tf-modules/ # Reusable Terraform modules
|
||||||
|
│ └── k8s-helm-charts/ # Generic Helm release install module
|
||||||
|
├── tf-*.tf # Terraform root configuration
|
||||||
|
├── flake.nix # Nix dev shell definition
|
||||||
|
├── flake.lock # Pinned Nix inputs
|
||||||
|
└── var.auto.tfvars # Environment-specific variables (gitignored pattern)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Getting Started
|
||||||
|
|
||||||
|
The environment is set up via Nix and provisioned with Terraform.
|
||||||
|
|
||||||
|
### Prerequisites
|
||||||
|
|
||||||
|
- [nix](https://nixos.org/) with flakes enabled
|
||||||
|
- A k3s cluster, configured as the `homelab` kubectl context
|
||||||
|
- Cloudflare account + API tokens (for DNS & cert-manager DNS-01)
|
||||||
|
|
||||||
|
### Setup
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# 1. Enter the Nix dev shell (sets up all tooling + PATH)
|
||||||
|
nix develop
|
||||||
|
|
||||||
|
# 2. Provide secrets and environment values
|
||||||
|
# Edit var.auto.tfvars (Cloudflare tokens, Gitea, Tailscale, VPN, globals...)
|
||||||
|
|
||||||
|
# 3. Initialise Terraform and apply
|
||||||
|
terraform init
|
||||||
|
terraform plan
|
||||||
|
terraform apply
|
||||||
|
```
|
||||||
|
|
||||||
|
> **Note:** `var.auto.tfvars` is matched by `*.tfvars*` in `.gitignore`, so the checked-in file won't be committed by default. Be careful — secrets should be kept out of version control.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Architecture Overview
|
||||||
|
|
||||||
|
### Key flows
|
||||||
|
|
||||||
|
- **DNS:** Terraform looks up your current IP (`https://api.myip.com/`) and creates `A` records on Cloudflare for every zone/record in `domain_mappings` (unless an IP is pinned).
|
||||||
|
- **Plugins:** Terraform uses the `k8s-helm-charts` module to install `external-secrets`, `metallb`, `cert-manager`, `argo-cd`, `tailscale-operator`, and `traefik`. Each lands in its own `plugin-<name>` namespace.
|
||||||
|
- **Infrastructure:** Terraform deploys `charts/infrastructure`, which creates the Argo CD repo credentials, `AppProject main`, the ApplicationSet that watches `charts/apps/*`, the MetalLB IP address pool, and External Secrets ClusterSecretStores.
|
||||||
|
- **GitOps:** The ApplicationSet automatically creates an Argo CD Application for each folder under `charts/apps/`, syncing it to its own namespace. Value files `values.yaml` + `values-<deployment>.yaml` are merged per deployment environment.
|
||||||
|
- **Load balancer:** Argo CD server is exposed on `10.0.100.2` (MetalLB pool) via the `kubernetes_service_v1.argocd-metallb` resource.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Terraform
|
||||||
|
|
||||||
|
### Providers
|
||||||
|
|
||||||
|
| Provider | Version | Purpose |
|
||||||
|
|----------|---------|---------|
|
||||||
|
| `hashicorp/helm` | `3.0.0-pre1` | Install Helm charts directly |
|
||||||
|
| `hashicorp/kubernetes` | `2.35.1` | Create namespaces, secrets, services |
|
||||||
|
| `hashicorp/http` | `3.4.5` | Discover public IP |
|
||||||
|
| `cloudflare/cloudflare` | `5.2.0` | Manage DNS records |
|
||||||
|
| `hashicorp/random` | `3.7.2` | Random values |
|
||||||
|
|
||||||
|
The Terraform **state backend** is stored in a Kubernetes secret (`secret_suffix = "homelab"` in the `default` namespace) on the `homelab` context.
|
||||||
|
|
||||||
|
### Key configuration files
|
||||||
|
|
||||||
|
| File | Purpose |
|
||||||
|
|------|---------|
|
||||||
|
| `tf-provider.tf` | Backend + provider configuration |
|
||||||
|
| `tf-variables.tf` | All input variables (see below) |
|
||||||
|
| `tf-dns.tf` | Cloudflare DNS records + public IP lookup |
|
||||||
|
| `tf-modules.tf` | Plugin Helm releases + `infrastructure` chart |
|
||||||
|
| `tf-ns-ingress.tf` | Argo CD server LoadBalancer service |
|
||||||
|
| `tf-secrets.tf` | VPN credentials secret in `secrets` namespace |
|
||||||
|
| `tf-svc.tf` | Service-related declarations |
|
||||||
|
| `var.auto.tfvars` | Environment values (DNS maps, globals, secrets) |
|
||||||
|
|
||||||
|
### Input variables
|
||||||
|
|
||||||
|
| Variable | Type | Description |
|
||||||
|
|----------|------|-------------|
|
||||||
|
| `myip` | `string` | Pin your public IP (default: auto-detected) |
|
||||||
|
| `cloudflare` | `object` (sensitive) | Cloudflare `api_token`, `account_id`, `cm_api_token` |
|
||||||
|
| `main_domain` / `secondary_domain` | `string` | Defaults `sosnovsky.ca` / `skyg.ca` |
|
||||||
|
| `domain_mappings` | `map(map(...))` | DNS zone → records → type/IP |
|
||||||
|
| `gitea` | `object` (sensitive) | GitOps repo username/password |
|
||||||
|
| `github_ssh_key` | `string` (sensitive) | SSH key for the helm-charts git repo |
|
||||||
|
| `tailscale` | `object` (sensitive) | Tailscale OAuth client id/secret |
|
||||||
|
| `vpn` | `object` (sensitive) | VPN user/password/provider/country |
|
||||||
|
| `globals` | `object` | Global app config (DNS, NFS, SMTP, deployment, email, timezone) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## `bin/dev` CLI
|
||||||
|
|
||||||
|
The `bin/dev` script is a small self-documenting CLI built on `common.sh`. Each subcommand matches a `run-<name>` function; running `bin/dev` with no args lists them with their doc comments.
|
||||||
|
|
||||||
|
| Command | Description |
|
||||||
|
|---------|-------------|
|
||||||
|
| `dev new-chart <path>` | Scaffold a new Helm chart under `charts/apps/` (Chart.yaml + values.yaml + templates/) |
|
||||||
|
| `dev upload-tfvars` | Push `var.auto.tfvars` into a `tfvars` secret in the `default` namespace |
|
||||||
|
| `dev download-tfvars` | Pull the `tfvars` secret back into `var.auto.tfvars` |
|
||||||
|
| `dev template <app>` | Render a deployed Argo app's Helm output locally (reads its spec from Argo CD) |
|
||||||
|
| `dev template-compose-app <name>` | Render a compose-app chart under `charts/apps/argoapps/compose-apps/` |
|
||||||
|
| `dev htpasswd [...]` | Generate an Apache-style htpasswd hash (`openssl passwd -apr1`) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Helm Charts
|
||||||
|
|
||||||
|
### `charts/infrastructure`
|
||||||
|
|
||||||
|
Cluster-scoped bootstrap resources. Version `0.1.15`. Rendered values come from `tf-modules.tf` (`repos.*`, `gitops.*`, `ns.*`, `global`).
|
||||||
|
|
||||||
|
Templates:
|
||||||
|
|
||||||
|
| Template | Contents |
|
||||||
|
|----------|----------|
|
||||||
|
| `argocd-appset.yml` | ApplicationSet that watches `charts/apps/*` in the gitops repo |
|
||||||
|
| `argocd-projects.yaml` | `main` AppProject (cluster/destination/source allow-all) |
|
||||||
|
| `argocd-repo-gitops.yaml` | Repository secret for the homelab gitops repo |
|
||||||
|
| `argocd-repo-additionals.yaml` | Additional repo secrets (e.g. GitHub `helm-charts`) |
|
||||||
|
| `es-cluster-store.yaml` | External Secrets ClusterSecretStore + RBAC per namespace |
|
||||||
|
| `ips.yml` | MetalLB `IPAddressPool` + `L2Advertisement` |
|
||||||
|
|
||||||
|
Key default `values.yaml`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
registry:
|
||||||
|
ip: 10.0.100.101
|
||||||
|
ns:
|
||||||
|
metallb: "change-me"
|
||||||
|
argocd: "change-me"
|
||||||
|
gitops:
|
||||||
|
url: http://minipc1.lab.internal:3000/ari/homelab.git
|
||||||
|
revision: main
|
||||||
|
data: { username: argocd, password: "change-me" }
|
||||||
|
repos:
|
||||||
|
github-helm-charts:
|
||||||
|
url: git@github.com:asosnovsky/helm-charts.git
|
||||||
|
metallb:
|
||||||
|
addresses:
|
||||||
|
- 10.0.100.1-10.0.100.255
|
||||||
|
enabledClusterSecretStores:
|
||||||
|
- secrets
|
||||||
|
```
|
||||||
|
|
||||||
|
### `charts/apps` (deployed by the ApplicationSet)
|
||||||
|
|
||||||
|
Each folder is auto-deployed by the ApplicationSet into a namespace named after the folder. Value merge order: `values.yaml`, then `values-<deployment>.yaml` (`dev`/`prod`), then inline `global`.
|
||||||
|
|
||||||
|
| Chart | Purpose |
|
||||||
|
|-------|---------|
|
||||||
|
| `authelia` | SSO / authentication portal |
|
||||||
|
| `certs` | Certificate management (issuers for TLS) |
|
||||||
|
| `dummy` | Minimal example app |
|
||||||
|
| `ingress` | Reverse proxies, cluster services, and ingress rules (Traefik + Tailscale) |
|
||||||
|
| `torrents` | Torrenting stack |
|
||||||
|
| `argoapps` | **Not** a normal app — creates additional Argo CD Applications (see below) |
|
||||||
|
|
||||||
|
### `charts/apps/ingress`
|
||||||
|
|
||||||
|
Renders several kinds of resources from `reverseProxy`, `services`, and `tailScaleServices` values:
|
||||||
|
|
||||||
|
- **Reverse proxies** — `Endpoints` + `Service` pointing at an external IP (e.g. Jellyfin, Scrypted, a "castle" box), plus optional external ingress and/or a Tailscale ingress.
|
||||||
|
- **Services** — ingress rules for in-cluster services via the `homelab.ingress` helper.
|
||||||
|
- **Tailscale services** — ingress via `homelab.ingress.tailscale`.
|
||||||
|
- Also provisions an Argo CD `Application` for the **homepage** dashboard.
|
||||||
|
|
||||||
|
Example snippet (`values-prod.yaml`):
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
reverseProxy:
|
||||||
|
jellyfin:
|
||||||
|
ip: 10.0.10.5
|
||||||
|
port: 8096
|
||||||
|
services:
|
||||||
|
- prefix: "auth"
|
||||||
|
service: "authelia"
|
||||||
|
namespace: "authelia"
|
||||||
|
group: "Admin"
|
||||||
|
tailScaleServices:
|
||||||
|
- name: "argocd"
|
||||||
|
namespace: plugin-argo-cd
|
||||||
|
selector: { "app.kubernetes.io/name": "argocd-server" }
|
||||||
|
port: 8080
|
||||||
|
```
|
||||||
|
|
||||||
|
### `charts/apps/argoapps`
|
||||||
|
|
||||||
|
This is a special chart **needed by Argo CD itself** (bootstrapped via `infrastructure`), which then creates further Applications:
|
||||||
|
|
||||||
|
- `compose-apps.yaml` — for each file in `compose-apps/*.yaml`, generates an `Application` pointing at `charts/compose-app` in the external `helm-charts` repo. Each compose-app is essentially a Docker Compose stack (e.g. `arrs`, `immich`, `paperless-ngx`, `smartboi`) deployed into a `comap-<name>` namespace.
|
||||||
|
- `nfs.yaml` — an `Application` for the `nfs-subdir-external-provisioner` (external GitHub repo) hosting the NFS storage classes.
|
||||||
|
|
||||||
|
Compose-apps are defined as docker-compose files; examples:
|
||||||
|
|
||||||
|
| File | Stack |
|
||||||
|
|------|-------|
|
||||||
|
| `arrs.yaml` | Media stack: sonarr, radarr, lidarr, prowlarr, nzbget, flaresolverr + postgres |
|
||||||
|
| `immich.yaml` | Photo library |
|
||||||
|
| `paperless-ngx.yaml` | Document management |
|
||||||
|
| `smartboi.yaml` | Smart device assistant |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Terraform module: `tf-modules/k8s-helm-charts`
|
||||||
|
|
||||||
|
A generic, reusable module that installs a map of Helm releases.
|
||||||
|
|
||||||
|
```hcl
|
||||||
|
module "k8s-helm-charts" {
|
||||||
|
source = "./tf-modules/k8s-helm-charts"
|
||||||
|
plugins = {
|
||||||
|
cert-manager = {
|
||||||
|
repository = "https://charts.jetstack.io"
|
||||||
|
version = "1.19.2"
|
||||||
|
values = [yamlencode({ installCRDs = true })]
|
||||||
|
}
|
||||||
|
# ...
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- One `helm_release` per key, `create_namespace = true`, default namespace `plugin-<key>`.
|
||||||
|
- Exposes an output `namespaces.plugins.<key>` with each release's namespace (used elsewhere in the root config).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Secrets Management
|
||||||
|
|
||||||
|
Secrets flow through multiple channels:
|
||||||
|
|
||||||
|
1. **Terraform provider vars** — declared `sensitive` in `tf-variables.tf`; supplied via `var.auto.tfvars`.
|
||||||
|
2. **Direct Kubernetes secrets** — e.g. the VPN credentials in `tf-secrets.tf`.
|
||||||
|
3. **Kubernetes secrets for Argo/plugins** — Cloudflare API token (`tf-ns-ingress.tf`/certs), git repo credentials (via the `infrastructure` chart).
|
||||||
|
4. **External Secrets Operator** — `es-cluster-store.yaml` creates a `ClusterSecretStore` per namespace (RBAC + ServiceAccount) so apps can pull secrets cross-namespace from e.g. the `secrets` namespace.
|
||||||
|
|
||||||
|
> **Security note:** `var.auto.tfvars` holds live credentials (Cloudflare tokens, GitHub SSH key, Tailscale OAuth, VPN creds). It is **correctly excluded from Git** by `.gitignore` (`*.tfvars*`) and is not tracked — verified with `git check-ignore`. Keep it out of version control. For additional hardening, consider moving these secrets off-disk to a secret manager (External Secrets + Vault) or encrypting the file with `sops`/`git-crypt`.
|
||||||
|
|
||||||
|
> **Also note:** the provider pin uses Helm `version = "3.0.0-pre1"` (a pre-release). Upgrade to a stable Helm provider release when possible.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## OS Configuration
|
||||||
|
|
||||||
|
### `os/raspberrypi2-raspbian`
|
||||||
|
|
||||||
|
Configuration for a Raspberry Pi 2 running Raspbian. This ARM device can't run Nix or the `linuxserver` docker images, so it's given a plain `compose.yaml` plus helper scripts. Contains a `systemd/` directory.
|
||||||
|
|
||||||
|
### `os/routers`
|
||||||
|
|
||||||
|
OpenWrt router configuration:
|
||||||
|
|
||||||
|
- `openwrt-mesh/` — mesh router setup in `main` or `ap` (access point) modes. `push.sh` copies the generated `create-<mode>.sh` + `.env` to the router over SSH and runs it. Requires a local `.env` (gitignored).
|
||||||
|
- `openwrt-themes/` — OpenWrt theme material overrides.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## GitHub Actions
|
||||||
|
|
||||||
|
`ossar.yml` runs the OSSAR static-analysis scan on the `stable` branch (push, PR, and weekly). Results are uploaded to the GitHub Security tab as SARIF.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Contributing Workflow
|
||||||
|
|
||||||
|
1. **Add a chart:** `dev new-chart <name>` scaffolds `charts/apps/<name>` where Argo CD will deploy it into the `<name>` namespace.
|
||||||
|
2. **Set values:** provide `values.yaml` and `values-dev.yaml`/`values-prod.yaml` (see `insert` chart for a worked example).
|
||||||
|
3. **Deploy:** commit to the gitops branch configured in `charts/infrastructure/values.yaml` (`gitops.revision`); Argo CD's ApplicationSet picks it up automatically.
|
||||||
|
4. **Secrets:** store them behind External Secrets / a `ClusterSecretStore` rather than in tfvars.
|
||||||
|
5. **Validate locally:** `dev template <app>` renders the deployed chart for review before pushing.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Common Tasks
|
||||||
|
|
||||||
|
| Task | Command |
|
||||||
|
|------|---------|
|
||||||
|
| Enter dev environment | `nix develop` |
|
||||||
|
| Plan / apply Terraform | `terraform plan` / `terraform apply` |
|
||||||
|
| Sync Argo app status | `kubectl -n plugin-argo-cd get applications.argoproj.io` |
|
||||||
|
| Render an app's Helm output | `dev template <app>` |
|
||||||
|
| Scaffold a new chart | `dev new-chart <name>` |
|
||||||
Reference in new issue
Block a user