Terraform¶
Terraform manages what is around the cluster: Proxmox, the Talos VMs, PBS, the Tailscale router, UniFi and Cloudflare. Each stack is its own root with its own state, run by hand from a laptop. Argo CD takes over after the Talos stack has installed it.
Stacks¶
| Root | Manages | State key |
|---|---|---|
azure/state |
the storage account that holds every other stack's state | critical/do-not-delete-state-backend.tfstate |
proxmox/hyper-cluster/datacenter |
Proxmox VE Datacenter and node level (proxmox.md) |
proxmox/hyper-cluster/datacenter.tfstate |
proxmox/hyper-cluster/k8s/talos |
the six Talos VMs, machine config, Cilium and the first Argo CD install (talos.md) |
proxmox/hyper-cluster/talos-genesis.tfstate |
proxmox/hyper-cluster/tailscale |
the subnet router VM and the tailnet policy (remote-access.md) |
proxmox/hyper-cluster/tailscale.tfstate |
proxmox/pbs |
Proxmox Backup Server (pbs.md) |
proxmox/pbs.tfstate |
unifi/network |
the UniFi site (unifi.md) |
unifi/network.tfstate |
unifi/dns |
local.bigd.no records (dns.md) |
unifi/dns.tfstate |
cloudflare/nordbye-it, bigd-no, logeverylift-com |
the three public zones (cloudflare.md) |
cloudflare/<stack>.tfstate |
cloudflare/watchdog |
the watchdog Worker | cloudflare/watchdog.tfstate |
The Proxmox and PBS stacks log in with Proxmox tokens that are created by hand.
BOOTSTRAP.md has the steps and the order to
apply pbs and datacenter in.
State backend¶
State lives in Azure: resource group rg-tfstate-homelab, storage account
sttfstatemvnhomelab, container tfstate, with blob versioning on
(main.tf). Every other root names these in a
backend "azurerm" block with its own key. azure/state stores its own state in the
same container. It uses an empty backend "azurerm" {} that a gitignored backend.conf
fills (version.tf). Its README has the
two-stage bootstrap.
The backend blocks carry no credentials, so the azurerm backend authenticates with the
Azure CLI login the shell has. The azurerm provider in azure/state sets use_cli = true
for the same reason. That login (az login, in the
azure/state README) is done by hand before you
run Terraform. Nothing in the repo automates it.
Local files¶
Git holds a terraform.tfvars.example for every stack but azure/state. The real values are gitignored
(*.tfvars, backend.conf, kubeconfig, talosconfig in
.gitignore) and exist only on the laptop that runs Terraform.
| Stack | Needs | Writes |
|---|---|---|
azure/state |
terraform.tfvars (subscription_id), backend.conf |
|
proxmox/hyper-cluster/datacenter |
terraform.tfvars (proxmox_api_token, pbs_backup_token) |
|
proxmox/hyper-cluster/k8s/talos |
terraform.tfvars (Proxmox token, nodes, versions) |
kubeconfig, talosconfig |
proxmox/hyper-cluster/tailscale |
terraform.tfvars (Proxmox token, SSH password, Tailscale OAuth client) |
|
proxmox/pbs |
terraform.tfvars (pbs_api_token) |
|
unifi/network, unifi/dns |
terraform.tfvars (unifi_api_key; network also the Wi-Fi passphrases and DDNS tokens) |
|
cloudflare/* |
terraform.tfvars (cloudflare_api_token) |
|
cloudflare/watchdog |
also account_id in terraform.tfvars, and BW_ACCESS_TOKEN and BW_ORGANIZATION_ID in the shell for the Bitwarden provider |
The Talos stack's kubeconfig and talosconfig are the cluster's admin credentials.
convert-secrets.sh
turns the stack's talos_secrets output into a talosctl secrets bundle.
Backing up the local files¶
scripts/backup-secrets.sh zips the paths listed in
.backup-manifest and the paths between the # backup:start
and # backup:end markers in .gitignore. It copies the zip over SMB to the NAS share
shared-data under documents/IT/Repo-Hidden-Files-Backups/Homelab/ and keeps the newest
14. Only paths are in the repo, never contents. The manifest lists the Talos kubeconfig,
talosconfig and controlplane.yaml, every stack's terraform.tfvars, and
azure/state/backend.conf. A new stack's gitignored files need a line there.
scripts/backup-secrets.sh --list # what would be backed up, and what is missing
scripts/backup-secrets.sh --dry-run # build the zip locally, no upload
scripts/backup-secrets.sh # prompts for the NAS login, uploads, verifies
Restore is unzip <file>.zip at the repo root. Where it fits among the other backups:
../backups/README.md.
CI¶
ci-terraform.yaml runs on pull requests
and pushes to main that touch terraform/. It runs terraform fmt -check -recursive, then
init -backend=false and validate in every directory that holds a .tf file, so a new
root is covered without a change to the workflow. It has no credentials and never reads
state. plan and apply stay on the laptop, because the Proxmox and UniFi APIs are LAN-only.