colmena

A Colmena hive that manages a small Tailscale mesh of NixOS machines. All hosts are connected via Headscale (MagicDNS mesh.net), deployed from the laptop over the mesh, and configured declaratively with sops-nix secrets.

Machines

Host	Location	Always-on	Role
remotelab	internet-exposed VPS	yes	Mesh control server + public services
homelab	home	yes	Primary data (ZFS) + NFS to laptop
backup	home	yes	Replica + offsite-ish copy of everything
console	home	no	Gaming (jovian/Steam)
tv	remote	no	Media device (jellyfin-mpv-shim)

remotelab — the public edge

The only host exposed to the internet (ports 80/443 only). Runs the mesh infrastructure and public-facing services:

• headscale — the Tailscale control server (hs.dymc.win); defines the mesh.
• ntfy-sh — push notification server (ntfy.dymc.win); every host reports backup/update results here.
• forgejo — git server (git.dymc.win); hosts this repo.
• caddy — reverse proxy + TLS for the above, and static sites (lizmclaren.com, dymc.win).
• pihole — DNS resolver for the mesh.
• fail2ban — jails for sshd, forgejo, ntfy.
• auto-update — weekly colmena apply-local pulled from Forgejo (reboots if the kernel changed).
• restic → Hetzner: forgejo/headscale/ntfy-sh state + ssh host keys.

Because it's internet-exposed, remotelab is treated as untrusted on the mesh (see ACL below): it can only reach DNS, and everything else must initiate to it.

homelab — the data

Primary storage. Runs ZFS (warhead pool), sanoid snapshots (30 daily), and exports NFS shares to the laptop over the mesh.

• restic → Hetzner (warhead/high-prio + ssh host keys).
• ZFS replication of warhead/high-prio is pulled by backup (syncoid), so homelab holds no credentials to the replica.

Also runs pihole, beszel-agent, ttyd, docker, zfs-scrub.

backup — the replica

Pulls homelab's replication (warhead/high-prio, pruned to 30 daily snapshots) and pulls remotelab's state (forgejo/headscale/ntfy-sh + ssh keys) into a warhead/remotelab dataset — pull direction only, so neither homelab nor remotelab has access to backup. Its own restic → Hetzner covers its ssh host keys.

Candidate to move offsite (e.g. a friend's house) to give a second independent location.

console / tv

Recreational. console is a Steam/gamescope box (jovian); tv runs a headless jellyfin-mpv-shim client. Both expose ttyd on the tailnet.

Interactions

• Mesh — headscale (on remotelab) provides control plane + MagicDNS; piholes on remotelab/homelab/backup resolve *.mesh.net.
• ACL (embedded in hosts/remotelab.nix's services.headscale.settings.policy, written to the store and referenced by path) — headscale policy. tag:trusted nodes (everything except remotelab and the phones) talk freely; remotelab is implicitly denied from initiating to the tailnet (only DNS + its own services inbound). Trust new machines with headscale nodes tag -i <id> -t tag:trusted.
• Backups — every host's backup/update job notifies via ntfy on remotelab (auth'd). Restic jobs include an integrity check (--read-data-subset=10%).
• Secrets — sops-nix; age keys derived from each host's ssh host key (which are themselves backed up in restic, so a rebuilt host can still decrypt).
• Deployment — colmena apply from the laptop over the mesh; remotelab also self-updates via colmena apply-local from the Forgejo copy of this repo.

Updating this document

Update this overview whenever the topology, services, or backup flows change — it's the reference for what this repo manages.