Snapshots largely, most everything is VMs and docker containers. I have one VM set aside for dev work to test configs before updating the prod boxes as well.

With NixOS, you get a reproducible environment. When you need to change your hardware, you simply back up your data, write your NixOS configuration, and you can reproduce your previous environment.

I use it to manage all my services.

reproducible

You tried writing bash scripts that set things up for you, haven’t you? It’s NixOS for you.

Terraform and Puppet. Not very simple to get into, but extremely powerful and reliable.

I’m just using Unraid for the server, after many iterations (PhotonOS, VMware, baremetal Windows Server, …). After many OSes, partial and complete hardware replacements, and general problems, I gave up trying to manage the base server too much. Backups are generally good enough if hardware fails or I break something.

The other side of this is that I’ve moved to having very, very little config on the server itself. Virtually everything of value is in a docker container with a single (admittedly way too large) docker compose file that describes all the services.

I think this is the ideal way for how I use a home server. Your mileage might vary, but I’ve learned the hard way that it’s really hard to maintain a server over the very long term and not also marry yourself to the specific hardware and OS configuration.

How do you manage your home server configuration

Poorly, which is to say that I just let borgmatic back up all my compose files and hope for the best

I got a bunch of docker compose files and the envs documented so its easy to spin things up again or rollback changes. It works well enough if I’m good about keeping everything all up to date and not making changes without noting it down for myself later.

I’m the only user of my setup, but I configure docker compose stacks, use configs as bind mounts, and track everything in a git repo synchronized every now and then.

I went the nuclear option and am using Talos with Flux to manage my homelab.

My source of truth is the git repo with all my cluster and application configs. With this setup, I can tear everything down and within 30 min have a working cluster with everything installed automatically.

I use snapshots, once a month an image is made of the entire drive, and I have Duplicati that backs up to cloud. Whatever choice you make tho, remember 3,2,1, and backups are useless unless tested on a regular basis. The test portion always gives me anxiety.

Well I use Unraid, so I just back up my whole config folder along with the OS itself in case I need to flash it to a new USB. In other words, I just clone the whole thing. It means I can be up and running in a few minutes if everything was corrupted.

A data drive loss is pretty simple too, I just simulate the lost data until I can get a new HDD in. That takes a little longer to fix tho.

Recently switched to ucore. While I cannot for the life of me get SELinux to let my containers run without Permissive mode (my server was previously Endeavour OS and either didn’t have it or I disabled it long ago), I’ve otherwise had great success.

The config is a single yaml file that gets converted into a json file for Ignition, which sets everything up on first boot. It’s an OCI-based immutable distro with automatic updating, so I can mostly just leave it to its own devices and everything has been smooth for the first week I’ve been using it.

My Docker root directory is on a separate drive with plenty of space, so setting up involves directing Docker to that new root directory and basically being done (which my Ignition config handles for me).

I use git and commit configs/setup/scripts/etc. to it. I at least have a road map for how to get everything back this way. Testing this can be difficult, but it really depends on what you care about really.

• Testing my kopia backups of important data? that I manually test every once n’ while.
• Testing if my ZFS setup script is 100% identical to my setup? that’s not that important, as long as I have a general idea I can figure out the gaps and improve the script for the next time around. Obviously, you can spend a lot more time ensuring scripts and what not stays consistent, but it depends on what you care about!

For a lot of my service config, git has always worked well for me and I can go back to older configs if needed. You can get super specific here and save versions in git, then have something update the versions (e.g. WUD)

I used to have a fille with every cli command and notes on how each thing was set up. When I had to reinstall it from scratch it took all day going through lots of manual steps and remembering how it should all go.

Recently I converted the whole thing to Ansible. Now I could rebuild my entire system on a brand new OS installation with one command that completes in minutes. It’s all modular and I can add new services easily whether they are docker containers or scripts or whatever. If I ever break anything, it will reset everything to its intended state and leave it alone otherwise. And it’s free and pretty easy to learn and start using.

Plus I use git along with it for version control, so I can always revert to any previous configuration instantly.

NixOS for configuration and restic for data

Terraform and ansible. Script service configuration and use source control. Containerize services where possible to make them system agnostic.

NixOS

MicroOS is a decent choice, because it can cold boot off a configuration that uses ignition and combustion files. microos.opensuse.org

And they have this file configurator so you don’t have to manually type all the syntax for your configs.

opensuse.github.io/fuel-ignition/edit

systemd unit files, because its all podman containers.

Git controlled docker-compose files and backed up docker data volumes.pretty easy to go back to a point in time.

Proxmox on the metal, then every service as a docker container inside an LXC or VM. Proxmox does nice snapshots (to my NAS) making it a breeze to move them from machine to machine or blow away the Proxmox install and reimport them. All the docker compose files are in git, and the things I apply to every LXC/VM (my monitoring endpoint, apt cache setup etc) are all applied with ansible playbooks also in git. All the LXC’s are cloned from a golden image that has my keys, tailscale setup etc.

Incus and ansible

Packer builds the terraformable/openTofuable templates to launch into the hypervisor where chef (eventually mgmtConfig) will manage them from there until they die.

All that is launched by git. Fire and forget. Updates are cronned.

There are no containers. Don’t got time to fuck about. If Systemd wasn’t an absolute embarrassment I’d not worry about updates even as much as I do, which isn’t much aside from the aforementioned cancer.

Carefully

Ansible!