I have a load-bearing raspberry pi on my network - it runs a DNS server, zigbee2mqtt, unifi controller, and a restic rest server. This raspberry pi, as is tradition, boots from a microSD card. As we all know, microSD cards suck a little bit and die pretty often; I’ve personally had this happen not all that long ago.
I’d like to keep a reasonably up-to-date hot spare ready, so when it does give up the ghost I can just swap them out and move on with my life. I can think of a few ways to accomplish this, but I’m not really sure what’s the best:
- The simplest is probably cron + dd, but I’m worried about filesystem corruption from imaging a running system and could this also wear out the spare card?
- recreate partition structure, create an fstab with new UUIDs, rsync everything else. Backups are incremental and we won’t get filesystem corruption, but we still aren’t taking a point-in-time backup which means data files could be inconsistent with each other. (honestly unlikely with the services I’m running.)
- Migrate to BTRFS or ZFS, send/receive snapshots. This would be annoying to set up because I’d need to switch the rpi’s filesystem, but once done I think this might be the best option? We get incremental updates, point-in-time backups, and even rollback on the original card if I want it.
I’m thinking out loud a little bit here, but do y’all have any thoughts? I think I’m leaning towards ZFS or BTRFS.
Another option is to use Image File Utilities on the Pi to create an image backup. You can use cron + a bash script to create incremental backups using the tool (e.g., take a ‘fresh’ backup each month, with daily incremental backups in between). I mount a network ‘backup’ drive (a local NAS, but you could use anything) to save the image to so I can actually access it. Then, just use balena etcher to flash the backup iso in the event of a failure.
Oh cool, this is exactly what I was looking for! Thanks :)
Happy to help! There’s plenty of other options too (e.g., SSD) as folks mentioned, but this works well out of the box with no additional hardware. SD has been absolutely fine for my use (Pi-Hole), while still requiring maximum uptime so the family doesn’t riot if the internet is out.
I also use those, but (shame on me) I’ve never tested the backups. Have you ever restored such an image backup?
Ha yes, once! I just used Balena Etcher and it worked exactly as expected.