1

I'm trying to implement a good back up strategy for my ubuntu server operating system. I have multiple harddrives on my server. So my plan is to use rsync to copy everything but my user data to another drive daily, and keep a weeks worth of copies. So if anything goes wrong I can just run rsync to copy the files back over to my "/" drive where my operating system is installed. I have two partitions on the drive where my operating system is installed nvme1p1 where the boot efi is saved and nvme1p2 where all the usual operating system folders are (bin, dev, etc, lib...)

So I'm running the bash script below daily:

#!/bin/bash

DATE=$(date '+%F')


sudo rsync --exclude={/mnt,/home} -avz / /mnt/data/storage1_1tb/backup_server_os/$DATE

I had a "busybox" and readonly filesystem issue with my server recently. so I ran

sudo rsync -av /mnt/data/storage1_1tb/backup_server_os/2023-11-09 /

I got a message that some files couldn't be copied, and after rebooting it still didn't fix the issues. ultimately I had to run fsck -y / from root and also ran fsck -y /dev/nvme1p2 and fsck -y /dev/nvme1p1 using live media. when run with live media it mentioned some dirty files on /dev/nvme1p1. I haven't been backing up the boot efi on /dev/nvme1p1, do I need to do that as well so I can copy it back over in case of issue.

My ultimate goal is that if something went wrong with my operating system (ex. hard shutdown causes read only file system error on boot) I could just copy the files back over from the previous days back up, reboot and hopefully everything runs fine like the day before. Does the strategy I've outlined make sense and if so what am I doing wrong with rsync, (since I still had to run fsck -y after running rsync to copy files back over)?

user3476463
  • 147
  • There are a wide variety of right answers. If it makes sense to YOU, then it makes sense, and it's the right answer for you. My solution for me is very different, and you don't need to care about my way. No solution is perfect. There is always room for improvement. – user535733 Nov 22 '23 at 03:13
  • 1
    Pardon me, but you are mixing data e.g. files which rsync can copy with data structure on a lower level i.e. filesystem which rsync can’t copy … That’s another job for disk imaging or at least partition/filesystem copying/backup software … See for example https://askubuntu.com/a/1422748 – Raffa Nov 22 '23 at 10:09

1 Answers1

1

I had a "busybox" and readonly filesystem issue with my server recently. so I ran

sudo rsync -av /mnt/data/storage1_1tb/backup_server_os/2023-11-09 /

When your filesystem on which / resides is mounted as read-only, then there is nothing that rsync in that context can do i.e. copy so, whatever expectations you based on running that command in that stated are most likely not going to see the light and nothing on the target filesystem should have changed at all ... Unless you had force mounted that system in a read and write mode while it's in inconstant state which might explain:

I got a message that some files couldn't be copied, and after rebooting it still didn't fix the issues. ultimately I had to run fsck -y / from root and also ran fsck -y /dev/nvme1p2 and fsck -y /dev/nvme1p1 using live media. when run with live media it mentioned some dirty files on /dev/nvme1p1.

... not to mention fsck -y / which suggests that you ran a check on an already mounted filesystem (bad idea), but rsync seems to have done its best in copying back what it could in such an inconsistent filesystem state and you can't blame it for the errors you encountered as those are the result of the circumstance you put in ... After all rsync works best for copying data e.g. files served/received by the mounted filesystem ... Filesystem integrity, however, is maintained utilizing special inodes/files on a lower level than rsync can handle (or even see) in such operations and can even be affected with even a more lower level elements like super blocks or partition tables.

I haven't been backing up the boot efi on /dev/nvme1p1, do I need to do that as well so I can copy it back over in case of issue.

... that should be a fairly small special partition which rarely changes its state and therefore fully cloning/backing up that partition with a partition cloning utility like partclone is the way to go IMO ... And that brings us to your last question:

My ultimate goal is that if something went wrong with my operating system (ex. hard shutdown causes read only file system error on boot) I could just copy the files back over from the previous days back up, reboot and hopefully everything runs fine like the day before. Does the strategy I've outlined make sense and if so what am I doing wrong with rsync, (since I still had to run fsck -y after running rsync to copy files back over)?

fsck is a filesystem/data structure layer tool that is figuratively one layer deeper below what rsync can see or handle which is data itself. Hence, rsync is not the tool to backup and restore/bring the filesystem state to one functional/clean point in the past ... For that you'll need a filesystem, partition or disk backup/imaging tool ... Please, see further in depth related discussions here, here and here.

Raffa
  • 32,237
  • 1
    thank you for this response, it's very helpful. it seems like rsync really isn't the right tool for backing up and restoring the operating system. it seems like repairing operating system issues with live media and commands like fsck -y /partition might be the way to go for operating system problems. rsync is more for backing up the user data. I looked over the links you sent they seem pretty involved, thank you for all the detail. – user3476463 Nov 22 '23 at 18:06