3

As per this answer to the question "Can I continue working when backup is in progress?," the default backup software in Ubuntu, Deja Dup, is prone to consistency issues when backing up. That is, if the user changes files on a writeable filesystem while the backup is being done, the backup might not be in a sensible state, and restoring it in full will restore a broken system. Even restoring individual files might restore broken files.

The advice to not do any work while the backup is going is a workaround, but not really practical. A user "not working" doesn't mean files aren't being modified behind the scenes.

The answer I linked to says this about snapshots:

This can be achieved using LVM or a newer generation filesystem such as BtrFS. This will snapshot the whole volume as if you were taking a picture of it. Changes/writes are still possible, but the backup process is being run from the read-only snapshot taken earlier.

For example, on Windows systems most backup software runs on a "shadow copy," a semantically consistent snapshot of the filesystem maintained by the OS. The user can continue working, and further modifications will not be part of the currently running backup.

However, I don't know that Deja Dup on Ubuntu uses any kind of snapshotting if it's available.

My question is: is there any automated backup software for Ubuntu that:

  • is easy to use, via a GUI, suitable for users unwilling to use the command line or write their own scripts
  • ensures consistency of backups using snapshots, or enforcing read-only on the source of the backup (heavy-handed but perfectly valid), or any other approach to avoid inconsistency?
detly
  • 3,382
  • Your question will likely be closed as "too broad" or "opinion based". For myself cron backs up daily and emails .tar (tape archive) file to my gmail account. https://askubuntu.com/questions/917562/backup-linux-configuration-scripts-and-documents-to-gmail/922493#922493 – WinEunuuchs2Unix Mar 14 '18 at 23:19
  • I use rsync if I'm backing up a folder. rsync seems to be the standard in the linux world. I don't know anyone that uses Deja Dup, and have read in some places that it doesn't back up properly. The GUI version of rsync is grsync, which I have never used. If I'm doing a backup of an entire filesystem, then I unmount it and use dd. I don't know of any GUI for dd. – SunnyDaze Mar 14 '18 at 23:22
  • @WinEunuuchs2Unix Shouldn't be, see chat. – detly Mar 14 '18 at 23:52
  • @SunnyDaze rsync has the same problems as Deja Dup here unless a filesystem is mounted read-only. – detly Mar 14 '18 at 23:55
  • @detly I didn't vote to close it. Only thinking others might vote that way. – WinEunuuchs2Unix Mar 14 '18 at 23:56
  • @WinEunuuchs2Unix Sure, I appreciate the heads up. If it's OT here I'll migrate it to the software recs SO. – detly Mar 15 '18 at 00:00
  • 1
    @detly Yes I just read Terdon's recommendation to do so on the chat link you posted. – WinEunuuchs2Unix Mar 15 '18 at 00:00
  • Did you look here and see if you can find one of the GUI backup applications that would suit your needs? – Terrance Mar 15 '18 at 01:28

1 Answers1

2

Unless you are a 24/7 shop generally speaking you have the server sign everyone off at say 2am and perform backups for two hours or whatever it takes. Usually this is done in conjunction with "End of Day Processing" which rolls up detail records to Master File records, closes Month Ends, Year Ends, etc.

The basic problem is some files will "change shape" if someone is adding or deleting records. Take for example ISAM (Indexed Sequential Access Method) files where there is a raw data file and then separate Index files for each key (ie Customer Number, Phone Number, etc.). If you were to say backup the Key file for Customer Numbers first, then a user added a new customer and you backed up the raw data file you would have a data integrity error.

SQL is the popular choice these days for databases in which case an SQL dump is performed without keys backed up I think. Learning SQL (pronounced Sea-Quill) has been on my to-do list for 30+ years.

Whatever your environment you have to study backup requirements carefully and test them periodically by restoring them to a test database.

Backing up programs is generally over-kill because they can be reinstalled. The exception being your own scripts you develop.

To reiterate the safest backups are those when all users are logged out of the system. When that can't be guaranteed then professional help should be sought.

WinEunuuchs2Unix
  • 102,282
  • I'm not running a server, nor am I talking about machines that are on 24/7. Just desktop workstations, not administered by a server. I just want something that can be used by someone who isn't versed in server administration. – detly Mar 14 '18 at 23:51
  • 1
    @detly this answer seems fully applicable to "desktop workstations" and to "someone who isn't versed in server administration". It's good advice. – user535733 Mar 15 '18 at 00:07
  • @user535733 It's definitely good advice, but it's not really an answer to my question. On Windows and OS X, casual desktop users can use reliable, built-in GUI software to take backups of a completely consistent system while they're getting on with their work. On Ubuntu, according to this answer, users should hire professional help to set up personal backups of a desktop system which they leave running after hours. If that's really the state of things, I suppose the answer to my question is simply: no. – detly Mar 15 '18 at 00:47
  • I don't mean to be harsh, but the context of my question is simply whether users without deep knowledge of *nix systems can expect to be able to use Ubuntu as a GUI desktop system without forgoing backups, like they can on other GUI desktop OSs. I think this answer, sound as it is, has missed that context somewhat. – detly Mar 15 '18 at 00:50
  • @detly Even in Windows if multiple users are updating the system 24/7 professional help should be sought for backups. Usually in this environment there are multiple in-house employees at 6 figure cost to do this though. Outsourcing for millions is of Mission Critical Apps is normal but that's not for backups. When files have to be synchronized for data integrity you don't want a backup image where File A at the start of backup doesn't logically jive with File Z at the end of the backup. Even on single user desktop accounting database the best backup is when no one is using the system. – WinEunuuchs2Unix Mar 15 '18 at 00:56
  • @WinEunuuchs2Unix What you say might be correct, but it still doesn't answer the question: is there an easy to use GUI that a user can configure to "backup when no one is using the system"? – detly Mar 15 '18 at 01:02
  • For example, out of the box Windows 10 allows GUI configuration of all of this: ensures backups run on semantically consistent snapshots of filesystems, including common databases, allows scheduling after hours, etc. More complex requirements can be met by paid-for 3rd party software, and indeed probably should be configured by a professional, as you say. I'm trying to figure out whether anything remotely similar exists for Ubuntu. – detly Mar 15 '18 at 01:04
  • Typically we are concerned with backing up data and not file systems. When we restore the data the file systems are dynamically created. If you are looking for Windows on Linux you won't find it. If you are looking for Linux on Windows you won't find that either. I think a GUI to select directories / files to backup is sensible, but it can still be a hidden cron job that does the heavy lifting at 2am. Not knowing the databases you want to backup makes it hard to recommend solutions. Also some applications have backup and data integrity checking software for their own data. – WinEunuuchs2Unix Mar 15 '18 at 01:10
  • At the end of the day (pun noted) you want backup software that works reliably for your data which is your most valuable treasure in the world. Whether it is GUI or CLI should be a mute point... You simply want the best rock solid solution. – WinEunuuchs2Unix Mar 15 '18 at 01:19
  • I am looking at the case where a user wants to back up their home directory, and maybe other regularly changing data. (I'm not sure why you've fixated on databases here, and at any rate, it's a solved problem.) I don't think it's unreasonable to consider that many users don't have the expertise, time, inclination or even ability to use CLI programs. The best rock-solid solution for those users necessarily includes something they can use. But if it doesn't exist, it doesn't, and they'll either have to make it or wait for it to be made. – detly Mar 15 '18 at 02:14
  • If you only want to backup /home/... directory you might want to exclude /home/me/.cache which on my system is 1 GB and too large to include in my daily cloud backups. – WinEunuuchs2Unix Mar 15 '18 at 02:24