11

I have a directory called my_projects_linux inside the Ubuntu file system, which contains all my work from many years. The directory contains files, subdirectories and so on.

For backup purposes, I occasionally copy this directory and all its contents to external hard drive.

Hence, the contents of my external drive look like:

/mounted_drive/my_projects_linux
/mounted_drive/my_projects_windows  # the same idea to backup Windows work

Hence, what I am looking is a command which:

  1. Would work as 

    cp /home/my_projects_linux /mounted_drive/my_projects_linux

    It should replace old files, subdirectories, files inside subdirectories, etc. in the external disk by the new content from my PC.

  2. Be fast. It should only copy modified files or those that have been newly created. Given that the size of my_projects_linux is >50 GB, copying everything takes more than an hour, which is too slow. In reality often only a few MB have changed since last backup, so theoretically a copy can be made much faster.

How to do that?

I googled that cp with the -u flag could possibly match my needs. Would that work (e.g. would it correctly handle subdirectories of subdirectories)?

Also, is storing file system on an external disk an appropriate way of doing a backup, or is there a fancier way? For example, using a cloud? Note that the fancier way should be simple, as otherwise it will not outweigh the ease of executing one shell command.

mercury0114
  • 602
  • 1
    Possible duplicate of Backup your home directory with rsync and skip useless folders – David Foerster Mar 31 '16 at 14:08
  • 1
    @DavidFoerster I disagree. In your link rsync is already mentioned and its particular feature of skipping a specific list is being discussed. Here I'm just asking how to backup files. I have not heard about rsync at the moment this question was posted. So although related, I don't think this question is a duplicate. – mercury0114 Mar 31 '16 at 14:13
  • For the cloud thing, you can check rclone: http://rclone.org/ --- it's like rsync but works with (some) cloud. Works ok with google drive. Once setup, it's just one command. – Rmano Mar 31 '16 at 14:30
  • ...and you can also check unison: http://askubuntu.com/questions/633673/syncronizing-with-unison-on-ubuntu --- I am using it since ages and never had a glitch. – Rmano Mar 31 '16 at 14:33
  • Whether the solution(s) mentioned in the linked question have already been discussed here is irrelevant. Questions dealing with the same underlying issue or having a very similar set of solutions should be flagged as duplicates. – David Foerster Mar 31 '16 at 15:24
  • @DavidFoerster But I still don't see how this question is a duplicate. For example, suppose question 1 asks how to find the shortest path in a grid, question 2 asks how to implement a queue using a vector. Now an answer for question 1 mentions queue as a way to solve a problem, where an answer for question 2 shows how queue is implemented. Does that mean two questions are duplicates? Probably not, it's just a coincidence that queue appeared in both answers. So rsync here is an analogy to a queue in my opinion... – mercury0114 Mar 31 '16 at 15:35

3 Answers3

14

You're sort of describing what rsync was designed for. From man rsync:

   Rsync  finds  files  that  need to be transferred using a "quick
   check" algorithm (by default) that looks  for  files  that  have
   changed  in  size  or in last-modified time.  Any changes in the
   other preserved attributes (as requested by options) are made on
   the  destination  file  directly  when the quick check indicates
   that the file’s data does not need to be updated.

There's also a lot of guff about networking, but it's happy doing local too.
Essentially all you need is:

rsync -rtv /home/my_projects_linux /mounted_drive/my_projects_linux

There are a ton of options available but -rtv will sync recursively, keeping the timecodes the same, while being verbose about what it's doing.

Oli
  • 293,335
  • 4
    -a is archive mode; it equals -rlptgoD, which includes -rt from the answer. OP's question did not require console output so you do not need to add -v if you do not want to see stuff scrolling by. – Amedee Van Gasse Mar 31 '16 at 14:41
  • 1
    Suppose the mounted_drive contains an old file that is no longer present in the /home/my_projects_linux directory. Will the command remove that file from the mounted_drive? – mercury0114 Apr 21 '17 at 11:56
1

To answer your second question, a more fancy way to backup is rdiff-backup.

rdiff-backup backs up one directory to another, possibly over a network. The target directory ends up a copy of the source directory, but extra reverse diffs are stored in a special subdirectory of that target directory, so you can still recover files lost some time ago. The idea is to combine the best features of a mirror and an incremental backup. rdiff-backup also preserves subdirectories, hard links, dev files, permissions, uid/gid ownership, modification times, extended attributes, acls, and resource forks.

And if you really want to go cloud-based: I use CrashPlan with a paid subscription. Then you never have to think about manual backups any more.

Amedee Van Gasse
  • 341
1

I know this question already has an accepted answer. However, I wanted to add a cross platform GUI solution as I see you also use windows. I use for the same purposes Freefilesync. It is also FLOSS.

The configuration is absolutely intuitive and you can save different synchronization jobs.

There is also a ppa, though it does not include the latest version for 14.04 at the moment. To install via ppa:

sudo apt-add-repository ppa:freefilesync/ffs
sudo apt-get update
Bruni
  • 10,542