38

You can easyly backup your home folder on an external harddrive with

rsync -a --exclude=.cache --progress /home/$USER/ /media/linuxbackup/home/$USER/

I excluded the .cache folder cause I think I will never need it when I have to re-install from this backup.

I found this list of all folders that I could exclude in a normal backup here:
What files and directories can be excluded from a backup of the home directory?

I created a list of this answer that contains some coments in this form:

#These directories may be excluded:

.gvfs                           # contains mounted file systems?
.local/share/gvfs-metadata
.Private                        # contains the actual encrypted home directory
.dbus                           # session-specific
.cache
.Trash                          # do I need to say more?
.local/share/Trash
.cddb                           # cached info about audio CDs
.aptitude                       # cached packages lists


#Flash-specific:


.adobe                          # Cache for flash, maybe others?
.macromedia   # except for Flash persistence, there is no reason to keep this


#Files:


.xsession-errors            # contains errors from the current graphical session
.recently-used              # recently used files
.recently-used.xbel
.thumbnails

Here is the full list at gist

How can I add this list to my rsync command?

rubo77
  • 32,486
  • 1
    besides .Trash, there's frequently also .Trash-1000 (under current Ubuntu-MATE 14 at least), so better settle for .Trash-*? – Frank N Feb 26 '17 at 12:43

4 Answers4

58

The exclude list may only contain filenames, foldernames and lines starting with #. A comment behind the foldername is not allowed. I created a Git repository with all known files and folders that are superfluous:

Download this ignorelist to /var/tmp/ignorelist

wget https://raw.githubusercontent.com/rubo77/rsync-homedir-excludes/master/rsync-homedir-excludes.txt -O /var/tmp/ignorelist

Then start the rsync with

rsync -aP --exclude-from=/var/tmp/ignorelist /home/$USER/ /media/$USER/linuxbackup/home/

Note:
In the ignorelist there is a commented section at the start with folders, that are probably not worth a backup either. Uncomment those, you don't need.

rubo77
  • 32,486
  • 2
    You can combine the two commands, since rsync can read from stdin: wget https://... -O - | rsync -a --progress --exclude-from=- ... – muru Nov 04 '14 at 18:08
  • 11
    good point, but I think it is recommendable first to look at the ignorelist before starting the rsync – rubo77 Nov 04 '14 at 18:17
  • surprisingly node_modules isn't on the list – jchook Nov 03 '18 at 21:11
  • don't you want to keep node_modules in some of your config directories? Imagine, you want to move your home directory to a new computer, you would need to keep those then. – rubo77 Nov 04 '18 at 15:28
5

From man rsync:

 --exclude-from=FILE     read exclude patterns from FILE
          This option is related to the --exclude option, but it specifies
          a FILE that contains exclude patterns  (one  per  line).   Blank
          lines  in  the  file  and  lines  starting  with  ’;’ or ’#’ are
          ignored.  If FILE is -, the list  will  be  read  from  standard
          input.
muru
  • 197,895
  • 55
  • 485
  • 740
  • There is such a list, I edited my question – rubo77 Nov 04 '14 at 17:40
  • @rubo77 I stand by my statement that there's no list of all folders you could exclude. That's a recommendation, and I doubt even Lekenstyn would say it's a complete list. – muru Nov 04 '14 at 17:43
  • The list doesn''t have to be complete I just would like to create my backup faster so skip obvious useless folders – rubo77 Nov 04 '14 at 17:44
  • @rubo77 I think you'll have to shift the comments to different lines. – muru Nov 04 '14 at 17:52
  • thx, I updated the gist file so the comments are on single lines – rubo77 Nov 04 '14 at 18:04
0

Use rsync-homedir-excludes on Github ->

  1. download the excluded list (rsync-homedir-local.txt)->

    wget https://raw.githubusercontent.com/rubo77/rsync-homedir-excludes/master/rsync-homedir-excludes.txt -O rsync-homedir-local.txt

  2. Edit the file to meet your needs ->

    nano rsync-homedir-local.txt

  3. Create a backup directory (some examples)

    BACKUPDIR=/media/workspace/home/$USER

    BACKUPDIR=/media/$USER/linuxbackup/home/$USER

    BACKUPDIR=/media/$USER/USBSTICK/backup/home/$USER

  4. Use the "-n" parameter so rsync will simulate its operation. You should use this before you start:

    rsync -naP --exclude-from=rsync-homedir-local.txt /home/$USER/ $BACKUPDIR/

  5. Check for permission denied errors in your homedir ->

    rsync -naP --exclude-from=rsync-homedir-local.txt /home/$USER/ $BACKUPDIR/ | grep denied

  6. if it is all fine, actually perform your backup ->

    rsync -aP --exclude-from=rsync-homedir-local.txt /home/$USER/ $BACKUPDIR/

Aindriú
  • 141
  • I get install: missing destination file operand after 'npkill' – rubo77 May 03 '22 at 08:27
  • How is this related to the Question? – rubo77 May 03 '22 at 08:27
  • If you have done a lot of programming exercises and saved them in your home folder you are going to have lots of node_module folders.... you can remove them by running the commands above. Be careful to delete only node_modules in your programming exercises – Aindriú May 03 '22 at 12:17
0

Could try this if directories and files within are all you want backing-up. Excludes all hidden directories.

rsync -aP --exclude=.* /home/$USER/ /media/$USER/folder

arth
  • 9
  • The shell could expand the * (maybe not in that place?). The paths should be quoted - the user name can not contain a space, but /media/ could be /Old Media/ or so... – Volker Siegel May 17 '15 at 17:02
  • I apologize I am not getting it. The '.*' is to not include any '.' pattern which are hidden directories at home directory '/home/$USER/' to backup destination: external hdd or pen drive '/media/$USER/folder'. By doing this, only the actual directories and files are being backup. If you're pointing something else, please elaborate. – arth May 17 '15 at 18:23
  • 6
    It is not really sensefull to exclude all directories starting with a dot, for example .thunderbird directory is one of the most important directories – rubo77 May 17 '15 at 21:34
  • @arth Try this in bash: echo foo --exclude=.* bar, shopt -s nullglob, echo foo --exclude=.* bar. – Volker Siegel May 17 '15 at 23:03
  • But I have to apologize for being terse and confusing. In part, I just had a reflex-like reaction when seeing unquoted globs and variables - it's a very frequent problem - but as you say, the variables may be actually save. – Volker Siegel May 17 '15 at 23:10
  • @Volker I was just making a suggestion. No need for condescending. Thanks. – arth May 18 '15 at 00:41
  • @arth Oh, absolutely not meant as condescending tone - it was a honest attemt to be polite, for making up for the initial terseness... (English is not my native language) – Volker Siegel May 18 '15 at 01:38
  • @arth Adding to the example above: shopt -s nullglob, echo foo --exclude=.* bar - here, the shell is interpreting the * (the command never sees it), echo foo --exclude='.*' bar - in this case, the command is seeing the *. With the default bash options unchanged, the shell tries to interpret the * in the first case, fails, and keeps it - so the command can see it. nullglob makes it insert nothing if nothing is found for a glob. – Volker Siegel May 18 '15 at 01:57
  • From this, I would expect that rsync -aP --exclude='.*' /home/$USER/ /media/$USER/folder would prevent the * problem - but I did not test it. – Volker Siegel May 18 '15 at 02:05
  • @Volker I see. I tested yours as stated and resulted the same as mine. However, I'll add that to my additional solution notes as my files grows and gets complicated. Thanks. – arth May 18 '15 at 02:45