I am using rsync command for incremental backup my requirement is incremental backup of modified files in different directory in same location name with current date but when i use rsync it store the incremental backup in full backup not in name date directory. i am using the following command for incremental backup.
rsync -av --backup --backup-dir=/mnt/`date +%Y.%m.%d.%T` /etc/ /mnt/
Depends on what you mean by incremental. One trick is to use hardlinks on an EXT4 drive. This means that if files don't change, they will appear in every snapshot, but will only exist (and be copied to the drive once). Here is script I've used before:
#!/bin/sh
NEW_FOLDER="$(date +%F\ %H-%M-%S)"
rsync --archive --human-readable --progress --link-dest /media/current "/etc/" "/media/incomplete/"
mv /media/incomplete /media/'$NEW_FOLDER'
rm -f /media/current
ln -s /media/'$NEW_FOLDER' /media/current
By using an incomplete folder, if the backup is stopped halfway through (very useful for large backups), it will pick up again when you next try. You can then browse each folder as a complete incremental backup. This is a similar approach to Apple Time Machine.
If you don't want to use hardlinks to save space, and don't mind file duplication, remove the --link-dest /media/current setting.
Rsync will perform incremental backups by default, but probably not with the result that you expect. The command
rsync -av /etc/ /mnt/
transfers only changed files. For that, it compares the content of /etc and /tmp. If you add a computed date, eg /tmp/$(date), the target folder is empty and therefore, all files will be transfered again.
The options --backup and --backup-dir are only useful, if you need a backup of the files, which would be deleted, if you start the rsync command with the option --delete.
Summary
To create true incremental backups, you will need another tool, eg. rsnapshot.
I've been looking for an answer here before and went with @Hadog's answer at first. It works perfectly, until you run rsync over a network connection and it has a hickup while connecting. Rsync will simply exit and not try again. The problem is that although it fails, the symlink is changed to a new folder that is non-existant (broken link), leading rsync to ignore --link-dest. That means, a full copy is made again instead of using hard links.
Here is my variation of @Hadog's script with improved error handling:
#!/bin/sh
determine how the new folder should be called
NEW_FOLDER="$(date +%F\ %H-%M-%S)"
rsync --archive --human-readable --progress --link-dest $PWD/current source "$PWD/destination"
get exit code of rsync. -> don't put anything between this line and the rsync command!
rsync_exit_code=$?
if the rsync command exited with a code that we consider "successfull" ...
if [ "$rsync_exit_code" -eq "0" ] || [ "$rsync_exit_code" -eq "23" ] || [ "$rsync_exit_code" -eq "24" ]
then
# move files to their actual destination
mv $PWD/incomplete $PWD/"$NEW_FOLDER"
# remove the old symlink
rm -f current
# set the symlink to the newest backup that we have just created
ln -s $PWD/"$NEW_FOLDER" $PWD/current
else
echo "No moving of incomplete folder! Rsync exited with code $rsync_exit_code"
fi
The significant difference is the condition. We should only reset the "current" symlink if we actually have a new working backup. For my example from above where building the network connection fails, that will mean that the backup script exits without retrying. A retry would be to re-run the script. Instead of using if conditions, you may want to use a do-while loop to retry the rsync command until it succeeds.
In my code, I consider rsync's exit codes 0, 23 and 24 successful. Here is what they mean (see rsync man page for all codes):
Update 4 months after posting: script as provided above hasn't caused any issues so far so I'm confident to say that at least for me it's stable.
