5

I'm looking for a backup solution for a specific use case. If none exists, then I'm tempted to make one.

I have a large and growing collection of video files. On my workstation I have a fairly large disk in RAID5, consisting of a few 2TB disks, which is easily expandable. These files are never changed. If I need to edit one, then I create a new file. I use hotswapable 2.5" harddisks for backup, which so far I've done manually. It works, but I'm afraid of doing anything wrong. I might forget to copy a file, mislabel a disk, etc.

I would like a solution that watches the video files on my workstation, automatically copies new files to the current backup disk and presents me with an overview of which backup disk contains which files, descriptions, dates and where those disks are physically stored. It would be nice if I could just insert an unformatted disk and it would be formatted, labeled and tracked automatically. When the free space of a backup disk reaches a certain level, I should be notified so that I can store it safely and replace it with a new one. Each backup disk should contain a copy of the current database and thumbnails for the videofiles contained on this disk.

For restore, I should be able to simply provide a root directory where the hierarchy should be recreated and it would then tell me to "insert disk x" -- like installing Windows 95 from floppies.

Does anyone know of a free backup solution that fits this profile, or should I go ahead and make one?

Amey Jah
  • 2,695
Jo-Erlend Schinstad
  • 29,531
  • 2
    You should look at: http://askubuntu.com/questions/2596/comparison-of-backup-tools – andrewsomething Jul 29 '11 at 17:48
  • IMO your need is too specific. You don't need version control. So don't go for the solution which has version control overhead. I would suggest you to use rsync and create a custom script to make backup utility.rsync maintains log via --log-file option. You might be able to parse this log and generate the report of your choice – Amey Jah Jul 29 '11 at 18:11
  • My need is too specific? I haven't looked for version control. I specifically said I don't need that. What I do need, however, is a solution that enables me to copy a very large file structure onto many small disks, 500-2000GB and then use those disks to recreate the structure and this should be automatic since it will take many days to copy. – Jo-Erlend Schinstad Jul 31 '11 at 19:29

3 Answers3

2

I don't think there's an existing solution that offers exactly what you describe. You would have to write a couple of scripts, starting with something like fsniper Install fsniper that watches for new files and then triggering the addition of this file to a text file tracking all files, copying the file to the harddisk, complaining if the disk is full etc. It is of course feasible, but involves a lot of work and has a lot of pitfalls that existing backup solutions take care of coping with (e.g. handling a crash during the copy of a file).

Another solution (that does not store plain files on the disk and does not allow any nice thumbnail preview) is using a powerful backup solution like bacula Install bacula, that is built for enterprise grade backups and can cope with backing up to multiple tape drives. You can then adapt such a solution to use harddrives instead of tape drives, as explained here on the bacula wiki

A software that might be useful for your current workflow (manual copying of new files) is “Virtual Volumes View” (short: VVV):

VVV is an application that catalogs the content of removable volumes like CD and DVD disks for off-line searching. Folders and files can also be arranged in a single, virtual file system. Each folder of this virtual file system can contain files from many disks so you can arrange your data in a simple and logical way.

Unfortunately it's not available in the repositories but has to be compiled from source.

Glorfindel
  • 971
  • 3
  • 13
  • 20
Marcel Stimberg
  • 41,762
1

I'm using a simple script for rsync. It syncs my laptop with an external hard drive.

It keeps (a) some folders in external drive only [LTDS list folder for that], (b) others are synced with local drive [LTDD list folders for that], (c) also dot files are backed-up [last 4 lines of the script do that].

My external hard dirve is called box. And in a script it's hardcoded, as a distination path is: DEST="/media/box".

#!/bin/bash

# This script provides two types of backups
# * LTDS (stands for 'list delete orig at source'): the one which delete files on the DEST, and
# * LTDD (stands for 'list delete orig at destination'): the one which doesn't.
# Note that starting DEST folders has to be before the sync.
# Type in the lists the folders subject to the sync (all paths reletive to the $HOME):

# for movies and soft (to keep them at the remote HDD only):
LTDS="/Box/mixed/Movies/"
# for songs, books, documents (to keep them synced both: at HOME and at remote HDD):
LTDD="/Box/sonic/ /Box/visual/ /Box/wordy/ /Documents/ /Downloads/ /Passed/ /Pictures/ /Study/ /Work/"

DEST="/media/box"

for i in $LTDS; do
    rsync -aq --progress $HOME$i $DEST$i
    rm -rf $HOME$i*
done

for i in $LTDD; do
    rsync -aq --delete-after --progress $HOME$i $DEST$i
done

# the following back-ups all the dot files in your home dir.
dotfiles=`ls -A | grep '\.'`
for i in $dotfiles; do
    rsync -aq --progress $HOME/$i $DEST/Misc/dotfiles
done

I'm using that script for quite now. It works fine.

Adobe
  • 3,891
0

You might want to look into rsync. I know it's a bit scary b/c its a command line tool, but there is a GUI for it, grsync which you can find in synaptic or do 

sudo apt-get install grsync

There are a ton of options for the backup and the -i option gives you an itemized changes list. It should fit the ticket. See the wikipedia page

Zanna
  • 70,465
Ctuchik
  • 271