[Dirvish] Offsite Disks

Petcher, Daniel danielp at audioprecision.com
Thu Aug 23 14:41:57 UTC 2007


My mileage varied significantly! Check the Dirvish wiki for more details on
my technique, but what I do generally is:
I run this post-execution script to build and update a symlink to today's
data called current. I'm a little paranoid, so my script also makes and
updates a symlink called recent by renaming yesterday's symlink:


#!/bin/sh
PATH=/usr/local/sbin:/usr/sbin:/sbin:/usr/local/bin:/usr/bin:/bin:/usr/X11R6
/bin:/usr/games:/opt/www/htdig/bin:/usr/lib/java/bin:/usr/lib/java/jre/bin:/
usr/share/texmf/bin
 cd $DIRVISH_DEST/../..

# test for usual failure symptom, then update if good

 if [ -x /$DIRVISH_DEST/rsync_error ]; then
#    echo "rm current"
    rm current
#    echo "cp recent current"
    cp recent current
#    echo "Keeping pointer to recent backup"
 else
#    echo "rm recent"
    rm recent
#    echo "mv current recent"
    mv current recent
#    echo "ln -s $DIRVISH_DEST current"
    ln -s $DIRVISH_DEST current
    echo "Updated pointer to current backup"
 fi

# That $DIRVISH_DEST variable points to dirvish's current tree of data.

I've got a roll-back script that I use when today's cron-based backup fails.
It deletes the current symlink and renames "recent" to "current", then it
deletes the .../vault/server-name/%Y%m%d folder and re-initiates a
dirvish-backup.

Next, I make separate server-offsite vaults on my removable disk pack and I
set their source to be server-name/current. This lets my weekly offsite
backup job get its data from the last good daily dirvish vault instead of
going out over the network and pulling fresh data from the servers. My
weekly backup window went from 20 hours to 6 hours with this trick.

I only use one offsite volume and bring it into the office weekly, but the
technique should work just as well with any N-volumes. Dirvish doesn't do
anything to the data it's captured; it just keeps a few log and summary
files in the vault. If you use my technique with multiple offsite vaults and
rotate them monthly, you should see a slower backup on the first job after a
rotation, since it's more likely to need an update on a given file.

Enjoy!

-dP

-----Original Message-----
From: dirvish-bounces at dirvish.org [mailto:dirvish-bounces at dirvish.org] On
Behalf Of Dave Howorth
Sent: Thursday, August 23, 2007 1:59 AM
To: dirvish at dirvish.org
Subject: Re: [Dirvish] Offsite Disks


Steve Ramage wrote:
> everyday after dirvish is done running
> doing an rsync of the vault with the external USB disk to sync them up 
> (I'm hoping this would preserve the hardlinking structure).

Sadly not, in my experience.

I have dirvish make a vault (it's actually a local historical mirror of a
remote server rather than a backup as such). Then I want to keep a backup
copy of this mirror - i.e. an exact copy of the vault. Obviously I don't
want to copy the entire vault every time, I just want to update the copy. So
I tried dirvish but found it replaced the hard links with copies of the
files. I tried using rsync by itself but it had the same problem.

What I've ended up doing is using rdiff-backup to make the copy. It is a bit
like dirvish but instead of using the rsync executable it uses librsync,
which is an alternate implementation of the algorithm. The actual commands I
use are:

/usr/local/bin/rdiff-backup /MIRROR/VAULT /BACKUP/COPY
/usr/local/bin/rdiff-backup --force --remove-older-than 2D /BACKUP/COPY

This works for me. YMMV.

Cheers, Dave


More information about the Dirvish mailing list