[Dirvish] need advice to mirror a (big) dirvish backup partition to another server

Paul Slootman paul at debian.org
Mon Sep 12 05:42:10 PDT 2005


On Mon 12 Sep 2005, Nicolas Schmitz wrote:

> I'm a *happy* user of  dirvish. I use it on a big nas on which I have a 
> lot of disk : 2 partition of 4Tera and one of 2Tera (8U server, 31 400g 
> disk on 3 3ware). It works great. (I only use the nas to backup other 
> servers with dirvish)

Heh, we have a similar system, 24 400G disks on 2 3ware controllers, 2 x
12 configured in RAID5, and the result is striped under linux md for a
total of 8.1TB.

> The problem is that we are a little bit paranoid here, and I'd like to 
> mirror those backups on another nas, which is not on the same location.

We have it the other way round: 3 systems with each 1TB of NAS, which is
copied to the "big" system.

> We have a dedicated 1giga link between the 2 servers.
> 
> The problem is that when I try to rsync (rsync -a -H) a directory 
> containing a set of backup it fails, except if it's a "small" set of 
> backup (not containing a lot of files) :

It's either running out of memory, or timing out.

The trivial solution would be to mirror not a set op backups, but only
the last-added tree (with a --link-dest to the prior one).

I had a similar problem. First the idea was when an image expires, not
to simply delete it but to first copy it to the "big storage". However
just after I had finished hacking dirvish-expire and was ready to test
this, management decided that the latest backup should be put on it
instead...

I use "image-temp: newest" for dirvish, which means that the newest
image is always in a directory called "newest". This makes it rather
simple to find what the newest image is...

I then have a script that for every vault, first does ssh to the "big
backup" and runs a script to rename "newest" to whatever is in the
image's summary file after "Image:" (the name it should have). The
script then returns that name to the caller. Now that name can be used
for the --link-dest argument; rsync is now called to copy over the
"newest" image to the "big backup".

For now it works OK, and is slowly filling up the "big backup".
I still have to figure out how to clean up old images there when that
begins to fill up...


So, my situation is a bit different than yours in that I keep two
different sets of backups: short-term and longer-term, wheras you want
to keep two identical sets. However, perhaps you could use my scripts as
a starting point. Let me know if you're interested.

> The config of both server is :
> PIV 3Gz EM64T

Hmm, I'm using Opteron.


Paul Slootman


More information about the Dirvish mailing list