[Dirvish] need advice to mirror a (big) dirvish backup partition
to another server
paul at debian.org
Mon Sep 12 05:42:10 PDT 2005
On Mon 12 Sep 2005, Nicolas Schmitz wrote:
> I'm a *happy* user of dirvish. I use it on a big nas on which I have a
> lot of disk : 2 partition of 4Tera and one of 2Tera (8U server, 31 400g
> disk on 3 3ware). It works great. (I only use the nas to backup other
> servers with dirvish)
Heh, we have a similar system, 24 400G disks on 2 3ware controllers, 2 x
12 configured in RAID5, and the result is striped under linux md for a
total of 8.1TB.
> The problem is that we are a little bit paranoid here, and I'd like to
> mirror those backups on another nas, which is not on the same location.
We have it the other way round: 3 systems with each 1TB of NAS, which is
copied to the "big" system.
> We have a dedicated 1giga link between the 2 servers.
> The problem is that when I try to rsync (rsync -a -H) a directory
> containing a set of backup it fails, except if it's a "small" set of
> backup (not containing a lot of files) :
It's either running out of memory, or timing out.
The trivial solution would be to mirror not a set op backups, but only
the last-added tree (with a --link-dest to the prior one).
I had a similar problem. First the idea was when an image expires, not
to simply delete it but to first copy it to the "big storage". However
just after I had finished hacking dirvish-expire and was ready to test
this, management decided that the latest backup should be put on it
I use "image-temp: newest" for dirvish, which means that the newest
image is always in a directory called "newest". This makes it rather
simple to find what the newest image is...
I then have a script that for every vault, first does ssh to the "big
backup" and runs a script to rename "newest" to whatever is in the
image's summary file after "Image:" (the name it should have). The
script then returns that name to the caller. Now that name can be used
for the --link-dest argument; rsync is now called to copy over the
"newest" image to the "big backup".
For now it works OK, and is slowly filling up the "big backup".
I still have to figure out how to clean up old images there when that
begins to fill up...
So, my situation is a bit different than yours in that I keep two
different sets of backups: short-term and longer-term, wheras you want
to keep two identical sets. However, perhaps you could use my scripts as
a starting point. Let me know if you're interested.
> The config of both server is :
> PIV 3Gz EM64T
Hmm, I'm using Opteron.
More information about the Dirvish