[Dirvish] dirvish-expire very slow

Eric Searcy emsearcy at gmail.com
Fri Jan 16 00:32:40 UTC 2009


Arjan Schrijver wrote:
> Hi list,
> Is anyone experiencing this same problem?
> We're backupping about 35 servers every night, which takes about 2 hours 
> to complete.
> The backup sizes vary from a few hundred MB's to a few GB's per server.
> The speed of the backup itself is quite good, but after the backup we 
> start dirvish-expire, which takes about 6 hours, and sometimes even up 
> to 10 hours or more.
> Is this normal? It seems to me that analyzing directory names and then 
> deleting them shouldn't take this amount of time.
> Can anyone explain this to me?

No, it isn't normal.  Another idea is that if you have directories that
have *lots* of files (thousands) not nested in other directories and are
using ext3 without -O dir_index, running rm's can take a long time
because it iterates at O(n) time.  Of course, this would also cause the
backup to go slowly as rsync would need to iterate the same folders.

One other aside: normally I would recommend running dirvish-expire
first.  Running it before or after will still delete the exact same old
images.  Only the most recent image is used for for the differential,
and it will never be removed.  But cleaning up before you run the backup
instead of after means you have more space to work with.

Eric

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: OpenPGP digital signature
Url : http://www.dirvish.org/pipermail/dirvish/attachments/20090115/94e4b3e8/attachment.bin 


More information about the Dirvish mailing list