[Dirvish] Moving a Dirvish Vault

Eric V. Smith eric at trueblade.com
Fri Jun 2 10:08:53 UTC 2006


Paul Slootman wrote:

< good stuff about find and rm deleted >

>>> Well, Jason Boxman is lazy, so he just uses `rsync -avH src/ target/`  :)
> 
> That doesn't work if your src is very large with many hardlinked files;
> rsync runs out of virtual memory, I can say from experience.

Very true, so I use tar.  tar will preserve hard links, and since it 
just streams the data as it gets to it there's no pre-computation of 
anything to consume time or memory.  With appropriate uses of pipes and 
ssh, you can grab the data from one machine and extract it to another. 
Sorry I don't have a command line handy for you, but I've used it for 
just this purpose for a 200+ GB vault, where each of the ~100 images 
would have individually consumed 100+ GB.  Other than taking forever due 
to the network involved, it had no issues.  One thing that's a potential 
concern is that it can't be interrupted and restarted (like rsync could, 
if it worked), you'd have to start again from the top.

I don't have any sparse files, but if you do, that's something you 
should think about with any solution.  I think tar might have an option 
for handling these.

Although I've never tried them for this purpose, I think cpio and pax 
have similar functionality.

Eric.



More information about the Dirvish mailing list