[Dirvish] Copying dirvish vaults (heavily hard linked)
Shawn (Red Mop)
redmopml at comcast.net
Thu Mar 6 20:49:21 UTC 2008
On Thursday 06 March 2008 11:28:31 am Keith Lofstrom wrote:
> On Thu, Mar 06, 2008 at 09:33:52AM -0700, Shawn (Red Mop) wrote:
> > Tar is actually very fast. I used it to split a bank about a month ago.
> > It was a little unnerving, so I ran rsync -aH on top of (after) tar, and
> > rsync did almost nothing. The du -hcs * in each vault already matched.
> "Almost nothing", unfortunately, can be a big difference. When I was
> testing, tar did not produce bootable images, because some of the special
> files did not copy. Perhaps it has been improved. rsync does produce
> bootable images (if it did not, then dirvish wouldn't be very useful).
> My point is that these file copy tools can take a surprisingly long
> time. My 500GB backup drive is now about 70% full, with about 100
> images on it (I do not expire them), meaning that to some tools, on
> the file level, the drive appears to contain around 60 terabytes.
> A drive copy at the file level will attempt to traverse a large
> fraction of that 60TB; if the interface is running at 20MB/sec
> that will take a large fraction of 800 hours. Yikes!
> > LVM + resize2fs is one of my very best friends. I grow and shring
> > partions all the time. It's a real kick to move a partition from one
> > disk to another while the partition is in use without rebooting, or even
> > taking the partition offline.
> Yes, fine tools. I wish resize2fs had an extra option for increasing
> or decreasing inode *density*. It does add or subtract inodes as you
> grow or shrink partitions, maintaining the ratio of inodes to data,
> but sometimes the problem is a lack or surplus of inodes. "Plan
> ahead" is not always possible or easy with heavily hard-linked file
> systems with unpredictable future usage.
> Is there a tool that permits the manipulation of inode density on
> a used file system?
I was worried about booting too. Thus, I did run rsync after. It did run
faster running tar then rsync over just rsync. Perhaps I should have added
More information about the Dirvish