[Dirvish] hardlink command

Loren M. Lang lorenl at alzatex.com
Thu Sep 12 03:22:35 UTC 2013


On 9/11/2013 8:07 PM, Loren M. Lang wrote:
> There is also three levels of duplication I can see. One is
> duplication of files on a single filesystem. If there are duplicated
> files on a server, they should be hard-linked on the original
> filesystem which will then transfer to dirvish/rsync automatically,
> but that can only be done if it's acceptable to have the same
> metadata. It doesn't work if they have different ownership, for
> example, due to some kind of per-user jail that is being done.
>
> The second is duplication between images. Dirvish/rsync should handle
> this automatically and only create duplicate copies of data when there
> is a metadata change.
>
> The third duplication is between vaults due to identical software
> installed on multiple servers. This will result in duplication when
> files are changed or added, but can be squashed post-rsync with a
> command like hardlink(1). But again, this will squash metadata to one
> version. While permissions will probably be the same, times may not
> be. Is it OK to squash this information?

Oh, there is one case that I forgot to mention. If a file is moved in
the source filesystem, but unchanged, Rsync won't track it and, instead,
will copy a fresh version of it. Rsync only detects unchanged files by
their original path. I've considered adding a method to track files by
their inode numbers to fix this case, otherwise, moving the root of a
large tree of files will cause the entire tree to be re-copied. Rsync
does detect and preserve hardlinks on the original filesystem so one
workaround is to hardlink the large tree into it's new location, then
trigger (or wait) for the next Dirvish cycle to complete. Once the files
are backed up at their old and new location, you can safely delete the
old file tree. You can use cp with the -l option to hard-link a whole
tree into a new location.

>
>> Cheers
>>
>> V.
>>
>>
>>
>> _______________________________________________
>> Dirvish mailing list
>> Dirvish at dirvish.org
>> http://www.dirvish.org/mailman/listinfo/dirvish
>
>
> -- 
> Loren M. Lang
> lorenl at alzatex.com
> http://www.alzatex.com/
>
>
> Public Key: ftp://ftp.tallye.com/pub/lorenl_pubkey.asc
> Fingerprint: 10A0 7AE2 DAF5 4780 888A  3FA4 DCEE BB39 7654 DE5B


-- 
Loren M. Lang
lorenl at alzatex.com
http://www.alzatex.com/


Public Key: ftp://ftp.tallye.com/pub/lorenl_pubkey.asc
Fingerprint: 10A0 7AE2 DAF5 4780 888A  3FA4 DCEE BB39 7654 DE5B

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.dirvish.org/pipermail/dirvish/attachments/20130911/e5cd2843/attachment.html 


More information about the Dirvish mailing list