[Dirvish] [Bulk] Re: Full backup every time

Doug.Hill at sonydadc.com Doug.Hill at sonydadc.com
Sun Mar 23 04:13:26 UTC 2008


Hi Ken,

Yes, dirvish is being ran as root. I have a user that I created called 
"backupuser". In my sudoers file, I have the user "backupuser" set to be 
ran as root without a password via "sudo". The contents of my rsync-new 
file...

#!/bin/sh
sudo /usr/bin/rsync

So, backupuser will have access to files that are owned by root. Is this a 
problem? 

My rsync version: 
rsync  version 2.6.6  protocol version 29
Capabilities: 64-bit files, socketpairs, hard links, symlinks, batchfiles,
              inplace, IPv6, 64-bit system inums, 64-bit internal inums

Filesystem:
XFS

any ideas?

Thanks,
Doug




Kenneth Lerman <Kenneth.Lerman at se-ltd.com> 
Sent by: dirvish-bounces at dirvish.org
03/22/2008 03:26 PM
Please respond to
Dirvish user and developer mailing list <dirvish at dirvish.org>


To
"Dirvish user and developer mailing list" <dirvish at dirvish.org>
cc

Subject
Re: [Dirvish] [Bulk] Re:  Full backup every time






Am I correct that dirvish is being run as root?
 
Ken
----- Original Message ----- 
From: Doug.Hill at sonydadc.com 
To: Dirvish user and developer mailing list 
Cc: dirvish at dirvish.org ; dirvish-bounces at dirvish.org 
Sent: Saturday, March 22, 2008 2:09 PM
Subject: Re: [Dirvish] [Bulk] Re: Full backup every time


Hey Gang, 

I'm having a similar problem. 

root at 2[deathstar]# du -csh * 
51G     deathstar_03-18-2008 
51G     deathstar_03-19-2008 
51G     deathstar_03-20-2008 
51G     deathstar_03-21-2008 
12K     dirvish 
204G    total 

As you can see, each backup is the same size!?! Jens - did you find a 
solution to your problem? I am backing up 13 servers, some debian, some 
Suse, some Ubuntu. In each case, it is the same deal. This leads me to 
think that it is on the drivish server side... Here is my summary file 
from "deathstar" for my most recent backup.... I use 
"/usr/sbin/dirvish-runall" to run my backups for all 13 servers. 

client: backupuser at deathstar 
tree: / 
rsh: ssh 
Server: deathstar 
Bank: /mnt/coraid/system_backups/dirvish 
vault: deathstar 
branch: default 
Image: deathstar_03-21-2008 
Reference: deathstar_03-20-2008 
Image-now: 2008-03-21 20:00:00 
Expire: +10 days == 2008-03-31 20:00:00 
exclude: 
        *~ 
        .nfs* 
        lost+found/ 
        backup/ 
        boot/ 
        dev/ 
        proc/ 
        sys/ 
        tmp/ 
        var/run/ 
        var/lock/ 
        mnt/ 
        home/admin/Shared/newakamaidownload/ 
SET permissions devices numeric-ids stats 
UNSET checksum init sparse whole-file xdev zxfer 


ACTION: rsync -vrltH --delete -pgo --stats -D --numeric-ids 
--rsync-path="/usr/local/bin/rsync-new" 
--exclude-from=/mnt/coraid/system_backups/dirvish/deathsta 
r/deathstar_03-21-2008/exclude 
--link-dest=/mnt/coraid/system_backups/dirvish/deathstar/deathstar_03-20-2008/tree 
backupuser at deathstar:/ /mnt/coraid/system_ba 
ckups/dirvish/deathstar/deathstar_03-21-2008/tree 
Backup-begin: 2008-03-22 00:57:37 
Backup-complete: 2008-03-22 02:18:13 
Status: success 

I would appreciate an help on this. I am running out of space! 

Thanks, 
Doug 



Jens Lang <jenslang at yahoo.de> 
Sent by: dirvish-bounces at dirvish.org 
03/21/2008 07:23 PM 

Please respond to
Dirvish user and developer mailing list <dirvish at dirvish.org>



To
dirvish at dirvish.org 
cc

Subject
Re: [Dirvish] [Bulk] Re:  Full backup every time








Keith Lofstrom schrieb:
>>> what could be the reason that dirvish make a full backup every time I
>>> start the backup? I am running the lates Ubuntu on both, client and 
>>> server.
>>> 
> On Fri, Mar 21, 2008 at 05:24:46PM -0400, Kenneth Lerman wrote:
> 
>> It is supposed to.
>> 
>
> Ken's reply is true, but you might want a little more information
> than that.  Dirvish uses rsync to create full and complete images. 
> On the server disk, there should be a succession of complete images.
> Rsync uses unix/linux hard links to share data file information; if
> a file does not change, there may be many links to it, but it is only
> stored on the backup disk once.  Thus, a 500GB disk can store what 
> appears to be hundreds of 200GB images - only the changed files and
> the directory information is added for each successive image.  For
> example, I have a 500GB backup disk with about 150 images on it,
> each apparently 200GB, and I have used only 350GB of it.  By using
> the "branch" feature of dirvish, files that are identical on many
> different machines can also share data space.
> 
Yes, I know that it is supposed to do a full backup every time. Maybe my
wording was not exact. On my system it transfers the whole directory
every time (e. g. a large photo collection). These files never change.
And, of course, they use disk space on the backup disk.
> This is true in the ideal case.  In reality, big files with little
> changes ( rotating-name logs, mbox mail folders, vmware images) 
> will chew up disk pretty fast.  Use "dateext" for logrotate, 
> Maildir format for mail repositories, and samba links to linux
> filespace from smaller vmware images to minimize the big file 
> changes getting backed up.  Use dirvish-expire to manage storage.
> And look on the mailing list archives, the wiki, and the many
> FAQs and writeups created by other dirvish users for more hints.
>
> If after doing all these things, dirvish is creating huge images
> and not using hardlinks, filling up your backup disk too fast, there
> may be other problems.  The helpful people on this list can help
> after you have made an effort characterize your problem.
The thing is that I have no idea how to find the cause. With another
computer it runs absolutely fine. I even don't know whether to search on
the client or on the server side.

The file system (for the backup) is ext3. I checked the size of the
directory with du?all backups have nearly 40 gb (except those who have
been interrupted). The remaining disk size (checked with df) decreases
by the same amount. I did not check the inode numbers, but I think the
decreasing disk size is proof enough.

On my client I am running a normal rsync server (version 2.6.9). The
server has enough rights to access the files.

I suppose that it for some reason thinks the file has changed although
it hasn't. But why? The file size and the date doesn't change.

Jens
_______________________________________________
Dirvish mailing list
Dirvish at dirvish.org
http://www.dirvish.org/mailman/listinfo/dirvish



_______________________________________________
Dirvish mailing list
Dirvish at dirvish.org
http://www.dirvish.org/mailman/listinfo/dirvish
_______________________________________________
Dirvish mailing list
Dirvish at dirvish.org
http://www.dirvish.org/mailman/listinfo/dirvish

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.dirvish.org/pipermail/dirvish/attachments/20080323/cd40c4e3/attachment-0001.html 


More information about the Dirvish mailing list