[Dirvish] problems initializing image

Dave Howorth dhoworth at mrc-lmb.cam.ac.uk
Mon Aug 6 09:41:56 UTC 2007


I'm glad the list is working, because I have a problem of my own to ask
about :)

I'm having problems initializing a 60 GB image. I'm still investigating
but wondered if anybody recognized what I'm doing wrong. I use dirvish
to mirror a remote archive, so I get a history of the archive as it
changes. I've been doing this successfully for a few years but recently
the archive changed it's rsync server and some of the underlying data. I
decided to initialize a fresh mirror, so I created a new vault, but when
I try to initialize it, divish runs for a couple of days and then dies:

  dirvish --config /etc/dirvish/wwpdb.conf --vault wwpdb-mirror --init
  dirvish wwpdb-mirror:default fatal error: write error, filesystem
probably full
  dirvish wwpdb-mirror:default fatal error (12) -- write error,
filesystem probably full

Now I don't think the filesystem is full because it has 180 GB of free
space. The exact errors in rsync_error are:


*** Execution cycle 0 ***

rsync: read error: Connection reset by peer (104)
rsync error: error in rsync protocol data stream (code 12) at io.c(515)
rsync: connection unexpectedly closed (25368544 bytes received so far)
[generator]
rsync error: error in rsync protocol data stream (code 12) at io.c(359)


*** Execution cycle 1 ***

rsync: connection unexpectedly closed (8547231945 bytes received so far)
[receiver]
rsync: writefd_unbuffered failed to write 4092 bytes: phase "unknown"
[generator]: Connection reset by peer (104)
rsync error: error in rsync protocol data stream (code 12) at io.c(359)
rsync error: error in rsync protocol data stream (code 12) at io.c(909)


and this is the summary file:


client: rsync.wwpdb.org
tree: :ftp
rsh: ssh
Server: suse1
Bank: /data/wwpdb
vault: wwpdb-mirror
branch: default
Image: 2007-08-03
Reference: default
Image-now: 2007-08-03 11:19:18
Expire: Never
rsync-option:
        --port=33444
SET devices init numeric-ids stats zxfer
UNSET permissions checksum sparse whole-file xdev


ACTION: rsync -vrltH --delete --stats -z -D --numeric-ids --port=33444
rsync.wwpdb.org::ftp/ /data/wwpdb/wwpdb-mirror/2007-08-03/tree
Backup-begin: 2007-08-03 11:19:18
Status: fatal error (12) -- write error, filesystem probably full



The log file indicates that it downloaded about 300,000 files and then
ends with:

write error, filesystem probably full
broken pipe
broken pipe
RESULTS: warnings = 0, errors = 4




So from the rsync_error file it looks like it only read 8 GB before
having the fatal problem. One feature that puzzles me is that there is
no 'tree' directory in the image. Whatever data it has downloaded was
apparently thrown away. This isn't the normal behaviour of dirvish/rsync
when a fault occurs, IIRC.

Can anybody see any clues?

Thanks and regards,
Dave


More information about the Dirvish mailing list