[Dirvish] large memory footprint

Arash Abghari arash.abghari at gmail.com
Thu Aug 30 04:53:46 UTC 2012


Hi,

I am using dirvish to backup my harddisk. The backup storage is on 
another harddisk all with in the same machine. I have 20G of RAM 
installed on this machine. I noticed that after dirvish run the memory 
footprint of the system increases from around 700Meg to almost 6Gig. If 
I disable dirvish, there won't be any memory usage increase. The 
interesting thing is you can not find any process that uses this amount 
of memory but free command showing this amount of memory is used (not 
cached). The only way to get rid of this extra memory usage is to 
restart the machine. Here is the output of free and smem command after a 
dirvish run:

==========================
# free -lm
              total       used       free     shared    buffers cached
Mem:         20116      14105       6011          0       1693 6240
Low:         20116      14105       6011
High:            0          0          0
-/+ buffers/cache:       6170      13945
Swap:        37347          0      37347


==========================
# smem -tk
   PID User     Command                         Swap      USS PSS      RSS
  2750 root     /sbin/getty 38400 tty2             0    92.0K 101.0K   
620.0K
  2751 root     /sbin/getty 38400 tty3             0    96.0K 105.0K   
624.0K
  2752 root     /sbin/getty 38400 tty4             0    96.0K 105.0K   
624.0K
  2753 root     /sbin/getty 38400 tty5             0    96.0K 105.0K   
624.0K
  2754 root     /sbin/getty 38400 tty6             0    96.0K 105.0K   
624.0K
  2749 root     /sbin/getty 38400 tty1             0   100.0K 109.0K   
628.0K
  1424 root     /sbin/portmap                      0   140.0K 159.0K   
532.0K
  3933 root     gnome-pty-helper                   0   160.0K 167.0K   
788.0K
  1926 root     /usr/sbin/atd                      0   180.0K 183.0K   
436.0K
  2489 root     /usr/sbin/acpid                    0   180.0K 198.0K   
648.0K
     1 root     init [2]                           0   200.0K 209.0K   
808.0K
  2053 root     avahi-daemon: chroot helpe         0   116.0K 218.0K   
528.0K
  2710 root     hald-addon-acpi: listening         0   212.0K 231.0K     
1.2M
  2709 root     /usr/lib/hal/hald-addon-cpu        0   212.0K 233.0K     
1.2M
  2707 root     hald-addon-storage: polling        0   224.0K 250.0K     
1.4M
  2688 root     hald-addon-input: Listening        0   228.0K 253.0K     
1.3M
  1861 root     /sbin/rpc.statd                    0   224.0K 259.0K   
856.0K
  3102 root     udisks-daemon: polling /dev        0   164.0K 265.0K   
888.0K
  2665 root     hald-runner                        0   260.0K 274.0K     
1.3M
19868 arash    /bin/bash ./run_parse              0   212.0K 300.0K     
1.3M
19850 arash    /bin/bash ./jeff_run_compil        0   212.0K 303.0K     
1.3M
  2017 root     /usr/sbin/cron                     0   284.0K 309.0K     
1.0M
  3068 arash    /usr/bin/dbus-launch --exit        0   324.0K 331.0K   
808.0K
  3065 root     /usr/bin/ssh-agent /usr/bin        0   356.0K 356.0K   
416.0K
19869 arash    ./ParserLib/parse_text -con        0   412.0K 418.0K   
900.0K
  2841 root     udevd --daemon                     0   120.0K 432.0K     
1.0M
   453 root     udevd --daemon                     0   148.0K 458.0K     
1.2M
  1980 root     /usr/sbin/kerneloops               0   456.0K 466.0K     
1.2M
21237 root     su                                 0   400.0K 488.0K     
1.7M
  3207 arash    /usr/lib/gvfs/gvfsd-burn --        0   440.0K 513.0K     
2.6M
  3082 arash    /usr/lib/gvfs/gvfsd                0   484.0K 541.0K     
2.5M
  3112 arash    /usr/lib/gvfs/gvfs-afc-volu        0   476.0K 554.0K     
2.5M
  2401 root     /usr/sbin/exim4 -bd -q30m          0   548.0K 556.0K     
1.0M
  2047 root     avahi-daemon: running [aras        0   444.0K 561.0K     
1.6M
   574 root     udevd --daemon                     0   452.0K 595.0K     
1.1M
  2115 root     /usr/sbin/sshd                     0   560.0K 622.0K     
1.1M
  2623 root     /usr/sbin/gdm3                     0   616.0K 673.0K     
3.0M
  2048 root     /usr/sbin/bluetoothd               0   560.0K 674.0K     
1.9M
  3110 arash    /usr/lib/gvfs/gvfs-gphoto2-        0   612.0K 684.0K     
2.5M
  3218 arash    /usr/lib/gvfs/gvfsd-metadat        0   696.0K 724.0K     
2.3M
  3198 arash    /usr/lib/gvfs/gvfsd-trash -        0   640.0K 732.0K     
3.1M
  2435 root     /sbin/wpa_supplicant -u -s         0   568.0K 763.0K     
2.1M
  2528 root     /usr/sbin/hddtemp -d -l 127        0   796.0K 802.0K     
1.1M
  2499 root     /usr/sbin/winbindd                 0   324.0K 825.0K     
1.6M
  2084 root     /usr/sbin/modem-manager            0   812.0K 847.0K     
2.5M
21117 root     sshd: arash at pts/2                  0   452.0K 892.0K     
1.9M
  2947 root     /usr/lib/gdm3/gdm-session-w        0   760.0K 898.0K     
3.3M
  3069 arash    /usr/bin/dbus-daemon --fork        0   844.0K 979.0K     
1.4M
  2951 root     /usr/lib/upower/upowerd            0   840.0K 988.0K     
3.5M
  3173 arash    /usr/lib/pulseaudio/pulse/g        0   772.0K 
1001.0K     3.5M
  1986 root     /usr/sbin/ntpd -p /var/run/        0   884.0K 1.0M     2.2M
  3101 root     /usr/lib/udisks/udisks-daem        0   864.0K 1.0M     3.4M
  2378 root     /usr/sbin/winbindd                 0   580.0K 1.1M     2.4M
  1979 root     /usr/bin/dbus-daemon --syst        0   996.0K 1.1M     1.7M
21112 root     sshd: arash [priv                  0   476.0K 1.1M     3.6M
  3098 arash    /usr/lib/gvfs/gvfs-gdu-volu        0     1.0M 1.2M     3.8M
  3116 arash    /usr/lib/bonobo-activation/        0     1.0M 1.2M     3.9M
  2629 root     /usr/lib/gdm3/gdm-simple-sl        0     1.1M 1.3M     4.2M
  2851 root     /usr/sbin/console-kit-daemo        0     1.4M 1.5M     3.6M
  2949 root     /usr/lib/policykit-1/polkit        0     1.4M 1.5M     4.1M
  1852 root     /usr/sbin/rsyslogd -c5             0     1.5M 1.5M     2.2M
  3123 arash    /usr/lib/policykit-1-gnome/        0     1.3M 1.5M     6.5M
  2475 root     /usr/sbin/cupsd -C /etc/cup        0     1.4M 1.6M     3.1M
  2051 root     /usr/sbin/NetworkManager           0     1.5M 1.7M     4.2M
  3079 arash    /usr/bin/seahorse-agent --e        0     1.9M 2.1M     6.4M
  3029 arash    x-session-manager                  0     1.9M 2.2M     7.6M
  3146 arash    /usr/lib/gnome-disk-utility        0     1.8M 2.2M     8.1M
  3175 arash    gnome-screensaver                  0     2.1M 2.3M     6.9M
  2946 Debian-gdm /usr/lib/policykit-1-gnome/        0     2.0M 2.3M     
7.9M
  3159 arash    /usr/bin/pulseaudio --start        0     2.9M 3.3M     5.3M
  3165 arash    /usr/lib/gnome-applets/mult        0     2.6M 3.5M    12.7M
21244 root     bash                               0     3.4M 3.5M     5.0M
  5179 arash    /bin/bash                          0     3.4M 3.6M     5.0M
  3934 arash    bash                               0     3.4M 3.6M     5.0M
  2664 root     /usr/sbin/hald                     0     3.7M 3.7M     5.3M
  3086 arash    /usr/lib/libgconf2-4/gconfd        0     4.1M 4.2M     6.4M
  3124 arash    gnome-volume-control-applet        0     3.5M 4.4M    12.1M
  3137 arash    update-notifier                    0     4.3M 5.0M    14.0M
  3919 arash    /usr/lib/notification-daemo        0     4.4M 5.0M    13.8M
  3121 arash    nm-applet --sm-disable             0     4.3M 5.1M    13.0M
  3093 arash    /usr/lib/gnome-settings-dae        0     4.6M 5.3M    12.0M
  3095 arash    /usr/bin/metacity                  0     5.0M 5.8M    14.2M
21351 root     /usr/bin/python /usr/bin/sm        0     6.5M 6.9M     8.8M
21118 arash    -bash                              0     7.1M 7.3M     8.7M
  3010 arash    /usr/bin/gnome-keyring-daem        0     8.1M 8.2M     9.7M
  6289 arash    gvim Grammar/Order.gra             0     8.5M 9.0M    15.7M
  3932 arash    gnome-terminal                     0     8.3M 9.3M    18.6M
  3154 arash    python /usr/bin/system-conf        0    10.3M 12.1M    
20.1M
  3133 arash    stardict -h                        0    15.1M 16.6M    
27.6M
  3099 arash    gnome-panel                        0    14.7M 16.7M    
28.7M
  3114 arash    nautilus                           0    20.1M 22.3M    
34.6M
  3130 arash    /usr/bin/python /usr/share/        0    24.3M 27.0M    
39.0M
  2646 root     /usr/bin/Xorg :0 -br -verbo        0    66.3M 70.0M    
75.0M
  3131 arash    /usr/lib/icedove/icedove-bi        0   182.6M 184.6M   
196.5M
-------------------------------------------------------------------------------
    94 3                                           0   464.9M 497.8M   
765.0M

As you can see there is a big difference between the memory usage 
reported by these two tools. Here is the spec of the machine and dirvish 
config files:

==========================
# cat /etc/issue
Debian GNU/Linux 6.0 \n \l

==========================
# uname -a
Linux arash-pc 3.2.0-0.bpo.3-amd64 #1 SMP Thu Aug 23 07:41:30 UTC 2012 
x86_64 GNU/Linux

==========================
# cat /etc/hostname
arash-pc


==========================
# cat /etc/hosts
127.0.0.1    localhost
192.168.xxx.xxx    arash-pc

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

==========================
# cat /etc/dirvish/master.conf
bank:
      /mnt/backup

image-default: %Y-%m-%d_%H:%M

log: bzip2

index: bzip2

xdev: 1

exclude:
         lost+found/

expire-default: +5 days

Runall:
        arash-desktop

==========================
# cat /mnt/backup/arash-desktop/dirvish/default.conf
client: arash-pc

tree: /

exclude:
     + /etc
     + /home
     + /home/arash
     - /home/arash/Lizard
     - /home/*
     - /*

xdev: 0

==========================


I really appriciate any help and thanks for your time.

Arash








More information about the Dirvish mailing list