Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

[root@centos /]# rsync -av --exclude thumbs /storage root@xx.27.1.xx:/storage root@xx.27.1.xx's password: building file list ...

been sitting for about an hour... it's 135GB of images and folders

/storage is a mounted ext3 scsi drive.

is it normal for rsync to be sitting this long calculating the files/directories?

share|improve this question
How many files? With very large numbers of files it's gonna take a while. – Helvick Dec 2 '10 at 19:24
probably thousands... images directory of user uploads... – Andrew Fashion Dec 2 '10 at 19:37
I have seen this portion of rsync take well over an hour when there's thousands and thousands and thousands of files. If you run 'top' are you seeing I/O wait? – troyengel Dec 2 '10 at 19:52
Just because I am curious, can you run df -i and tell us what you have for IUsed on /storage? It will give us a rough idea of how many files+folders you have. – Zoredache Dec 2 '10 at 21:06
1383641, is that the amount of files? – Andrew Fashion Dec 2 '10 at 21:24

rsync 2.x does build a full file list up front.

Add the -P option and you'll see a progress indicator.

If you think it really is hanging, in another terminal find the pid of rsync and then

  1. Run strace -p PID and see what it's doing. (Hit ^C to stop.)

  2. Run ps -o wchan PID to see where it is in the kernel.

share|improve this answer
Wow strace is cool! You'll probably find that something circular is going on: Like a link that refers to one of its ancestor directories. In my case I was using a virtual filesystem that had no limit on its depth, i.e. it had "infinitely" many subdirectories tagfs/books/+/books/+/books/+/ ... – Zaz May 17 '15 at 16:28

You should upgrade to rsync 3.0.x where you will get the benefit of incremental file lists, explained here. I am rsyncing millions of images (~200GB total) and saw an enormous speedup when going from rsync 2.x to 3.x.

Even so, it will probably take a long time to go through all that data. In my case it still takes over an hour between two pretty powerful DL380 G5 servers.

share|improve this answer
Also note that certain flags will force the old behavior; for me, it was --delay-updates. – Xiong Chiamiov Jun 6 at 22:02

You noted earlier that du was hanging as well. Something is up with this filesystem, and rsync is hitting whatever du was.

One quick thing you can check is for kernel error messages indicating a disk problem. Type dmesg and see what's at the bottom.

share|improve this answer

Based on this and your other question of du /storage hanging makes me think that neither du or rsync is the problem but rather there are some issues with /storage.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.