Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have faced with a strange problem. I usually use rsync to sync files between servers and now this utility behaves in a strange way.

Firstly, here is the command I use:

server1# rsync -av -e ssh ./server1_dir/ root@192.168.1.1:/server2_dir/

It starts the sync process as it should, but no file is synced, only directories. Not all directories actually, as the rsync process hungs up for a long time resulting in the timeout error.

If I kill the process and make another attempt, it does not start at all. The only message I see:

sending incremental file list

The first thought was - firewall. But both servers don't have it installed. I've even tried manually compiling the latest rsync version with no success though.

Could someone help me on this problem? Many thanks.

Update. strace output on the server1

root@server1 [~]# ps auxf|grep [r]sync
root     13958  0.0  0.0  70676  1232 pts/0    S+   23:29   0:00  |       \_ rsync -avv -e ssh directory1 root@192.168.1.1:/home
root     13959  0.0  0.2  58436  3256 pts/0    S+   23:29   0:00  |           \_ ssh -l root 192.168.1.1 rsync --server -vvlogDtpre.isf . /root

root@server1 [~]# strace -p 13959
Process 13959 attached - interrupt to quit
select(7, [3 4], [], NULL, NULL
share|improve this question
1  
How big is your selection of files? Are you talking hundreds, thousands, or millions? Have you tried looking at the iotop output on the destination server? Is rsync doing anything? How about attaching strace to the rsync process on the destination. Is that showing any activity? What about strace against the source rsync process? –  Zoredache Jun 14 '12 at 22:24
    
Both servers have zero load (CPU, I/O, etc.); I'm syncing a directory with the php and text files inside. The directory is 4M in size. I'm attaching the strace output. –  Andrew Jun 14 '12 at 22:28
1  
is the rsync working fine if you sync to a local directory? did you strace the rsync process that is started on the remote server? Was your manually compiled version installed to BOTH servers? –  rackandboneman Jun 15 '12 at 0:10
    
Yes, rsync has been compiled same on both servers. strace hang up on the 2-nd server as well. –  Andrew Jun 15 '12 at 0:15
1  
BTW, "sending incremental filelist" is a dead giveaway that both sides negotiated rsync 3.x protocol which will start transferring files very quickly even with several terabytes worth of millions of files. Just for future readers with similar problems :) –  rackandboneman Jun 15 '12 at 0:58

2 Answers 2

I ran into the same symptoms recently. Both sides strace showed they were in select() waiting for the other side. I then noticed that the server end had a large send queue in netstat, so I began looking for network-level solutions. I tried reducing the MTU as above, but it didn't make a difference. Then I disabled SACK on both sides, and rsync began to work again:

echo 0 > /proc/sys/net/ipv4/tcp_sack

There's some discussion out there of Cisco bugs with selective ack and sequence number randomization, which is at least a plausible reason for this making a difference.

share|improve this answer
up vote 1 down vote accepted

Finally, the problem has been resolved. It's hard to believe but it was incorrect MTU set on the main network interface. After changing MTU to 1460 the sync process has been started and completed immediately. Thanks all for answers.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.