2

I have a problem with a defective round robin data mover node I'm rsync'ing data from. One of the nodes keeps dropping the data connection to the file system my data is coming from and needs to reconnect, which can take up to several minutes. During this time, the rsync on that particular file does not abort, it just hangs, for up to several minutes, until the download continues. Because this is time critical data, that is not a viable option, but instead should drop the connection asap when it stalls, and reconnect (which would then be load balanced to the next node anyway).

Can anyone think of a method to tell rsync to abort a transfer that's been stalled for more than e.g. 20 seconds?

The operator of the broken data mover nodes are slow to react to this problem, they're aware of it, but not doing anything at the moment, so I need to find a solution at my end.

I've tried using the --timeout option, but that doesn't seem to work when a data connection is stalled.

0

If you have an idea of the max duration for each rsync run, you can use timeout:

$ timeout 120 /usr/bin/rsync <options>
| improve this answer | |
  • Indeed, I've been doing that, but it's not ideal because the rsync wildcard catches such a huge number of files that the individual transfer timeout vs. the full download timeout is in a bad relationship to re-commencing the transfer every time. – John Feb 27 '17 at 16:15
0

What rsync transport are you using? If SSH, then you might be able to pass configuration options to SSH to help you time out your connection. I don't have a good method to test, but maybe the ServerAliveInterval, and TCPKeepAlive options might give you want you need?

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.