42

Note to serverfault users: This closed question is answered here by four comments by hynekcer. It is then well accepted in other comments as a regularly answerable question. It's now being voted for re-opening.


When both source and destination are remote, rsync complains:

The source and destination cannot both be remote. rsync error: syntax or usage error (code 1) at main.c(1156) [Receiver=3.0.7]

Is there an insurmountable technical obstacle to making rsync do this? Or is it simply a case of not-yet-implemented? It seems relatively easy to create a local buffer in memory that mediates the transfer between two remotes, holding both hashes and data.

10
  • 1
    It would involve a remote src rsyncd sending data to a remote dest rsyncd. You can work around it by ssh'ing to the src system and invoking rsync. – Alex Holst Nov 17 '12 at 14:18
  • @AlexHolst I don't think that would work in my particular case. see edit – loopbackbee Nov 18 '12 at 1:35
  • 4
    Sorry (moderators), this is an answerable interesting question: The reason is that the rdiff algorithm can not be symmetric. A bigger CPU and memory overhead is on the "active" side. The "passive" side needs only to compute checksums of all blocks (see --block-size parameter) of all modified files and to resend them. That is done with very small memory requirements and the most of operations can be done in the first level CPU cache. The "active" side needs to search by a checksums where the same data block is located now... – hynekcer Nov 13 '16 at 20:27
  • 2
    That requires to hold 24 bytes of memory per every block and a frequent random access to that big temporary memory database to search 4 bytes per every byte of misplaced data, computed on the fly by rolling hash algorithm for all bytes of the block. It is clear that this big range must use the slow uncached memory. It would be a controversial idea to implement the "active" part on a remote side if you imagine that it could be a file server that must execute more similar concurrent requests, or even a cheap NAS server... – hynekcer Nov 13 '16 at 20:29
  • 2
    I think this is a great question (I had exactly this question, that's why I'm here) and the closing reason is bogus. I still want to know the answer! – reinierpost Mar 16 '17 at 9:49
8

why not try and connect to the remote machine and start the transfer from there. If you are using ssh-keys you can use agent pass though to manage the authenticate for you.

ssh -A remotehostA rsync /remote/file/on/host/a remoteHostB:/destination/

This command will log you on the remoteHostA and run rsync from there.

3
  • 5
    There's some security considerations involved. See edit – loopbackbee Nov 18 '12 at 1:38
  • 4
    Also, that won't work if you don't have direct access between the two systems... And if getting direct access involves waiting two weeks for a security section to maybe correctly implement firewall rules... – Gert van den Berg Nov 25 '14 at 12:38
  • 1
    Scenario: Server A has key-based root access to server B and C. You want to sync from B to C using root access on both. But you don't want B or C to have root access to each other. – thomasrutter Nov 12 '17 at 23:05
6
scp -3r <remote src> <remote dest>

has no trouble doing this.

3
  • 4
    scp doesn't do deltas though, AFAIK, and it's badly needed in this case. More details on my edit – loopbackbee Nov 18 '12 at 1:42
  • 4
    Btw, scp have to be with options -3 if you'd like to be like a proxy between hosts – alterpub Dec 3 '14 at 15:18
  • Unfortunately scp lacks important features of rsync eg: option to not cross over filesystems (eg network drive mounted in user folder), archive mode (in which "symbolic links, devices, attributes, permissions, ownerships, etc. are preserved"), etc... – Bastion Sep 12 '19 at 4:08
0

You can work around this by mounting one (or both) of the remote filesystems with sshfs. Then, rsync will treat it as if it were local.

Unfortunately, this will result in a lot of bandwidth usage on the machine whose filesystem is mounted with sshfs, so I would recommend only doing this with the machine that has a lot of bandwidth between you and the third machine.

Of course, the ideal solution is for the machines to talk directly to each other. I can't think of any good reason why they should not.

9
  • see edit. The reason you're referring to is security. A (root) compromise any one of the two machines must not lead to filesystem access to the other. But maybe I'm attacking this from the wrong angle and there's a solution that doesn't involve a third machine... – loopbackbee Nov 18 '12 at 1:41
  • Hmm. I think those details really should be added to this question. As for a root compromise, you really should be using SELinux. – Michael Hampton Nov 18 '12 at 1:47
  • I left this one because someone else may be interested in this behaviour on rsync in particular. AFAIK, SELinux on any of the machines cannot tell if the other has been compromised if the locally executing rsync has the exact same behaviour in both cases (filesystem access to a particular directory). – loopbackbee Nov 18 '12 at 1:52
  • I'm not sure if you understand how SELinux works. Its whole point is that it prevents a service (even a compromised service) from accessing things it isn't explicitly allowed to access, even if that service runs as root. – Michael Hampton Nov 18 '12 at 1:59
  • I have only a basic understanding of SELinux policies, but rsync is supposed to access the filesystem, isn't it? The exact same kind of (local) access patterns are undesirable if the remote machine is compromised. – loopbackbee Nov 18 '12 at 2:19

Not the answer you're looking for? Browse other questions tagged or ask your own question.