44

Note to serverfault users: This closed question is answered here by four comments by hynekcer. It is then well accepted in other comments as a regularly answerable question. It's now being voted for re-opening.


When both source and destination are remote, rsync complains:

The source and destination cannot both be remote. rsync error: syntax or usage error (code 1) at main.c(1156) [Receiver=3.0.7]

Is there an insurmountable technical obstacle to making rsync do this? Or is it simply a case of not-yet-implemented? It seems relatively easy to create a local buffer in memory that mediates the transfer between two remotes, holding both hashes and data.

10
  • 1
    It would involve a remote src rsyncd sending data to a remote dest rsyncd. You can work around it by ssh'ing to the src system and invoking rsync.
    – Alex Holst
    Nov 17, 2012 at 14:18
  • @AlexHolst I don't think that would work in my particular case. see edit Nov 18, 2012 at 1:35
  • 4
    Sorry (moderators), this is an answerable interesting question: The reason is that the rdiff algorithm can not be symmetric. A bigger CPU and memory overhead is on the "active" side. The "passive" side needs only to compute checksums of all blocks (see --block-size parameter) of all modified files and to resend them. That is done with very small memory requirements and the most of operations can be done in the first level CPU cache. The "active" side needs to search by a checksums where the same data block is located now...
    – hynekcer
    Nov 13, 2016 at 20:27
  • 2
    That requires to hold 24 bytes of memory per every block and a frequent random access to that big temporary memory database to search 4 bytes per every byte of misplaced data, computed on the fly by rolling hash algorithm for all bytes of the block. It is clear that this big range must use the slow uncached memory. It would be a controversial idea to implement the "active" part on a remote side if you imagine that it could be a file server that must execute more similar concurrent requests, or even a cheap NAS server...
    – hynekcer
    Nov 13, 2016 at 20:29
  • 2
    I think this is a great question (I had exactly this question, that's why I'm here) and the closing reason is bogus. I still want to know the answer! Mar 16, 2017 at 9:49

3 Answers 3

8

why not try and connect to the remote machine and start the transfer from there. If you are using ssh-keys you can use agent pass though to manage the authenticate for you.

ssh -A remotehostA rsync /remote/file/on/host/a remoteHostB:/destination/

This command will log you on the remoteHostA and run rsync from there.

3
  • 5
    There's some security considerations involved. See edit Nov 18, 2012 at 1:38
  • 5
    Also, that won't work if you don't have direct access between the two systems... And if getting direct access involves waiting two weeks for a security section to maybe correctly implement firewall rules... Nov 25, 2014 at 12:38
  • 1
    Scenario: Server A has key-based root access to server B and C. You want to sync from B to C using root access on both. But you don't want B or C to have root access to each other. Nov 12, 2017 at 23:05
6
scp -3r <remote src> <remote dest>

has no trouble doing this.

4
  • 5
    scp doesn't do deltas though, AFAIK, and it's badly needed in this case. More details on my edit Nov 18, 2012 at 1:42
  • 4
    Btw, scp have to be with options -3 if you'd like to be like a proxy between hosts
    – alterpub
    Dec 3, 2014 at 15:18
  • 1
    Unfortunately scp lacks important features of rsync eg: option to not cross over filesystems (eg network drive mounted in user folder), archive mode (in which "symbolic links, devices, attributes, permissions, ownerships, etc. are preserved"), etc...
    – Bastion
    Sep 12, 2019 at 4:08
  • I wish it wasn't this slow (
    – Nakilon
    Jun 29, 2021 at 21:17
0

You can work around this by mounting one (or both) of the remote filesystems with sshfs. Then, rsync will treat it as if it were local.

Unfortunately, this will result in a lot of bandwidth usage on the machine whose filesystem is mounted with sshfs, so I would recommend only doing this with the machine that has a lot of bandwidth between you and the third machine.

Of course, the ideal solution is for the machines to talk directly to each other. I can't think of any good reason why they should not.

9
  • see edit. The reason you're referring to is security. A (root) compromise any one of the two machines must not lead to filesystem access to the other. But maybe I'm attacking this from the wrong angle and there's a solution that doesn't involve a third machine... Nov 18, 2012 at 1:41
  • Hmm. I think those details really should be added to this question. As for a root compromise, you really should be using SELinux. Nov 18, 2012 at 1:47
  • I left this one because someone else may be interested in this behaviour on rsync in particular. AFAIK, SELinux on any of the machines cannot tell if the other has been compromised if the locally executing rsync has the exact same behaviour in both cases (filesystem access to a particular directory). Nov 18, 2012 at 1:52
  • I'm not sure if you understand how SELinux works. Its whole point is that it prevents a service (even a compromised service) from accessing things it isn't explicitly allowed to access, even if that service runs as root. Nov 18, 2012 at 1:59
  • I have only a basic understanding of SELinux policies, but rsync is supposed to access the filesystem, isn't it? The exact same kind of (local) access patterns are undesirable if the remote machine is compromised. Nov 18, 2012 at 2:19

Not the answer you're looking for? Browse other questions tagged or ask your own question.