2

I'm in a situation where I have a significant amount of data that I need to transfer from Server A to Server C. However:

  • Server A does not have network access to Server C
  • I have authority to SSH to both Server A and Server C
  • In order to SSH to Server C, I must first go through jump host Server B
  • The data is large enough that I can't realistically download it to my laptop disk from Server A and then upload it to Server C
  • This is a "one time" transfer
  • The servers are running linux and the laptop is a Mac

A brief diagram depicting the situation:

Server connectivity

Is there some way to 'stream' the files through my laptop from Server A to Server C?

2

If your jumpserver doesn't restrict you too much, you can do it in one command on your notebook:

ssh jumpserverB ssh serverC tar cf - dir1 dir2 | ssh serverA tar xvf -

Voila!

Edit: uh - that was wrong direction. So let's do the other way:

ssh serverA tar cf - dir1 dir2 | ssh jumpserverB ssh serverC tar xvf -

dir1 and dir2 being two directories containing the data you want to transfer.

I'm EDITing this answer again - although FGreg alredy solved his original problem - because his solution using -J will work on some machines, but not on others. Thas is because their ssh might not know about -J flag, just as it is the case with mine. So here come some answers to some comments:

  • I used tar (tape archiver), because it can handle an unlimited amount of data and - if you use standard output as your "tape", you can pipe it through anything, without storing it anywhere inbetween. This way, as long as you can ssh from one machine to the next, you can transport data over an unlimited amount of hops.
  • Such things as "Host key verification failed" can be sorted out by ssh-ing to the first machine, then typing "ssh nextmachine" and so on. Of course if you have to change user inbetween or to specify pubkeys using -i, you have to do that in your pipe ssh commands, too.
  • Yes, of course, tar can compress the data for you; in most cases this will speed up the process. I omitted it for simplicity.
  • FGreg's solution in his own answer is somewhat differen in that it doesn't use a pipe. It uses ssh's port forwaring and is a great way to handle situations where a pipe just doesn't do the job. Again, you can add as many port forwarings in a row as you want. Like on host1: ssh host2 -L 3333:host3:22. This takes you to host2 as usual. Again on host1 in another window you type ssh localhost -p 3333 -L 4444:host4:22; this takes you to host3. ssh -p 4444 ... you get the idea.
  • Sometimes you need reverse forwarding, as FGreg had to when he connected to serverA. Adding to our -L examples above we could do that using ssh serverA -R 4444:localhost:4444 serverA; there we can ssh -p 4444 localhost to reach host4 or scp -P 4444 localhost.
  • Of course we can tunnel completely other stuff to and from from/to other ports, but that would become an ssh general blog post, so I'll stop here.
| improve this answer | |
  • When I tried this I got "Host key verification failed". If I modify it slightly though and use ssh serverA tar cf - file | ssh -J serverB serverC tar xvf - it seems to work. Why use tar? – FGreg Sep 14 '17 at 16:18
  • If the data compresses well, you might do with tar czf (and tar xzvf) to compress in between. That way less data is sent across the wire. I would presume that you have more cpu on both sides than network bandwidth. – lsd Sep 14 '17 at 21:14
1

Thanks to Andrzej A. Filip for pointing out https://unix.stackexchange.com/questions/85292/scp-from-one-remote-server-to-another-remote-server

I was able to solve this using one of the answers on that question. Here are the steps I took:

  1. Add ServerA's public SSH key to authorized_keys on ServerC
  2. From laptop, forward remote port on ServerA to local port on laptop

    [user@laptop]$ ssh -R 5001:localhost:5002 serverA -N
    
  3. From laptop, forward local port on laptop to remote port on ServerC (through jump host ServerB)

    [user@laptop]$ ssh -J userb@serverB -i ~/.ssh/id_rsa.pub -L 5002:localhost:22 userb@serverC -N
    
  4. From ServerA, scp files over port 5001

    [usera@serverA]$ scp -P 5001 ./* userb@localhost:/data/location/on/serverC
    

It's a little verbose but seems to work.

The data never resides on laptop and is essentially going through an SSH tunnel like so:

ServerA:5001 -> laptop:5002 -> ServerC:22
| improve this answer | |
  • "The data is large enough that I can't realistically download it to my laptop from Server A and then upload it to Server C.": Aren't you doing this, in fact? – gf_ Sep 14 '17 at 16:36
  • @gf_ I hope not? Haha I'm not 100% sure but my understanding is because this method is essentially port forwarding through my laptop the data is never actually being stored on my laptop disk. – FGreg Sep 14 '17 at 16:38
  • Well, yes. It's not stored on your disk, but still you're downloading and re-uploading it. – gf_ Sep 14 '17 at 16:40
  • @gf_ Right, but I don't think I can get around that since ServerA can't communicate directly with ServerC (or ServerB). So no matter the solution I need to use my laptop as the "go-between". – FGreg Sep 14 '17 at 16:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.