Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

ftp supports the put "|..." "remote-file.name" command to pipe data to an ftp connection. Is there something similar available for sftp?

In sftp i get the following error:

sftp 'jmw@backupsrv:/uploads'
sftp> put "| tar -cx /storage" "backup-2012-06-19--17-51.tgz"
stat | tar -cv /storage: No such file or directory

as above the sftp client doesn't obviously execute the command.

i want to use the pipe command to directly redirect the file stream to sftp. (because there is not enough space left to create a backup file on the same disk before uploading it to sftp server.)

share|improve this question
    
I thought about using a FIFO for this, but neither SFTP nor SCP will read from a FIFO. –  Tom Anderson Jul 5 '12 at 12:27
1  
also good idea, this is mine: qxs.ch/2012/07/05/sftp-upload-tool –  JMW Jul 5 '12 at 14:41
add comment

3 Answers

output-stream-generating-command | ssh user@remotehost 'input-stream-accepting-command' is an option, if your remote user has a valid shell.

share|improve this answer
1  
This is NOT sftp! –  Jiri Xichtkniha Jun 19 '12 at 18:23
2  
@JiriXichtkniha No, it isn't, however SFTP is almost always implemented as a subsystem of your SSH server, and this is the closest analog to the functionality I see being requested from the question (pipe data through SFTP to be handled by a program on the other end). Sometimes the answer is "You're using the wrong tool - do this instead." - this strikes me as one of those times. –  voretaq7 Jun 19 '12 at 18:31
1  
sftp is protol, and is different from just piping via ssh, that's it! Your suggestion would not work if remote part would be SFTP only. –  Jiri Xichtkniha Jun 19 '12 at 19:03
    
@JiriXichtkniha Hence the caveats in my answer. If you have a better alternative for them which works for a pure SFTP server that cannot handle SSH (or where the user is disallowed login, doesn't have a valid shell, etc.) by all means please post an answer with that solution - I would be interested in seeing it myself. –  voretaq7 Jun 19 '12 at 19:32
1  
@JMW in that case as far as I know you're pretty well screwed here - I've never seen a version of sftp that supports piping to a command on the remote system (probably because it would defeat the Secure bit by letting someone in your situation who doesn't have a shell run programs anyway). I could be wrong though - I'm mostly familiar with the SFTP functionality in OpenSSH and the proprietary SSH server Sun used to ship... –  voretaq7 Jun 22 '12 at 16:00
show 3 more comments

I had a lot of fun figuring out a solution to this problem. It requires the tool nc (netcat) on both machines, and SSH (SFTP isn't needed).

In this example, I shall call the machine that has the data that needs to backed up linux-a, and the machine that needs to receive the backup linux-b.

On linux-a, have netcat listen on a port (I took 2000) and redirect it to a file. This will just sit there and wait until something comes through on that port.

[kenny@linux-b /var/backups]$ nc -l 2000 > backup.tgz

On linux-b, open up an ssh tunnel to linux-a, I used port 2000 again. This will redirect anything you throw at TCP port 2000 on localhost to TCP port 2000 on linux-a, where netcat is listening.

[kenny@linux-a /var/data]$ ssh -L 2000:localhost:2000 -CfN linux-b

Now create the tar archive, but send the output to stdout (using -) and pipe it to gzip for some compression. Now pipe that to another netcat that sends it to localhost on TCP on port 2000.

[kenny@linux-a /var/data]$ tar cf - important-data | gzip -fc | nc localhost 2000

We're done! On linux-b the netcat is no longer listening, and a new file is created. The best part is that the tar archive was never placed on the hard disk of linux-a.

[kenny@linux-b /var/backups]$ file backup.tgz 
backup.tgz: gzip compressed data, from Unix, last modified: Thu Jul  5 13:48:03 2012

I know it's not exactly what you asked for in the question, but if you have netcat available, its a viable solution to your type of problem.

Edit: I forgot about one thing: if you follow these instructions, you'll still have an SSH tunnel floating around on linux-a. Find out what the process ID is and kill it.

[kenny@linux-a /var/data]$ ps -ef | grep "ssh -L"
kenny     5741     1  0 13:40 ?        00:00:00 ssh -L 2000:localhost:2000 -CfN linux-b
kenny     5940  3360  0 14:13 pts/1    00:00:00 grep --color=auto ssh -L
[kenny@linux-a /var/data]$ kill 5741
share|improve this answer
1  
i like it, but there are 2 problems: sftponly users usually cannot use port forwarding and sftponly users cannot access nc on the sftp server (remember they do not have ssh access) –  JMW Jul 5 '12 at 12:23
1  
That's entirely valid criticism. –  Kenny Rasschaert Jul 5 '12 at 12:27
    
Off-topic, but why is there a screen shot of the "The Prestige" embedded in this post? –  Rilindo Jul 5 '12 at 14:10
add comment
up vote 1 down vote accepted

voretaq7 pointed out, that the sftp client doesn't support piped data transfer for users, that are allowed to use sftp only to connect to the server.

fortunately, there is libssh2, that supports sftp. so we just need 2 other clients using libssh2, that i've called:

  • sftp_stdin_upload (to upload to a sftp server)
  • sftp_stdout_download (to download from a sftp server)

the source code can be found under the following URL: http://www.qxs.ch/2012/07/05/sftp-upload-tool/


since i'm not that experienced in libssh2 programming, i'm happy for any feedback to the source code.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.