Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm trying to create a backup script using FreeBSD's dump to take a snapshot of a filesystem, but instead of storing the snapshot on disk I want to pipe the output from dump into cURL or rsync so its stored remotely. I've got it working piped via SSH to dd the output to the remote end, however I need to use cURL or rsync to do it for security reasons.

I've tried a number of different command line arguments:

dump -0u -a -L -f - /dev/ad0b | curl -v --data @- http://1.2.3.4/
dump -0u -a -L -P "curl -v --data - http://1.2.3.4/" /dev/ad0b
dump -0u -a -L -P "rsync --progress --stats rsync://1.2.3.4/backup" /dev/ad0b

No matter what I do I either get a broken pipe or just no data coming through.

I can manually dump it to a file then upload this via cURL or rsync, but this defeats the purpose of what i'm trying to do.

Any help or pointers would be awesome.

share|improve this question
add comment

1 Answer 1

The basic approach with either of your two curl attempts looks ok. However your dump is binary data, so --data is wrong (the data will be mangled). Depending on what your web server expects, you may have more luck with one of

  • curl -v --data-urlencode @- http://1.2.3.4/, if the expected format is URL-encoded data.
  • curl -v --data-binary -H 'Content-Type: octet/stream' @- http://1.2.3.4/, if the expected format is raw binary data.
  • curl -v -F "mydump=@-" http://1.2.3.4/, if the expected format is a form.

This all hinges on what the HTTP server expects. Check the documentation of whatever CGI you're using (or post it here if you need help).

If the server listening on 1.2.3.4 expects raw data rather than an HTTP POST request, then curl is the wrong tool (and http is wrong too, of course). Netcat is the most popular tool to pipe data into a TCP socket: nc 1.2.3.4 5678.

Rsync is unlikely to work. The main purpose of rsync is to copy only the parts of a file that are not the same on both sides, which requires a lot of seeking inside the source and target files. Furthermore, rsync is designed to work on a hierarchy of files, not a single file. It would be technically possible to write an rsync client that always copies the whole file and so doesn't need to seek the source data, but this is such an unusual need that I doubt any exists.

share|improve this answer
    
Thanks Giles, much appreciated. –  Jeremy Jun 23 '11 at 2:28
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.