Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm downloading periodically gzipped database dumps from my AWS setup (roughly 150MB).

Although normal http/https download speeds (via Apache file serving) are >>300kb/sec, when I use scp for downloading the dump I'm only getting around 30-50kb/sec speeds.

What's the reason it's so slow?

How can I securely download database dumps files much faster?

Thanks!

share|improve this question
add comment

4 Answers 4

up vote 1 down vote accepted
+50

I had a similar problem, and eventually it turned out to be due to my ISP throttling (or otherwise messing with) traffic on non-standard ports. They apparently think that anything which isn't on ports 25/80/443 etc is file sharing and intentionally slow it down.

I suggest you try to switch ISP, or at first you can also try configuring the sshd to listen on port 443 (assuming you don't have a web server with SSL on that machine) and try this way.

share|improve this answer
add comment

Try changing the cipher_spec for the scp session with -c blowfish-cbc (or Ciphers in your .ssh/config file). This will lighten the encryption load slightly.

On my local LAN, I saw a speed increase from ~28MB/s to ~35MB/s.

share|improve this answer
    
both the server and my machine are strong so that the compression load for a 300kb/s stream is insignificant (far from being accountable to an order of magnitude decrease in throughput) –  GJ. Feb 4 '11 at 11:49
    
+1 to parent. I agree that it shouldn't be the issue, but WinSCP in particular isn't terribly fast by default. Have you tried giving it a shot? –  Jeff McJunkin Feb 9 '11 at 7:29
add comment

Why not use gpg locally to encrypt the dumps to yourself? Then you can transfer them over HTTP, or over FTP, or post them on usenet, or do as you please, without imperiling your security (well, you probably shouldn't post them to alt.binaries).

I do that with my colo'ed box's backups, and can then happily store them on my colo's online storage, without endangering myself. My colo'ed box doesn't need my private key, either - only the public one, to encrypt to. I'd do the decryption at home if the need ever arose.

If you do this, don't forget to turn off ASCII-armouring for gpg, otherwise your backup file will get 30% bigger than it should be.

share|improve this answer
add comment

Since you're already running apache, why not create a separated VirtualHost, a self-signed certificate and add basic auth to the directory where the database dumps are located? That should allow you to securely download database dump files faster. It maybe a workaround, but it should accomplish the goal. Two useful links are:

http://www.akadia.com/services/ssh_test_certificate.html

http://httpd.apache.org/docs/2.2/howto/auth.html

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.