I recently asked a similar question about how to transfer large (many gigabyte to many tens of gigaytes) files over the Internet when latency is high. I got a great answer, but only for downloads. When I want to upload something, from my computer that is behind a firewall that only allows outgoing TCP connections, to a server, I can't get lftp or any other tool to satuate my gigabit upload (to a gigabit server that is 115ms away). lftp seems to cap out around 36.88mbps, even when doing a parallel transfer, such as with this command:

lftp -e 'mirror --parallel=1 --use-pget-n=10 -R . /destination/path' sftp://user@destinationdomain

I also tried some variations of this command (e.g. using put instead of mirror, and different flags), but nothing gets anywhere near saturating my gigabit upload. It's possible in this case that there's a bad link between me and the server, since although both are gigabit neither is a particularly good network (both have issues when sending data to far-away places, sometimes). To test this hypothesis, I ran a speedtest.net speed test to another ISP in the same city as the destination server (the host of the server does not host a speedtest.net server), and I got this result:

116ms latency, 622.46mbps download, 488.81mpbs upload, significant increase in upload and download speed as test progresses

which seems to rule out any issue with intermediate networks (at least, anything severe enough to cause 36.88mbps), which is further strengthened by the fact that not only does my route to this speedtest.net server use the same long-distance transit provider (Hurricane Electric), but it even seems to go through the same datacenter that my server is hosted in with the same provider!

So, I ask, how can I easily (meaning with a single command) upload from my computer to a remote server while saturating my gigabit connection, or, at least, a significant fraction of it as speedtest.net is able to do?

  • Use lftp in the opposite direction? – Michael Hampton Oct 4 '20 at 2:29
  • @MichaelHampton I tried that and got 36.88mbps, as depicted in the question. – john01dav Oct 4 '20 at 2:33
  • With parallel=1? Are you only downloading one file? – Michael Hampton Oct 4 '20 at 2:38
  • Also note the cli tools are likely to measure speed in bytes while speedtest measures in bits so you're already reaching ⅔ of your maximum (per speedtest) upload rate. (unless lftp measures in bits as well) – Ginnungagap Oct 4 '20 at 7:20
  • @michaelhampton I'm only uploading one file. Also, I misunderstood your first comment. I can't use LFTP on my server to download from my home computer because my ISP blocks port forwarding (they want US$70 per month to enable it). – john01dav Oct 5 '20 at 7:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.