I am looking to copy some large files accross a flaky WAN connection. Is there a tool I can use which will auto resume when the network fall over briefly and let me manually resume if I have to fix it manually?
asked Jun 16, 2009 at 23:26
-
What OS is this for?– JosephJun 16, 2009 at 23:52
-
The os is windows– jimconstableJun 17, 2009 at 2:01
Add a comment
|
8 Answers
The native windows tools to do this are xcopy /z and using BITS. Xcopy /z is pretty self explanatory. Bitsadmin is how you manage a bits tranfer in windows 2003, howver if you can install powershell v2 it contains cmdlet modules to do bits transfers. See Transferring (Large) Files Using BITs. My favorite exe tool is XXcopy.
-
1Robocopy would also be a built-in (for some versions of Windows) solution. But XXcopy is probably preferable. Jul 19, 2012 at 21:23
-
1@drknexus the nice thing about using bits is that all connection problems are auto-resolved so even if you reboot the client (or server naturally) during a transfer it will auto-resume the transfer. I've had transfers run into patch windows and this came in real handy.– Jim BJul 20, 2012 at 12:44
Use something like rar to create an archive that can be broken up into chunks, say 100MB. Use xcopy or robocopy to copy the files across.
robocopy source dest *.rar /w:30 /r:30 /z
will wait 30 seconds when the connection drops and retry 30 times.
I've used that when I need to copy a multi-gig file across a link that won't stay up long enough to do the copy.
curl has several retry options (along with being able to specify infinite retries).
rsync doesn't have any built in retry schemes, you could wrap it in a script to do infinite retries until it completed successfully.
answered Jun 16, 2009 at 23:29
FTP client with queue function (with auto retries). Try SmartFTP or FileZilla.
-
-
FileZilla is an FTP client, so you'll need a FTP server on one of the servers to use it.– pauskaJun 17, 2009 at 3:42
My friend suggested I use KillCopy for this purpose. It works well - its user interface is a bit clunky, but it's a drag-and-drop solution (assuming you're using a destination you can access in Explorer).
answered Jun 17, 2009 at 1:27
How short are the network outages?
If they are not long, then OpenVPN might be useful. I use it in various places for other reasons, but when using it over a slow unreliably GPRS connection on a moving train I found it did a good job of surviving the regular connection drops and, if the outage was not long enough to cause timeouts, the applications (web browser, mainly) transferring stuff over the link survived too.
This in combination with a client that supports partial sends and retrys, as suggested by others, might cover all the bases for you.
If using rsync, make sure you have the options set such that it can reuse partial transfers otherwise you may waste much bandwidth if the connection drops just before the end of synchronising a large new (or much changed) file.
It might seem like a strange suggestion, but use uTorrent. Instructions on how to do it are here:
http://www.poromenos.org/tutorials/bittorrent
It is great with lousy connections. I am assuming you can start the download from the other side. If not, then this idea won't work.
-
That link no longer works but I'm intrigued with using bittorrent to copy large files in a private capacity– Nick.McJun 23, 2020 at 9:06
If you're on Windows you may want to take a look at RichCopy a 'successor' to RoboCopy mentioned previously.
Except copying files is not always that simple. What if you're copying thousands of files across a slow connection? What happens if your network hiccups and interrupts the copy? What if you want to make sure that you preserve particular file attributes, such as a Last Modified date, but not other attributes, like security descriptors? What if you want to filter the files you're copying from source to destination based on filename or extension? If any of these apply to you, the simple Ctrl+C/Ctrl+V combination probably won't cut it. And though Robocopy GUI certainly improves on that basic functionality, it still has some limitations in terms of granular control and usability. What you need is RichCopy.
