2

The source server is Windows Server 2003 (a VM hosted in Xen). Target is Windows Server 2016.

They're in different locations and the connection is roughly 30Mbit/s and they're both on the same vpn.

I need to transfer files from one to the other.

File sizes range from a few MB all the way to a few 25 GB outlook pst's.

Compressing files on target to speed things up (and have the ability to verify the copy) is out of the question (at least for the huge files), as I only have 2GB of space left in the source server.

I need something that can compress the data on the fly, transfer, and then have it uncompressed on the target server, which would speed things up greatly, and I also need to be able to verify the integrity on the files in the target server.

This is a one-time thing, as the source server will be decommissioned once I'm done.

Does anyone have suggestions?

Thanks!

Alex

Edit 2019-10-06: I didn't know a question could be down-voted, and the one downvote I got states (when hovering the mouse on the count) that no research seems to have gone into the question. On the contrary, I have spent days looking for viable alternatives to simply mapping a drive and doing a windows copy (which I also tried). Hence my question.

Edit 2019-10-06.2: I ran a few tests today:

  • The large files (over 1GB amount to about 160GB).
  • FTP server.Transfer uses about half the bandwidth I have available (maxes at 1.5MBytes/s without compression. Desisted on compression (mode z) because the overhead made the transfer slower. FTP would be more practical for the smaller files, as clients make the process easier.
  • Justbeamit.com seems to use all my bandwidth, yielding speeds of about 3MBytes/s, which is the direction I'll probably follow with the 15 very large .pst files I have to deal with. Have to do it one by one, but for such a small number of files it's not a big deal. SHA-256 tests on source and copy of test file confirm that transfer is reliable. (Justbeamit.com allows sending multiple files but I'm hard-pressed to let it zip 160GB worth of data.) - from https://superuser.com/questions/314325/direct-transfer-of-a-file-between-two-computers-on-the-internet
Improve this question
6
  • Resilio Sync. Not sure it does compression though.
    – Tim
    Oct 6, 2019 at 3:36
  • rsync (in Linux) has the option '-z' to compress on the fly, and decompress after copying. Either you can mount your drives in Linux servers or you can try the rsync version for windows (I'm not windows user, so I don't know if it will work in the same way).
    – lepe
    Oct 6, 2019 at 4:10
  • 2GB left is a pretty tight fit, I thought I would mention that I have found WinDirStat to be invaluable for rooting out various space wasters. windirstat.net
    – Rob Pearson
    Oct 6, 2019 at 5:33
  • It is thanks to WinDirStat that I know the file sizes range I'm dealing with.
    – alex_223
    Oct 6, 2019 at 14:35
  • Hi Tim, I eyed resilio as a possibility, but even the oldest version I could find (still called btsync) won't install on Windows Server 2003.
    – alex_223
    Oct 6, 2019 at 15:10

3 Answers 3

Reset to default
3

Robocopy is good for moving a lot of data. Command help here - https://ss64.com/nt/robocopy.html

Improve this answer
2
  • 2
    Robocopy was unbearably slow, but thanks!
    – alex_223
    Oct 6, 2019 at 20:04
  • Yeah i prefer to use it more when backing up things than massive moves, but it does the job
    – Edwrd_T_Justice
    Oct 8, 2019 at 3:56
3

Is downtime acceptable at all? Sneakernet is still a thing, and depending on the physical distance it may provide enormous bandwidth.

For machines with limited allowed downtime it might still be an idea to try creating an initial seed to a locally attached drive using robocopy, then send the drive to the second data center and follow-up with a second robocopy job across the network to catch the differences.

Improve this answer
1
  • Sneakernet is definitely in the realm of possibilities. Especially because the VM's used to run a piece of industry-specific software that I haven't even started looking into how to migrate to a different vm. Downtime is limited to 3 days (Friday + weekend). The robocopy seed idea seems fantastic, except for how abysmally slow robocopy has been in my tests, but it might work with rsync+cygwin. Thank you!
    – alex_223
    Oct 7, 2019 at 3:51
0

There are plenty of both commercial and open source file transfer packages that optimize the bandwidth between two geographically distant end points. I work for FileCatalyst and that's what we specialize our software offering, to fully utilize your entire bandwidth between two end points over fast internet connections and over long distances.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.