I've to do a migration of two servers with large SAN attachments to our new VMWare environment.

EDIT: I have to supply some additional intelligence as I have good answers regarding VMWare solution.

Ok so, I can't attach a previous EMC LUN on the New system due to some technologies limitations on the server.

I can't use VMWare Converter to clone the missing volumes on my new VM as VMWare Converter can't see EMC PowerPath Pseudo-devices and that the previous admin used those Pseudo-device to built LVM2 and/or ASM volumes on top of.

Those two physicals servers are attached to an old EMC² CX-340 SAN and handle 5TB of data.

Those 5TB of data are small PDF and I need to transfer them to the new machine through our 1Gbit/s LAN.

I've tried using rsync, but it's really to slow and have a strong impact on the RAM and CPU performance.

I've try using NC with TAR but the transfer rate is quite slow as I've an average throughput of about 50MB/s on a 1Gbit/s link with barely zero traffic.

Could you give me some advice or return of experience with this kind of migration and how you manage to have it finished correctly within reasonable amount of time?

  • 1
    If you indeed get an effective 50 Mega Byte per second (multiply by 8 to get Mega bit /s) that is already 40-50% of your maximum wirespeed so not too bad... Since PDF's are already compressed you can safely omit any compression options. With many files you also can run into some file-system overhead and simply copying the whole SAN LUN with dd might be more speedy. – HBruijn Apr 27 '15 at 9:59
  • No, indeed it was a Typo mistake, I've an average of 50Mbits/s on a 1Gbits/s link and so about 5MBytes/s transfer rate, which is really slow as I already achieved 720Mbits/s transfer on this link with another machine but with large files of around 300GBytes. – Dr I Apr 27 '15 at 10:02

If you really need a quick way to transfer files, and both systems are Linux-based, you can try UDR.

This is really a form of rsync-over-UDP (using the open-source UDT framework) and is particularly handy for moving large numbers of files or transferring over high-bandwidth or high-latency links. In addition, encryption is disabled by default, so the RAM/CPU hit is lower than traditional rsync. SSH is not involved either.

I can easily get wire-speed transfers over 1Gbps with 10-million small TIFF files in a directory tree.

Your syntax will be slightly modified from rsync. All rsync flags need to appear before the source/destination specification:

udr rsync -avP --stats --delete /data/ server2:/data/

Easy to build... You'll need g++ and openssl-devel:

git clone https://github.com/LabAdvComp/UDR.git
cd UDR/
cp src/udr /usr/local/bin/

Do that on the source and destination.

See: Possibility of WAN Optimization for SSH traffic

| improve this answer | |
  • Thanks a lot for your deep insight on this topic, but what is the difference between UDR-Rsync and a Netcat/Rsync or Netcat/TAR in UDP transfer? As I try on my relatively small test server, with NC/TAR I don't have encryption or other overhead (Except for archives but its not significant) and achieve the test on a 1TB set of data with around 45 Millions of PDF files, my average bandwith is... well... around 2 or 5 MBytes/s on a 1Gbps link. – Dr I Apr 27 '15 at 11:46
  • @DrI UDP is the real difference. Plus, UDT/UDR is an optimization tool purpose-built for data transfer acceleration. Netcat is just netcat. – ewwhite Apr 27 '15 at 11:52
  • Yeah that what I read on your optimization link, the "magic" is not coming from the UDP stack but rather from the UDT library which is something similar to the diagram optimizer provided by silverpeak on its WAN Optimizer solution. I'll give it a try after the sFTP test. – Dr I Apr 27 '15 at 11:57
  • 1
    SFTP will be slow for the same reasons rsync+SSH will be. – ewwhite Apr 27 '15 at 12:00
  • @ewwhite, Yep, you're right, It's exactly the same things as the pipe use SSH tunnel and checksum. Just tested it, I've got the exact same 5.5MBytes/s throughput. – Dr I Apr 27 '15 at 13:27

If used in daemon mode without encryption, rsync can efficiently transfer large amount of small files. Give it another try using it in daemon mode.

| improve this answer | |

Have you not thought of exposing the SAN LUNs directly to the new VMs - this generally works just fine and can be faster than copying the files into a VMDK - though it can 'lock' the VMs onto their initial host. But you could use this to get things going then migrate the files into a VMDK at your own pace - with rsync - and later cut the link to the original LUNs.

| improve this answer | |
  • Oups, I forgot to mention that I can't attach the old SAN LUNs to the new VMWare Infrastructure. But yeah, originally I would have done it just like that if I had the link. – Dr I Apr 27 '15 at 11:41

If the destination VMs aren't yet built, you might try using the free VMware Converter to copy the data over.

In fact, even if they are built, you could clone the disks to a dummy VM then attach them to existing VM once the clone is done.

In any event, the converter uses two methods to clone files from source to destination, the full details of which can be found here.

If the destination disks are configured to be smaller than the source, it will clone individual files into the new VM.

However, if the destination disks are setup to be equal or larger, it clones blocks. This would make the amount of files on disk pretty much irrelevant and it should run relatively quickly.

I doubt you'll fill a 1Gbps pipe, but you should get more than 50Mbps.

Just remember that you're still looking to move 5TB, so it's going to take some time.

| improve this answer | |
  • thank you very much for your answer, I've edited my question to improve the informations regarding my situation as I didn't provided enough insight earlier. Anyway, long story short, VMWare Converter can't clone/find/see all my attached disks as the previous admin used PowerPath. – Dr I Apr 27 '15 at 12:40
  • In theory, if the running OS can read the disks, so can the Converter Agent. Assuming of course you use hot clone mode, instead of cold clonning. – GregL Apr 27 '15 at 12:42
  • I use the hot clone mode, my VMC is not able to see the pseudo devices, I've found numbers of people facing the same issue with powerpath pseudo-devices which can't be cloned as /dev/emcpowerXY device is not detected by VMC regarding some obscure reason. But thanks a lot for the in deep KB post about VMC anyway. – Dr I Apr 27 '15 at 12:44
  • 1
    Hm, well then. I don't know what to say! – GregL Apr 27 '15 at 12:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.