Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Background/Environment Architecture:

My current environment for $corp_overlords$ is set up in a hub-and-spoke model with a technologically well-endowed home office hub (SAN, bladecenter/bladesystem ESXi cluster, fiber internet connection, etc.) connected to a number of remote site spokes that are not so well off, and typically contain single ESXi host server and connect to the home office hub via a T1. All traffic originating at any remote site routes back to the home office over a "MPLS network" (which is really just a T1 connecting the remote site to the home office).

At the home office, on the SAN, we have a number of VM templates that I have created to deploy VMs from. They are stored in an NFS volume, that is a vSphere datastore, attached to the home office datacenter object within vSphere.

Each remote site has a corresponding vSphere datacenter object, containing a datastore object that's connected to the locally-attached storage on the ESXi host server physically located at the remote site.

As these VM templates exist on the NFS volume, they occupy ~ 40 GiB (thin-provisioning). As files on NTFS (or a Linux FS), they occupy ~100 GiB.

Question:

How should I copy this 40 GiB of thin-provisioned data (that occupies 100 GiB of filesystem space) between my sites?

I am under the constraints that I have approximately 5 days to do so, and cannot interfere (noticeably) with "normal network traffic."

share|improve this question
1  
Please note that while I answered this question, I'm still looking for answers. Ideally an Option 4 answer for a bounty award, but I'd be happy with, and upvotey towards any good answer that uses options 1, 2 or 3. –  HopelessN00b Jan 29 '14 at 19:42
    
You have a bladecentre at home?! –  Tom O'Connor Jan 29 '14 at 20:50
    
@TomO'Connor Heh. Not my home office, but the corporation's "home office" site. Although, I'm sure if I asked nicely, I could haul the old EVA SAN and HP Bladesystem away for my personal use... expect that I don't have the ~$25,000 it would cost me to run the things at home. –  HopelessN00b Jan 29 '14 at 20:54
    
Ohhh. That makes more sense.. just –  Tom O'Connor Jan 29 '14 at 21:22

6 Answers 6

up vote 10 down vote accepted
+100

How about using ovftool to copy the templates directly between hosts?

I have used this for VMs before, and it works pretty well. Not sure if that also works for templates, but if not then you can just covert the templates temporarily to VMs for copying them.

Instructions, with an example are here.

You could also use ovftool to convert your templates to .ovf packages, which should be very compact, and then transfer the packages between datacenters with BITS or FTP or SCP or whatever protocol you want.

share|improve this answer
    
Nice option!! I forget about the cli tools often. –  ewwhite Jan 30 '14 at 19:31
    
I edited your answer, and added the last sentence there, as it's what I actually ended up doing. Converting the templates into .ovf packages made them several GB each, which I was able to easily transfer between sites with BITS. –  HopelessN00b May 21 '14 at 12:44

Options:

The way I see it, I have three possible approaches, though I dearly hope I'm missing a better one that someone here can point me at. (Ideally one that has me only moving the 40 GiB of actual data, and in a resumable, "background" or speed-throttled method.)

  1. Copy the files between datastores through the vSphere client.
    • Advantage: Moving only ~40 GiB, not ~100 GiB.
    • Disadvantage: Everything else - not resumable, not background/speed-throttled, interface SUCKS.

  2. Copy the file between Windows guests using BITS
    • Advantage: Resumable, background transfer.
    • Disadvantage: Moving ~60 GiB of data that doesn't really exist.
    • Bonus: Uses PowerShell. <3
    • Double Secret Probation Bonus: PowerShell Remoting makes it possible to do this in one single command.

  3. Copy the file between ESXi hosts via SCP
    • Advantage: Speed-throttled and potentially resumable.
    • Disadvantage: Moving ~60 GiB of data that doesn't really exist. Not background transfer.
    • Bonus: Neck beard. Extra neck-beard for resumability.

  4. Better option suggested on Server Fault.
    • Advantage: Resumable, speed-throttled background transfer that only moves ~40 GiB of data that exists.
    • Disadvantage: Awarding a bounty costs rep.
    • Bonus: Learn something new, justify playing ServerFault at work.
share|improve this answer
    
What about shrinking the datastore with powerCLI then using BITS to move the file? Obviously try this with a clone first. –  Nathan C Jan 29 '14 at 20:21
    
@NathanC Not a bad thought, but the datatstores on the home office SAN are actually 2TB NFS volumes that contain more than just the templates in question. We also lack free space on the SAN, so we can't allocate an additional NFS volume to create a new datastore for this purpose (or transfer things around to end up with one datastore containing only what we need to copy). –  HopelessN00b Jan 29 '14 at 20:37
    
Er, oops...wrong term. The shrink happens on the volume, not the datastore. I need a drink, clearly. –  Nathan C Jan 29 '14 at 20:39
1  
Option 5. Copy the templates to removable storage and ship it to the remote sites. –  joeqwerty Jan 29 '14 at 20:44
    
@joeqwerty Yeah, sneakernet's always an option. Maybe not for this, for non-technical reasons, but that doesn't mean it isn't a good answer for the general case. (I've been expecting someone to put up FedEx/UPS/USPS as an answer to this at some point.) –  HopelessN00b Jan 29 '14 at 20:48

Here's a somewhat interesting idea for you. It won't help with your initial seeding, but I wonder if using something like Crashplan's free product would help you with your templates.

https://www.code42.com/store/

It does dedupe and block level differentials, so you could install it on one local server there at HQ as the "seeder", and on each spoke server (in a VM I guess) as a "receiver". Setup the backups to only include the folder where the templates will be stored on the HQ Server. It can also backup to multiple destinations (such as each "spoke") https://support.code42.com/CrashPlan/Latest/Getting_Started/Choosing_Destinations

The steps (after setting up the Crashplan app on each side) would work something like:

  1. Copy the templates from the datastore(s) to the "seed" server to the directory on it that Crashplan is monitoring. On a gigabit network this might take a little time but shouldn't be too bad.
  2. Crashplan should monitor and start backing up the files to the spokes/receivers. This will obviously take quite a while.
  3. After the initial seeding/backups, when future templates change copy them from the actual datastore(s) to the "seed" server's directory Crashplan is monitoring, overwriting the original template copy. Then Crashplan will dedupe and only replcate the block level changes across to the spokes.

Just an idea...might be an interesting road to venture down and see if it works as a poor man's dedupe/block level replication for just these files.

share|improve this answer

I've done this type of move a number of ways, but given what you've described...

FedEx or UPS, with a twist...

I know that the servers in use are HP ProLiant and Dell PowerEdge servers. VMware does not have good support for removable devices (e.g. USB) as datastore targets. However, using a single drive RAID 0 logical drive (in HP-speak) at the main site can work. You can add and remove locally-attached disks on HP and Dell systems and use that as a means to transport datastores.

Being templates, you can move/copy them to your local disk via vCenter. Ship the disks. Insert into the receiving standalone server. The array and datastore will be recognized via a storage system rescan. Copy data. Profit.

I've also used this as a means to seed copies for vSphere replication, as 24 hours of deltas is a lot easier to manage than multiple full syncs.

share|improve this answer

This is a method I use fairly often for this kind of scenario. It seems counter-intuitive because you are uploading files from inside a VM stored on the datastore, to the datastore itself. However, this gives you a lot more control over how the transfer is accomplished.

  • Use WinRAR or 7Zip to break your template into 1GB-2GB chunks.
  • Create a VM on the ESXi server at each remote site. Minimal resources are needed, this is just a staging area.
  • Attach a VMDK to each of these VMs that's big enough to hold the data you're transferring.
  • Install an OS and transfer tool of your choice (I use an SFTP server for this).
  • Upload the RAR'd template to the staging VM.
  • Uncompress the RAR'd template.
  • Use vSphere or web UI to upload the template from the staging VM to the ESXI datastore. (this will be a FAST transfer).

Pros:

By breaking the template into smaller pieces you reduce the risk of data corruption during transfer. (If a file gets corrupted, you only need to re-upload that piece of the RAR, rather than the entire 40GB file.)

You only transfer 40GB (probably less as RAR'ing will compress further).

You get your pick of transfer utilities as you're doing the transfer inside the OS of your choice.

Cons:

You have to create a staging VM. I make this easier by having a pre-created template that is <1GB that has just a bare OS install + SFTP server.

Compressing/decompressing a 40GB template will take ~4-6 hours depending on your CPU resources.

share|improve this answer

I've dealt with this same issue quite a few times and about half the time I find that I'm far better off to just to build new machines in the remote location. This is especially true for what I call "template" machines. My version of that is a pretty basic machine. Your version may be something a little different.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.