Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I was attempting to migrate 2 virtual machines that are domain controllers between 2 datacentres running ESX 3.5 and ESX 4.1. I was advised to shut down both domain controller at the same time during the migration process. This is to avoid USN Rollback and other replication issues.

The following are the steps that I was planning to perform: 1. Shutdown both DC. 2. Copy both VMs files across to new datacentre using Veeam FastSCP (connection to both vCentre through IP address instead of hostname) 3. Power them up at new datacentre. 4. Configure Network interface/DNS/DHCP for both DCs in new datacentre

I have tried to use Veeam FastSCP rather than VMware Standalone Converter is because its copying rather than converting. Someone also suggested that I use backup and restore app like Veeam backup and replication software.

Sounds like a simple job, but after shutting down both DCs, the transfer rate using FastSCP is so slow registering only 1KB/s as oppose to the normal 1MB/s (or more).

When that attempt to transfer failed, I tried to cold clone both DCs resulted in the both ESX hosts get disconnected. I have tried troubleshooting by referring to this - VMware KB - Diagnosing an ESX Server that is Disconnected or Not Responding in VirtualCenter

It seems that the DNS being down is the caused of all unusual occurrence. The moment I powered up the DCs via VMware console command, the ESX host were able to connect to the vCentre again.

How can I avoid such a pitfall again? Am I doing it correctly? Any help would be greatly appreciated! Thank you.

share|improve this question
up vote 2 down vote accepted

I don't see any reason to shut down both DC's at once. USN rollback is only going to be an issue if you restore a point in time copy\clone\snapshot of a specific DC after the source machine has been allowed to continue on. For your migration, shut down a DC and move it\copy it using whatever mechanism works best then restart it on the new site. At that point make 100% sure that the source VM can't be restarted as that will cause USN rollback. Once you have one DC moved successfully, repeat the process for the other(s).

Provided both sites are active and can see each other you can migrate the DC's individually while still keeping at least one active at all times, which you will need to do if the migrations take any length of time at all. As you have found out with the vCenter issues that arise when your DNS goes down you can't do much in an AD managed environment when you take all AD services down.

If both sites are completely independent of each other - so that when a DC is migrated to the new site it cannot communicate with the original site - then you will have to be much more careful and both DC's will have to be shut down for some period of time to ensure USN rollback doesn't happen. You can migrate the first DC you want to move while keeping the other one active but you must keep that DC powered off on the original site until you have migrated the second DC. If you don't then there is a risk of USN rollback.

To get around this I'd suggest the following. Make a clone of each DC on the source side using vCenter. Be very careful with these as you do not want to power them up with network connectivity on the source site.

  1. Power off one DC (DC01) and clone it (DC01-clone). Keep it powered off for now.
  2. Clone the second DC (DC02) while it is powered on so you now also have DC02-clone. Ideally you should have a third DC that you can keep powered on while shutting this one down but I'm assuming you don't have one and don't want to build one.
  3. You can now power the first source DC VM back on (DC01).
  4. Disable the network adapters on these new clone VM's (DC01-clone & DC02-clone) immediately just to be certain.
  5. Migrate these using whatever mechanism you want to use.
  6. Re-enable the network interfaces when you have them migrated to the new site and power them up.
  7. Delete the clones on the original site.
share|improve this answer
Thanks Helvick! I just need to clarify with the sequence of events. You have mentioned that "At that point make 100% sure that the source VM can't be restarted as that will cause USN rollback." So if DC01 is migrated 1st, it should remain shutdown before DC02 is powered off so that there is no chance of them replicating? After DC02 is powered off, I can then power on the source DC01 as I need at least 1 DNS server to be running. Does it sounds right? – Imagineer Dec 31 '10 at 1:44
No - think about these as if they were physical systems you were moving between sites. Shut DC01 down on Site 1, migrate it to site 2, start it up, make sure its healthy and let it replicate. Once you know it's fine then repeat the procedure with DC02. That way you are never without AD services at any stage. The only risk is that since you are copying a VM to do this you must make 100% sure that the original copy of these DC's never gets restarted once you've started them up on the other site - that is what will cause USN rollback issues. – Helvick Dec 31 '10 at 1:56
The 2 datacentre doesn't talk to each other - so source DC01 in old datacentre and cloned DC01 in new datacentre will not be talking to each other and I'm going to leave source DC01 in old datacentre running until its decommissioned. I'm not particular about user account not in sync. I still need DC01 to be running in old datacentre after it has been shutdown for migration. Will it still cause USN rollback issues? – Imagineer Dec 31 '10 at 2:56
Ah that explains the original statement that you will have to shut down both DC's. I've expanded my answer to deal with your scenario, having no communication between the datacenters does complicate things if you want to keep the original running. – Helvick Dec 31 '10 at 11:06

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.