You can't do an upgrade from x86 to x64. You need to do a complete reinstall of the OS.
I assume that you have at least 2 DCs (you're a bit unclear here), so if that's the case, follow these steps:
- Take full backups of both DCs just in case and make sure both are Global Catalogs and run DNS
- Demote one of the DCs to a member server, moving any FSMO roles to the other server.
- Reinstall that server with 2008 R2.
- Run adprep /forest and adprep /domain as required to add 2008 R2 DCs to your domaun
- Run dcpromo on the new server and make it a DC. Also install DNS and make sure it's a GC.
- Wait for replication to happen, then demote the other DC, moving the FSMO roles to the other DC.
- Install the OS on the newly demoted server
- Run dcpromo on that server and make it a RODC using the Advanced button on the dcpromo wizard.
You should only need to touch your backups in the result of a disaster. Replication will take care of making sure that the AD database has the right info on each server if you follow these steps.
If the DC is a virtual machine, would taking a copy of the .VHD file be helpful?
Not really. If you want to restore a DC, you need to put it DSRM (Directory Services Restore Mode) and restore the system state. This is the only way to do a complete roll back of an AD environment from backups. You should never restore one from a snapshot or bring an old .VHD online in a multi-DC environment. Just take full system backups (with windows backup if you don't have a better tool) and follow my instructions and you should be fine.