Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

How would I go about doing a 100% data swap between two storage devices of identical size and keep both data partitions without using a third device as a temporary buffer?

In other words, I have two VERY large storage arrays each with two identically sized VERY LARGE volumes as such:

DRIVE 1: VOLUME A

DRIVE 2: VOLUME B

I want the data from volume a to go to volume b and the data from volume b to go to volume a. All data must be preserved. I have no means of obtaining a third device for temporary storage.

I figure that theoretically this is possible to program at the bit level. As long as there is some free space on both volumes (which there is), that free space can serve as temporary storage for swapping data. Worst case, I can just do this manually by doing a batch copy of directories. However, surely there must be software that does this at the bit level?

Unfortunately I have scoured the net and came up empty. All the software I see does CLONING of an old drive to a new one, but no direct bi-directional volume data swap.

Any ideas?

** Update **

Just to be clear - I realize that it's possible to move data around manually into multiple temporary locations and use the backup/restore process. That's what I'm trying to avoid having to do because of the time and effort involved in restoring the data from multiple locations and through multiple processes. There just isn't any other array large enough to serve as a temporary location for the entire set of data/files.

** SOLUTION **

I ended up manually swapping data back and forth since the data sets were modular. The Altap Salamander file manager has the ability to queue file transfers so I basically created a queue of file transfers back and forth with sizes calculated to the amount of free space that would be available on each drive after a set of files copied. Worked like a charm but way too time consuming vs just using a spare storage array to temporarily house an entire copy. Would be cool if there was software that automatically did this, but clearly it's one of those problems where extra hardware beats a software solution.

share|improve this question
    
Why? Maybe there is another way to accomplish what you want without having to move all of the data. –  psusi May 9 '12 at 2:51
    
Well, basically I have a production server and a development server. The development server has a much newer storage array that I would rather utilize on production. Thus I want to swap the hardware but keep the data on each respective server. Unfortunately the storage arrays are very large and the data is very bulky and I have no budget for a temporary storage location. –  Matias Nino May 9 '12 at 4:56
1  
I need you to clarify, are you wanting to merge the data or simply have a copy of A at B and B at A? Also what's the hardware/software being used, this last bit is seriously important. –  Chopper3 May 9 '12 at 8:28
    
Can you provide information about the size of the data and the equipment/software options at your disposal? Without that, this isn't much of a question. –  ewwhite May 10 '12 at 18:20
    
You now have multiple solutions. You now need to do both - the quick 'in-memory' swap solution with the fallback of restoring from backups when all the data gets hosed. You do have backups… right? –  MikeyB May 10 '12 at 18:24
add comment

7 Answers

up vote 5 down vote accepted

The only safe solution is to use additional hardware. Speak to your storage vendor about hiring something to help you get the job done. Trying to do it any other way will end in disaster and you'll be back here asking how to recover...

share|improve this answer
add comment

I would just write a program to open up both volumes in raw mode and allocate two large buffers in memory. Fill one buffer from volume A and one from volume B. Then write the buffer you read from volume A to volume B and vice-versa. Repeating until you've finished the swap.

Note that if you get interrupted anywhere in this process, it's going to be very ugly.

I can give you more specific suggestions if you mention your platform.

share|improve this answer
1  
To memory kind of scares me a little. I would just write it to disk instead of memory. He would need to dump a list of all files sorted by size, use a zip() like function, and then determine if he has a large enough capacity to store the largest file. Then write an application that will go down the list swapping the files back and forth using rsync or such. –  Joseph Kern May 9 '12 at 3:40
1  
@JosephKern: That's not a bad idea. It sounds a lot safer. –  David Schwartz May 9 '12 at 3:48
    
That would be badass to write, but I'm a Windows Server admin, which kinda limits my ability to casually write low-level utility programs. :) Nevertheless, if I see .NET code, I'm not one to shy away from it. –  Matias Nino May 9 '12 at 4:59
    
Also note that this will mean a longer downtime, especially if the storage arrays are really VERY large. Not sure if that is OK for a production system. –  Oliver May 9 '12 at 6:12
add comment

Copy the dev data to the prod hardware and then restore the prod data to the dev hardware from backups. Am I missing something?

share|improve this answer
    
Backups of the data are spread into multiple systems throughout the organization depending on priority and context. On top of that, the backup process is not very uniform. So I COULD do this, but the restore would take MUCH more time than a simple data swap. –  Matias Nino May 10 '12 at 17:33
    
I see, well if the simple data swap fails you'll at least get to test those restore processes thoroughly which would be a Good Thing (TM )^^ –  Oskar Duveborn May 13 '12 at 18:45
add comment

There are many ways to do this, however you need to prioritize based on safety. If you lose your production data, I imagine you'd be hosed, so let's focus on a method that secures that and reduces the amount of time it's unavailable for production.

  • Step one: back up your dev environment and turn it off.

  • Step two: unallocate the disks from your dev environment

  • Step three: allocate those disks to the prod environment (in such a way that the prod environment doesn't try to use the old filesystems)

  • Step four: migrate your prod to the new disks. You can use your OS to mirror the data between both volumes and the restart the services on the new hardware.

  • Step five: unallocate the old prod disks from the prod server, and allocate them to the dev server

  • Step six: recover your dev data onto the disks presented to the dev server.

edit: Restoring or using a scratch space are your only options, and I question your definition of this as a "prod server" if it's not backed up in a way that it can be recovered.

share|improve this answer
    
There is no bulk backup process for the entire array because it is so large. –  Matias Nino May 10 '12 at 17:40
1  
Your server, not the array. I assume you are keeping a tape backup of your server, regardless of the size? –  Basil May 10 '12 at 18:14
add comment

Are you sure you don't have a temporary storage? - I am sure you have backups of your systems. There you have plenty of temporary storage.

Why don't you use these backups to accomplish this? Restore the two latest backups onto the desired storage array and you are done.

share|improve this answer
    
There's plenty of temporary storage around. Just nothing large enough to temporarily house the ENTIRE set of data file on the array. I was hoping to have to avoid having to rely on the restore process, which is very fractional and requires manual attention to restore individual components. I would be nice if there was a program that could swap the data and be done with it. –  Matias Nino May 10 '12 at 17:44
    
Perhaps the focus would be better directed at automating the restore process or automating the deployment of environment configurations - if those two are such a bother you want to avoid them, I'd want to fix them before doing anything else at all, really. –  Oskar Duveborn May 13 '12 at 18:52
add comment

This really requires a hardware solution. I have a device that takes a pair of SATA drives and duplicates one to the other at the press of a button. No computer required.

Without such a hardware device there must be some form of intermediate buffering because the system doing the copying must read from one device before writing it to the other. However, this can be done very fast because the bottleneck is normally the I/O interfaces, not the intermediate system.

share|improve this answer
    
That would be a one way copy which the question explicitly states is not what he is looking for. –  psusi May 9 '12 at 15:33
    
@psusi, while the hardware device I have is certainly a one-way operation the software solution can be a two-way operation because it's not necessary to buffer an entire drive, only small chunks. –  John Gardeniers May 10 '12 at 3:40
    
That doesn't seem to be at all what you said in your answer. –  psusi May 10 '12 at 14:18
    
@psusi, while my answer did talk about the need for an intermediate buffer I dod not say it must be the size of the drive being copied. I've written copiers using buffers the size of a single logical sector. –  John Gardeniers May 11 '12 at 0:48
add comment

If you can't do the copy+restore from backup method Oskar suggested ( and if you don't have backups, shame on you ), then the nature of development systems makes this pretty simple and straight forward. Simply get your development work to the point where you publish it to the production server, and when development no longer has any delta from production, swap their roles, and then resume development on the other server.

share|improve this answer
    
See update note. Swapping dev and prod is not a time-effective option because of other related hardware and configuration info that is particular to each environment. –  Matias Nino May 10 '12 at 17:49
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.