up vote 0 down vote favorite
share [g+] share [fb]

Is it possible to span one huge Virtual Machine across several physical commodity servers?

Here is our user case: we need to implement a 32-processor db server with 64 GB of RAM. We don't have a physical server of such capacity but we do have a lot of servers with smaller resources.

Is there a technology or (better) product that lets us utilize these servers to create a VM with required capacity? Say, are we able to combine 8 physical 4-processor machines with 8 GB of RAM each into one 32-processor "logical unit" with 64 GB of RAM and set up an Oracle server that uses all this capacity?

Before posting this question, we read similar questions but didn't find an answer.

http://serverfault.com/questions/141953/combine-several-physical-servers-into-1-virtual-server http://serverfault.com/questions/137086/cloud-computing-multiple-physical-computers-one-logical-computer

Maybe, someone could give us a hint now?

Thanks,

link|improve this question
75% accept rate
feedback

3 Answers

up vote -1 down vote accepted

There is a commercial product from ScaleMP called vSMP. It allows you to aggregate multiple x86 systems into a single virtual instance. I've never personally tried this before though, but I've been through a presentation from them. If I remember correctly, there are specific requirements for this to work, and you'll need to get some additional hardware (Infiniband for fast, low latency interconnects). It might cost a pretty penny too!

link|improve this answer
ScaleMP doesn't emulate an x86 environment. You'll never get Windows or any other standard x86 OS to run in the virtual environment. You The only flavors supported are various versions of Linux built on SMP type architecture. And of that type of architecture... there are SEVERAL flavors. Even free ones. – TheCompWiz Sep 20 '10 at 13:02
OP was not specific about the other requirements. I only answered what I could gather from his/her post. – ryanlim Sep 20 '10 at 15:26
feedback
up vote 4 down vote

There is no way to get the exact same functionality as a single 32-processor machine... with several separate servers. Your best bet is to look at clustering or grid computing. Done right, you can end up with comparable performance... and a higher level of high-availability. A lot of your question also depends on your "db" type. Microsoft SQL Server works significantly differently than MySQL or Oracle... and the scalability is also done completely differently.

Alternatively... you may want to consider letting someone do the database for you... like using the EC2 RDS...

Sadly, there is no way to combine several physical servers together & slap vmware on them and end up with a singular uber-powerful virtual server.

link|improve this answer
TheCompWiz, thanks for the answer. OK, if the answer depends on my db type, let it be Oracle or Microsoft SQL Server. With these corrections, is it still impossible? Yes, we know about EC2 but we need exactly Oracle or Microsoft SQL Server to test issues with a software product we deliver for a customer... – user54614 Sep 17 '10 at 20:01
Also, why only vmware to take into account? We don't mind any other hypervisors... – user54614 Sep 17 '10 at 20:03
The ability to traverse multiple servers is a HUGE logistics nightmare... not to mention the lack of available bandwidth between devices. Think about how fast a CPU is... then all of the things you would have to do that would slow down the process... i.e. CPU -> bus -> PCI-bridge -> network card -> ethernet cable -> network stack -> ... even before it's reached the other server? You wouldn't want to wait 1 second to be able to add 1+1. Clusters typically are able to do this because tasks are assigned in "Jobs" and a job is issued to a compute node which does all the tasks in that job... – TheCompWiz Sep 17 '10 at 20:20
... and then sends the answers back to the management node. Windows doesn't. There is no way to setup a virtual X86 environment (or X86_64) that would even attempt doing this. – TheCompWiz Sep 17 '10 at 20:23
1  
@user54614 - You're absolutely not going to be able to replicate their scenario by strapping machines together. I would suggest talking to both your client and Oracle support to pinpoint and identify the problems. – Chris Thorpe Sep 18 '10 at 1:38
show 4 more comments
feedback
up vote 0 down vote

"TheCompWiz" answered your question usefully.

I'd still like to say that, yes, you could build a hypervisor which allowed a single VM to span several physical hosts and it could run that VM "correctly" where everything functioned.

But, even with really good, high-speed networks between the physical hosts, the performance of such a thing would be truly awful, running much more slowly than a smaller VM that fit within a single one of those hosts. You'd have to simulate the cache coherency properties of a single VM by intercepting every single memory read or write that the guest OS and application did, which would multiply the cost of memory access by thousands, if not millions.

So no commercial hypervisor vendor enables such a thing. It's been tried in the lab. Nobody has bothered to make a product out of it.

To underscore the point one more time, look toward clustering for a solution.

link|improve this answer
But what if a software product we deliver works fine for most customers but fails to work properly on the environment of a huge customer which runs our application on an 32-proc Oracle server with 64 GB of RAM. We want to reproduce this failure in our environment. – user54614 Sep 17 '10 at 20:37
1  
I know nothing of your software, but what's happening at 32-processors and 64GB RAM that's not happening at 2-processors and 8 GB RAM? If there's truly something repeatably wrong at that level, then it's an Oracle/OS/driver/IO/hardware issue. – gravyface Sep 17 '10 at 20:46
You will never get a hypervisor to traverse physical machines. They're still confined within the physical core of the machine. That being said... I bet you could build a mainframe type architecture similar to those of archaic behemoths from long ago... but you'd never get anything x86 running on it. – TheCompWiz Sep 17 '10 at 21:00
Your huge customer should have a second QA instance of that monster database server. If they don't have that available, it truly is their problem. In 15 years of IT work, I have never seen anybody expect a software vendor to duplicate their infrastructure (unless it is part of a service contract specifying exactly that, and the customer pays for it). Especially when that infrastructure is esoteric (although a 32-core 64-GB server can be had for about $22K from Dell these days). – rmalayter Sep 17 '10 at 22:03
What about ScaleMP (scalemp.com)? – user54614 Sep 18 '10 at 13:54
show 1 more comment
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.