15

Is it possible to span one huge Virtual Machine across several physical commodity servers?

Here is our use case:

  • We need to implement a 32-processor db server with 64 GB of RAM
  • We don't have a physical server of such capacity
  • We do have a lot of servers with smaller resources.

Is there a technology or (better) a product that lets us utilize these servers to create a VM with required capacity? Say, are we able to combine 8 physical 4-processor machines with 8 GB of RAM each into one 32-processor "logical unit" with 64 GB of RAM and set up an Oracle server that uses all this capacity?

Before posting this question, we read similar questions but didn't find an answer.

Maybe, someone could give us a hint now?

Improve this question
1
  • 2
    This is not an answer for your question, but It's feel strange that nobody advice about to look at the software limitations. If your company build apps for middle range business, it's seems to me obvious that the problem are software limitations probably the software architect and designers didn't think in a database with billions of records or with huge temp tables or procedures, think on that and create some self test and error reporting for the slow queries for my it's the way to solve the problem... think on the 3.3GB limit in the x86
    – user131482
    Commented Aug 8, 2012 at 21:33

4 Answers 4

Reset to default
8

There is a commercial product from ScaleMP called vSMP. It allows you to aggregate multiple x86 systems into a single virtual instance. I've never personally tried this before though, but I've been through a presentation from them. If I remember correctly, there are specific requirements for this to work, and you'll need to get some additional hardware (Infiniband for fast, low latency interconnects). It might cost a pretty penny too!

Improve this answer
3
  • 1
    ScaleMP doesn't emulate an x86 environment. You'll never get Windows or any other standard x86 OS to run in the virtual environment. You The only flavors supported are various versions of Linux built on SMP type architecture. And of that type of architecture... there are SEVERAL flavors. Even free ones.
    – TheCompWiz
    Commented Sep 20, 2010 at 13:02
  • OP was not specific about the other requirements. I only answered what I could gather from his/her post.
    – ryanlim
    Commented Sep 20, 2010 at 15:26
  • 1
    This looks bloody cool. I suspect that a 32 core box (possible with 2x 16 Core AMD chips) might be cheaper than a cluster with Infiniband, but there we go. This solution earns more bragging rights.
    – Tom O'Connor
    Commented Aug 8, 2012 at 21:53
9

There is no way to get the exact same functionality as a single 32-processor machine... with several separate servers. Your best bet is to look at clustering or grid computing. Done right, you can end up with comparable performance... and a higher level of high-availability. A lot of your question also depends on your "db" type. Microsoft SQL Server works significantly differently than MySQL or Oracle... and the scalability is also done completely differently.

Alternatively... you may want to consider letting someone do the database for you... like using the EC2 RDS...

Sadly, there is no way to combine several physical servers together & slap vmware on them and end up with a singular uber-powerful virtual server.

Improve this answer
10
  • TheCompWiz, thanks for the answer. OK, if the answer depends on my db type, let it be Oracle or Microsoft SQL Server. With these corrections, is it still impossible? Yes, we know about EC2 but we need exactly Oracle or Microsoft SQL Server to test issues with a software product we deliver for a customer...
    – user54614
    Commented Sep 17, 2010 at 20:01
  • Also, why only vmware to take into account? We don't mind any other hypervisors...
    – user54614
    Commented Sep 17, 2010 at 20:03
  • The ability to traverse multiple servers is a HUGE logistics nightmare... not to mention the lack of available bandwidth between devices. Think about how fast a CPU is... then all of the things you would have to do that would slow down the process... i.e. CPU -> bus -> PCI-bridge -> network card -> ethernet cable -> network stack -> ... even before it's reached the other server? You wouldn't want to wait 1 second to be able to add 1+1. Clusters typically are able to do this because tasks are assigned in "Jobs" and a job is issued to a compute node which does all the tasks in that job...
    – TheCompWiz
    Commented Sep 17, 2010 at 20:20
  • ... and then sends the answers back to the management node. Windows doesn't. There is no way to setup a virtual X86 environment (or X86_64) that would even attempt doing this.
    – TheCompWiz
    Commented Sep 17, 2010 at 20:23
  • 2
    @user54614 - You're absolutely not going to be able to replicate their scenario by strapping machines together. I would suggest talking to both your client and Oracle support to pinpoint and identify the problems.
    – Chris Thorpe
    Commented Sep 18, 2010 at 1:38
-1

"TheCompWiz" answered your question usefully.

I'd still like to say that, yes, you could build a hypervisor which allowed a single VM to span several physical hosts and it could run that VM "correctly" where everything functioned.

But, even with really good, high-speed networks between the physical hosts, the performance of such a thing would be truly awful, running much more slowly than a smaller VM that fit within a single one of those hosts. You'd have to simulate the cache coherency properties of a single VM by intercepting every single memory read or write that the guest OS and application did, which would multiply the cost of memory access by thousands, if not millions.

So no commercial hypervisor vendor enables such a thing. It's been tried in the lab. Nobody has bothered to make a product out of it.

To underscore the point one more time, look toward clustering for a solution.

Improve this answer
8
  • But what if a software product we deliver works fine for most customers but fails to work properly on the environment of a huge customer which runs our application on an 32-proc Oracle server with 64 GB of RAM. We want to reproduce this failure in our environment.
    – user54614
    Commented Sep 17, 2010 at 20:37
  • 2
    I know nothing of your software, but what's happening at 32-processors and 64GB RAM that's not happening at 2-processors and 8 GB RAM? If there's truly something repeatably wrong at that level, then it's an Oracle/OS/driver/IO/hardware issue.
    – gravyface
    Commented Sep 17, 2010 at 20:46
  • You will never get a hypervisor to traverse physical machines. They're still confined within the physical core of the machine. That being said... I bet you could build a mainframe type architecture similar to those of archaic behemoths from long ago... but you'd never get anything x86 running on it.
    – TheCompWiz
    Commented Sep 17, 2010 at 21:00
  • 1
    Your huge customer should have a second QA instance of that monster database server. If they don't have that available, it truly is their problem. In 15 years of IT work, I have never seen anybody expect a software vendor to duplicate their infrastructure (unless it is part of a service contract specifying exactly that, and the customer pays for it). Especially when that infrastructure is esoteric (although a 32-core 64-GB server can be had for about $22K from Dell these days).
    – rmalayter
    Commented Sep 17, 2010 at 22:03
  • What about ScaleMP (scalemp.com)?
    – user54614
    Commented Sep 18, 2010 at 13:54
-2

VMWare does. It is called DRS or Distributed Resources Scheduler. It allows you to combine the resources of 16 servers. You can then distribute that total to one or more virtual environments.

Improve this answer
1
  • 1
    No, this is not at all what DRS does. DRS lets one automatically vMotion machines around the cluster to even load out among the host nodes. It does not in any way give access to multiple hosts from one single VM.
    – EEAA
    Commented Jul 25, 2017 at 2:40

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .