-3

I was just wondering why there is a need to go through all the trouble of creating distributed systems for massive parallel processing when, we could just create individual machines that support hundreds or thousands of cores/CPUs (or even GPGPUs) per machine?

So basically, why should you do parallel processing over a network of machines when it can rather be done at much lower cost and much more reliably on 1 machine that supports numerous cores?

3
  • 1
    Why do you believe it would be cheaper to have one huge bespoken machine then it be to just grabbing a bunch of commodity equipment and making it all work together?
    – Zoredache
    Apr 3, 2012 at 23:51
  • 1
    And what do you do if you need two of those 1000 core boxes? Or god forbid 4 or 8?!?.
    – pfo
    Apr 3, 2012 at 23:54
  • In any case any computer system with thousands of CPUs is going to take some serious engineering, and will almost certainly will be designed for some specific processing task. The specific details for that task will be what matters. There simply is no generic answer that could possibly apply to a system like that.
    – Zoredache
    Apr 3, 2012 at 23:59

2 Answers 2

3

So basically, why should you do parallel processing over a network of machines when it can rather be done at much lower cost and much more reliably on 1 machine that supports numerous cores?

You should do parallel processing over a network of machines when it can't be done at much lower cost and more reliability on one machine that support numerous cores.

There quickly comes a point when multiple processors, many more than a single motherboard can handle, is necessary to handle a computation. Beyond that, there is a sweet spot of power consumption to processing gain that is often found in slower processors. A bank of Xeon 7500 processors uses an enormous amount of power, whereas lower performing CPUs might be able to perform half the processing at 1/3rd the power and thus scale much better.

There's also the concept of single point of failure. Would you want a single device to cause a computation to cease being computed at all? The alternative, having multiple nodes and even multiple datacenters, is much better to handle disaster.

3

Cost: Thousand of cores on one server compared to several low-end x86 servers.

Reliability: one server instead of many servers.

Not the answer you're looking for? Browse other questions tagged .