Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I was just wondering why there is a need to go through all the trouble of creating distributed systems for massive parallel processing when, we could just create individual machines that support hundreds or thousands of cores/CPUs (or even GPGPUs) per machine?

So basically, why should you do parallel processing over a network of machines when it can rather be done at much lower cost and much more reliably on 1 machine that supports numerous cores?

share|improve this question

closed as not constructive by Zoredache, Scott Pack, Tim Brigham, Wesley, SvW Apr 4 '12 at 0:02

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.If this question can be reworded to fit the rules in the help center, please edit the question.

1  
Why do you believe it would be cheaper to have one huge bespoken machine then it be to just grabbing a bunch of commodity equipment and making it all work together? –  Zoredache Apr 3 '12 at 23:51
1  
And what do you do if you need two of those 1000 core boxes? Or god forbid 4 or 8?!?. –  pfo Apr 3 '12 at 23:54
    
In any case any computer system with thousands of CPUs is going to take some serious engineering, and will almost certainly will be designed for some specific processing task. The specific details for that task will be what matters. There simply is no generic answer that could possibly apply to a system like that. –  Zoredache Apr 3 '12 at 23:59

2 Answers 2

So basically, why should you do parallel processing over a network of machines when it can rather be done at much lower cost and much more reliably on 1 machine that supports numerous cores?

You should do parallel processing over a network of machines when it can't be done at much lower cost and more reliability on one machine that support numerous cores.

There quickly comes a point when multiple processors, many more than a single motherboard can handle, is necessary to handle a computation. Beyond that, there is a sweet spot of power consumption to processing gain that is often found in slower processors. A bank of Xeon 7500 processors uses an enormous amount of power, whereas lower performing CPUs might be able to perform half the processing at 1/3rd the power and thus scale much better.

There's also the concept of single point of failure. Would you want a single device to cause a computation to cease being computed at all? The alternative, having multiple nodes and even multiple datacenters, is much better to handle disaster.

share|improve this answer

Cost: Thousand of cores on one server compared to several low-end x86 servers.

Reliability: one server instead of many servers.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.