I'm looking at running a NoSQL cluster and the question of tcp/ip saturation came up. I'm wondering if a flat network neighborhood topology mightn't alleviate internode congestion. The idea is to have 3 or 4 NIC cards on each node & interconnect them using inexpensive switches. I suspect however this is easier said than done. Am I dreaming?
see also: http://cgi.aggregate.org/cgi-bin/fnncgi.cgi
The concern particularly arises with either many or poorly written federated results being returned to a controlling node for final aggregation.