I identified that data transfer within my cluster will be bottlenecked by the network interconnect (I'm saturating the dual SAS connection to my storage with sequential workloads), and I'm undecided between a 10 GbE or 40/56 Gb Infiniband to mitigate the problem.
I'm leaning towards using dual port 10 GbE NICs and link aggregation to increase the throughput between my servers. However, I've read that the throughput doesn't increase linearly with the number of links. What kind of throughput should I expect? If it depends on my working set, how do I go about estimating the throughput?