Web server to DB server connection speed

Question

we are designing a cluster of 2xnginx, 2xapache, 2xmemcache, 2xmysql servers for a high traffic web site.

Current web site runs on 1 dedicated web server and 1 dedicated db server. the current web server sees a peak of ~60Mbps incoming traffic and 25Mbps outgoing traffic. the current db server sees a peak of ~60Mbps outgoing traffic and a peak of ~20Mbps incoming traffic.

For our new setup we were thinking of having 1Gbit connections between the various nodes but the provider supports 1Gbit only if we dont use KVM cards as they only run at 100Mbit.

Based on the figures above I have the feeling that 1Gbit connectivity will not give us any benefit and the above limitation is not a show stopper.

Can any seasoned web cluster admins confirm or advise against this?

Answer 1

To what level of granularity are you measuring your peak traffic? If your peak is 60Mbps measured at a 1 millisecond level, then extra bandwidth is unlikely to help. But if it's at a 5 minute level of granularity, then it's likely that you're experiencing a bit of clipping of transient peaks in network traffic. You'd need to do more detailed measurement to quantify the amount of clipping and its effect on site performance.