I'm doing HTTP load testing benchmarks (using Apache Benchmark and Siege) on a small Java EE 1.7.0 / Tomcat 7.0.26 application running on a Debian Squeeze 6.0.4 x64 virtualized with Virtualbox 4.1.8. The computer host is Ubuntu 11.10 x64.
I've modified those parameters in the Tomcat
<Connector port="8080" protocol="HTTP/1.1" connectionTimeout="200000" redirectPort="8443" acceptCount="2000" maxThreads="150" minSpareThreads="50" />
The application executed on the server takes around 300ms.
This app is running well until a certain amount of concurrent connections like those one :
ab -n 500 -c 150 http://xx.xx.xx.xx:8080/myapp/ ab -n 1000 -c 50 http://xx.xx.xx.xx:8080/myapp/ siege -b -c 100 -r 20 http://xx.xx.xx.xx:8080/myapp/
A lot of
socket connection timed out happens and this completly overload the host processor (but the CPU load inside the VM is normal).
htop on the host, i can see that the Virtualbox processus is running under 300% CPU and never come down even after the load test is finished. (I've allocated 4 processors to the VM, if I allocate only one processor, CPU load goes under 100%).
Restarting Tomcat don't do anything, i'm forced to restart the whole VM.
I've tryed to launch those ab/siege commands locally on the VM and everything goes well.
I first thought it was related to a linux network limit as explained here: Running some benchmarks using ab, and tomcat starts to really slow down So I've modified those TCP parameters :
echo 15 > /proc/sys/net/ipv4/tcp_fin_timeout echo 30 > /proc/sys/net/ipv4/tcp_keepalive_intvl echo 1 > /proc/sys/net/ipv4/tcp_tw_recycle echo 1 > /proc/sys/net/ipv4/tcp_tw_reuse
It seems to be better, but it continues to overload the host CPU and output
socket connections time out at a certain amount of concurrent connections.
I'm wondering if this is not related to how Virtualbox handles external concurrent connections.