Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm doing HTTP load testing benchmarks (using Apache Benchmark and Siege) on a small Java EE 1.7.0 / Tomcat 7.0.26 application running on a Debian Squeeze 6.0.4 x64 virtualized with Virtualbox 4.1.8. The computer host is Ubuntu 11.10 x64.

I've modified those parameters in the Tomcat server.xml :

<Connector 
    port="8080" 
    protocol="HTTP/1.1"
    connectionTimeout="200000"
    redirectPort="8443" 
    acceptCount="2000"
    maxThreads="150"
    minSpareThreads="50" />

The application executed on the server takes around 300ms.

This app is running well until a certain amount of concurrent connections like those one :

ab -n 500 -c 150 http://xx.xx.xx.xx:8080/myapp/
ab -n 1000 -c 50 http://xx.xx.xx.xx:8080/myapp/
siege -b -c 100 -r 20 http://xx.xx.xx.xx:8080/myapp/ 

A lot of socket connection timed out happens and this completly overload the host processor (but the CPU load inside the VM is normal).

Doing an htop on the host, i can see that the Virtualbox processus is running under 300% CPU and never come down even after the load test is finished. (I've allocated 4 processors to the VM, if I allocate only one processor, CPU load goes under 100%).

Restarting Tomcat don't do anything, i'm forced to restart the whole VM.

I've tryed to launch those ab/siege commands locally on the VM and everything goes well.

I first thought it was related to a linux network limit as explained here: Running some benchmarks using ab, and tomcat starts to really slow down So I've modified those TCP parameters :

echo 15 > /proc/sys/net/ipv4/tcp_fin_timeout
echo 30 > /proc/sys/net/ipv4/tcp_keepalive_intvl
echo 1 > /proc/sys/net/ipv4/tcp_tw_recycle
echo 1 > /proc/sys/net/ipv4/tcp_tw_reuse

It seems to be better, but it continues to overload the host CPU and output socket connections time out at a certain amount of concurrent connections.

I'm wondering if this is not related to how Virtualbox handles external concurrent connections.

share|improve this question
    
Have you installed the virtualbox guest additions? –  Nils Mar 4 '12 at 21:06
    
Yes, guest additions are installed. –  aschuler Mar 4 '12 at 21:35
    
So how did you set up the network for the VM? NAT, Bridge, ...? –  Nils Mar 4 '12 at 21:42
    
Bridge on wlan0. –  aschuler Mar 4 '12 at 21:50
    
w like "Wireless"? Could that be the limiting factor here? –  Nils Mar 6 '12 at 15:09
show 1 more comment

3 Answers 3

I've seen this problem before when you have a loop that causes your network traffic to be re-sent forever. Try checking to see if you're getting a absurdly high amount of traffic going through any of your network interfaces.

If that doesn't work, does the CPU usage stop when the guest OS is stopped, or do you need to shutdown the whole VM container?

share|improve this answer
add comment

First of all you don't say anything about the VBox host. How powerful is it? When I do VBox load testing I use a machine with 12 cores and 32G of RAM to run a virtual server with 2 to 8 cores and up to 12G of RAM. In otherwords, the underlying host is more powerful than the VBox guest so I am reasonably sure that the guest has dedicated cores and RAM.

Next, I tune both the host and the guest OSes to handle load. Things like the max open files, max sockets being opened, and various TCP/IP settings like buffers. These need to be set right to do load testing. If you graph your data with a range of loads, you will see it get suddenly much worse when you hit an OS limit. Fix this before you load test your app.

share|improve this answer
add comment

Are you testing from within the VM? Or from elsewhere? If its the VM try pointing to 127.0.0.1 instead for the testing.

ab -n 500 -c 150 http://127.0.0.1:8080/myapp/
share|improve this answer
    
I've tried both. As i said, local tests are successful. This is when I launch thoses tests outside the VM (from host or another network), CPU overload happens. –  aschuler Mar 8 '12 at 15:42
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.