Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I got a 10GBit Server with proxmox 2.1 installed. There's an OpenVZ container running with Ubuntu 10.04 64bit. I can get 200-300MB/sec with wget from data-center internal servers (-O /dev/null) which is okay. However if there are multiple connections, it's not possible to get more than 800MBit/sec in total.

I have an apache2 webserver running which works as a proxy and accesses the data from several other (not only data-center-internal servers). Its only proxying through, so the HDD don't get accessed (beside of the logfiles of course). As this server has two SSD's in it, it is very unlikely that this is the reason. As soon as 800MBit are reached, I only get 300kb/sec from data-center internal servers with "wget -O /dev/null"", even on the host system itself, not within the OpenVZ container.

I noticed that the process ksoftirqd/0 is requiring heavy CPU load (up to 100%):

 4 root      20   0     0    0    0 R   78  0.0 67:38.99 ksoftirqd/0 # (78% in this case)

Sometimes also the process events/0 requires many CPU power and it seems to compete with ksoftirqd/0 for the CPU power:

 35 root      20   0     0    0    0 R   50  0.0 18:04.63 events/0
  4 root      20   0     0    0    0 R   50  0.0 24:37.64 ksoftirqd/0

The output of top about the CPU power looks like this:

Cpu(s): 13.9%us,  7.3%sy,  0.0%ni, 64.0%id,  0.2%wa,   0.0%hi, 14.6%si,  0.0%st

As you can see, si is very high, which should normally stay on 0. Also the hard disk isn't the bottleneck. (0.2% wating).

I also got other 1-Gigabit-Servers with the same configuration. They even reach 900MBit/sec which is okay for a 1GBit server. However there's also a high load of ksoftirqd/0 (60-70%).

Currently there are about 300MBit going through the 10GBit server and the process ksoftirqd/0 stays at 0% CPU.

However later in the evening when the traffic increases, it will go up to 100 and cut the bandwith at 800MBit.

Maybe there's a problem with OpenVZ? I could also install ubuntu directly on the server without proxmox just for testing, but it would be better if I could stay with proxmox/OpenVZ.

However it's important that I'm able to use at least a few GBit with this server as soon as I need it. Of course also with multiple connections.

As far as I can read the output of lspci -vv, my card supports MSI-X.

The system has an Intel® Xeon® Processor E5606 (8M Cache, 2.13 GHz, 4.80 GT/s Intel® QPI) processor

This is my network card:

root@ns231828:~# lspci  | grep -i network
02:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
02:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
05:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
05:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)

I also uploaded the sanitized output of lshw here: http://jsfiddle.net/R8QgL/

This is also the content of lspci -vv: http://pastebin.com/7PqqbzHM

i have a "testing environment" with a little batch script that downloads big speed test files from a lot of servers (in fact i took just the ubuntu iso mirror list). i am able to reach 2 gbit in that environment.

after i tried ethtool -C eth2 rx-usecs 1022 i was able to reach 3.5 gigabit.

however with optimized rx-usecs settings (i tried the values 1 and 1000 as well, 1022 turned out best) in a live environment with real clients this only gained about 100 mbit.

i uploaded the other outputs of ethtool -k and ethtool -c here: http://pastebin.com/rEtmuV6i

Has anyone experienced similar issues or has a hint how I can address this problem?

share|improve this question
    
What are your current rx-usecs and InterruptThrottleRate settings? Perhaps you have things set to minimize latency rather than maximize throughput. –  David Schwartz May 6 '12 at 0:27
    
Have you tried tuning InterruptThrottleRate and related settings of the driver? –  rackandboneman May 6 '12 at 0:41
    
i am currently playing around with that but i am having issues working with the kernel modules since my provider seems to have static kernels that have some weird modules stuff going on (for example /proc/modules) doesnt exist.... but i am curreently installing another kernel –  The Shurrican May 6 '12 at 0:57
    
You can build a linux kernel that is completely unable to load modules .. –  rackandboneman May 6 '12 at 1:08
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.