46

Recognize that Ubuntu 14.04 uses Apache 2 with PHP running through an mpm_prefork module, of which an editable file is in /etc/apache2/mods-enabled/mpm_prefork.conf. Also, recognize that starting in Apache 2.4, MaxClients is now renamed as MaxRequestWorkers, and so any documentation regarding MaxClients needs to be switched to MaxRequestWorkers. Stop the ...


44

Processor advancements, clock speed and IPC calculations can make it almost impossible to try to reasonably compare decade old CPUs to modern ones. Not only are the instructions per cycle going to vary, but newer processors have instruction sets dedicated to complex calculations (Intel has added AES-NI as an example), clock speed is no longer a reasonable ...


32

I don't want to sound terrible by emphasizing something that should be obvious here, but you're comparing a high-end server processor from 2014 to a high-end server processor from 2007. I don't think this requires much more explanation. There's a reason an HP ProLiant DL580 G5 is available so inexpensively today. They were large, slow and lack many ...


24

somaxconn determines the maximum number of backlogged connections allowed for each TCP port on the system. Increasing it (recommended for servers) can prevent "connection refused" messages, but it can result in slow connections if the server can't handle the increased load. You can check the current backlog with netstat -ant | grep -c SYN_REC ...


23

1 x 10Gb link for performance. Otherwise if a single server needs to use 1.1Gbs to another server it can't because most load balancing systems use destination MAC or IP (Which would be the same). This also eliminates issues where links are busier then other links because of the same fact, if the hash works out to be on the same link they end up on the same ...


22

This great article helped me a lot... http://www.inmotionhosting.com/support/website/server-usage/view-level-of-traffic-with-apache-access-log I had created a set of prepered commands that I am using to analyze apache log: request per hour cat access.log | cut -d[ -f2 | cut -d] -f1 | awk -F: '{print $2":00"}' | sort -n | uniq -c request per hour by ...


19

Is it ever sane to use a Virtualized solution when performing I/O heavy workloads? Yep, very sane indeed, in fact for most organisations now virtual is the default and doing things on physical boxes is the very much the exception. We have over 100k VMs of all forms and many of them are >40k IOPS with no issue at all. What are the best practices around ...


15

Short answer: disable comp-lzo. I realize this is an old post, but I was also suffering from poor OpenVPN performance. I had tried everything, adjusting the MTU, changing the snd and rcv buffers, mss clamping, you name it. CPU load was negligible. On a whim, I disabled compression (removed comp-lzo from the client and the server) and performance increased ...


14

Taskset is for binding a process to one or more CPUs; essentially specifying where it can run at initial execution or while it's running. If using RHEL/CentOS on modern server equipment, numactl is recommended over taskset. Cpuset/cset is for CPU shielding and is a framework build around Linux cgroups. Cset was never popular on certain distributions (like ...


13

At first, the client requests will get queued, until there is a process/thread that gets free on the apache server. So, the clients will see a delay in loading the page. See the MaxClients parameter documentation for more information. When placed in the backlog queue, the client request can eventually time out on the client side. Then the user will see ...


11

Honestly, I wouldn't be using Ubuntu for this... but there are options that can be applied to any Linux variant. You'll want to increate your network stack buffers: net.core.rmem_default = 10000000 net.core.wmem_default = 10000000 net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 If the application is writing to disk, maybe a scheduler/elevator ...


11

I have a similar (albeit smaller) setup, with 12x 2TB disks in a RAID6 array, used for the very same purpose (rsnapshot backup server). First, it is perfectly normal for du -hs to take so much time on such a large, and used, filesystem. Moreover du accounts for hardlinks, which cause considerable and bursty CPU load in addition to the obvious IO load. Your ...


10

Is it ever sane to use a Virtualized solution when performing I/O heavy workloads? Does a database server regularly pulling 1gb/second random IO count? Have one here. Or a virtual file server delivering up to 600mb/second to a HPC cluster. That one is running off 8 Velicoraptors in a Raid 10, dedicated. What are the best practices around this sort of ...


8

You can use "ulimit" in the "start" section of the pgbouncer init script: /etc/init.d/pgbouncer: [... snip ...] case "$1" in start) # Check if we are still disabled in /etc/default/pgbouncer [ "${START:-}" = "0" ] && exit 0 log_daemon_msg "Starting PgBouncer" $NAME test -d $PIDDIR || install -d -o ...


7

If you're going the route of high performance typically you'll want to run as few other (scheduled) processes as possible as they'll interfere with your application. Linux, like the classical UNIX operating systems, is designed to run multiple applications concurrently in a fair way and tries to prevent resource starvation and you'll be aiming for the ...


7

The reason for using noatime or relatime on an actual disk is that without it reads will also involve writing to disk. This additional writing can slow down the system, cause disks to spin up which would otherwise have remained spun down for much longer, as well as cause wear on flash media. But with tmpfs no data ever has to be written to disk. The data ...


6

Upgrade your server I know you don't want to hear it. I know you client doesn't want to hear it. But it's what you need to hear! If the client is running 4 websites (production no less) they should fork out some cash for a better server. But why? micro instances are NOT designed for production use (AWS says so themselves), they suffer from contention, don'...


6

This Filesystem stores a huge amount of small files with very many SEEK operations but low IO throughput. 🎉 This is thing that catches lots of people nowadays. Alas, conventional FSes do not scale any well here. I can give you probably just a few advices when it comes to the set-up you already have: EXT4 over RAID-6 on HDDs: Lower vm.vfs_cache_pressure ...


6

Check the driver version. There was the bug with correct accounting of the rx_no_dma_resources, when the rx buffer is full. So check the length of the ring buffers (ethtool -g <iface>) and increase it (ethtool -G <iface> rx <size> tx <size>, but it will cause a short time break in the packet processing). Note: After update of the ...


5

That image says that the maximum UDP transfer rate over 1gbps ethernet is 119635891 Bytes per second. Multiple 119635891 with 8 (one byte is 8 bits) and you get 957087128 == 957,08mbps


5

What's the question here? You don't seem to be comparing apples-to-apples. So I'm not sure if you just want this system to run like your other systems, or if you really care about the cause of the performance difference. You could obtain another of the server model and SKU that you're familiar with. You could use the same type of disks you used in previous ...


5

You are correct in your assumptions. For your particular case stacking Mhz's instead of vCores is a right way to go. Take a look at this processor: (features 4x 3,5-3,7 Ghz physical cores + HT). However, I cannot see the real issue from what you have described unless you can provide your actual "CPU Ready" value.


5

I re-ran the test from Mumbai with no bandwidth restriction and got a TTFB of 237ms for the page and 7ms for a static resource. You restricted the test to 2G, which is very low bandwidth. So the problem isn't the website, it's the restrictions you placed on your test.


4

In newer kernel version sudo perf record -e context-switches -a # record the events # then ctrl+c sudo perf report # inspect the result This will give you the exactly result about context-switches events. And you may be find the reason caused the context-switches by append "-g" flags (The readable result determined by symbol information) sudo perf ...


4

Congratulations, you've managed to use nearly all of your swap space. The first obvious problem here is that you went very deep into swap. This is probably what's causing the system to thrash so hard (lots of time spent in system, I/O wait and software interrupts). First thing to do is to reduce the number of Apache processes that are running. You don't ...


4

You are on right track. Here are some things which should help you figure out remaining details. You need detailed monitoring of Apache on all stats provided by mod_status "_" Waiting for Connection "S" Starting up "R" Reading Request "W" Sending Reply "K" Keepalive (read) "D" DNS Lookup "C" Closing connection "L" Logging "G" Gracefully finishing "I" Idle ...


4

I don't think it's possible to say: Buy X, install Y, and your problem will be instantly solved. This is going to take several iterations to get through, and you'll need to put more work into identifying the bottlenecks when they inevitably appear. For the most part I'm going to avoid recommending specific hardware, as that would be out of date by the time I ...


4

innodb_buffer_pool_instances: You cannot see its effect unless innodb_buffer_pool_size is few gigs, dividing the buffer pool into separate instances can improve efficiency. This is also a tuning practice for innodb_buffer_pool_instance so that each buffer pool instance is at least 1GB. eg, If innodb_buffer_pool_size = 4GB then, ...


4

This site reports some more suggestions, including increasing the NIC txqueuelen. These suggestions are mainly aimed at high latency networks, however I noticed that under Fedora24 Infiniband interfaces get by default a lower txqueuelen value (256) than other interfaces (1000), so the txquelen suggestion, besides the others, may also make sense in this case. ...


4

I think the bottleneck here is the PCIe link between the ConnectX and the host. The ConnectX-3 has a Gen. 3 x8 PCIe connection, which is bound to a theoretical maximum of 63.04 Gbps (according to this answer), and that doesn't include overhead (see here).


Only top voted, non community-wiki answers of a minimum length are eligible