46

Recognize that Ubuntu 14.04 uses Apache 2 with PHP running through an mpm_prefork module, of which an editable file is in /etc/apache2/mods-enabled/mpm_prefork.conf. Also, recognize that starting in Apache 2.4, MaxClients is now renamed as MaxRequestWorkers, and so any documentation regarding MaxClients needs to be switched to MaxRequestWorkers. Stop the ...


43

Processor advancements, clock speed and IPC calculations can make it almost impossible to try to reasonably compare decade old CPUs to modern ones. Not only are the instructions per cycle going to vary, but newer processors have instruction sets dedicated to complex calculations (Intel has added AES-NI as an example), clock speed is no longer a reasonable ...


38

I've found that when I've had to tune for lower latency vs throughput, I've tuned nr_requests down from it's default (to as low as 32). The idea being smaller batches equals lower latency. Also for read_ahead_kb I've found that for sequential reads/writes, increasing this value offers better throughput, but I've found that this option really depends on ...


32

I don't want to sound terrible by emphasizing something that should be obvious here, but you're comparing a high-end server processor from 2014 to a high-end server processor from 2007. I don't think this requires much more explanation. There's a reason an HP ProLiant DL580 G5 is available so inexpensively today. They were large, slow and lack many ...


28

Arpit, if you imagine that the absolutely smallest likely web response, even if it's a static text file is one Ethernet packet (~1,500 bytes) then 500,000 of them works out at around 750,000,000 bytes, or roughly 7.5 gigabit. So unless your server has very easily offloaded 10Gb NICs (and it doesnt't, the one you've got is one hundred times slower) and have ...


23

1 x 10Gb link for performance. Otherwise if a single server needs to use 1.1Gbs to another server it can't because most load balancing systems use destination MAC or IP (Which would be the same). This also eliminates issues where links are busier then other links because of the same fact, if the hash works out to be on the same link they end up on the same ...


19

Is it ever sane to use a Virtualized solution when performing I/O heavy workloads? Yep, very sane indeed, in fact for most organisations now virtual is the default and doing things on physical boxes is the very much the exception. We have over 100k VMs of all forms and many of them are >40k IOPS with no issue at all. What are the best practices around ...


17

somaxconn determines the maximum number of backlogged connections allowed for each TCP port on the system. Increasing it (recommended for servers) can prevent "connection refused" messages, but it can result in slow connections if the server can't handle the increased load. You can check the current backlog with netstat -ant | grep -c SYN_REC according to ...


15

First advice If you cannot afford to lose any data (I mean once a user entered new data, if that cannot be lost in the coming seconds) and because you do not have something like a UPS, then I would not remove the write barrier, neither would I switch to writeback. Removing write barriers If you remove write barrier, then in case of crash or power loss, the ...


13

Have you increased maxThreads in the AJP 1.3 Connector on port 8009?


13

First of all, you should write a new benchmarking tool. You're actually benchmarking ab not nginx.


13

At first, the client requests will get queued, until there is a process/thread that gets free on the apache server. So, the clients will see a delay in loading the page. See the MaxClients parameter documentation for more information. When placed in the backlog queue, the client request can eventually time out on the client side. Then the user will see ...


12

Here's the schedule of tuned-adm configurations... I think it helps to see them in tabular form. The main thing to note is that the default RHEL6 settings suck!! The other thing is that the enterprise-storage and virtual-guest profiles are identical except for reduced swappiness on the virtual guest side (makes sense, right?). As for a recommendation on ...


12

Short answer: disable comp-lzo. I realize this is an old post, but I was also suffering from poor OpenVPN performance. I had tried everything, adjusting the MTU, changing the snd and rcv buffers, mss clamping, you name it. CPU load was negligible. On a whim, I disabled compression (removed comp-lzo from the client and the server) and performance increased ...


11

the delay between the tx and rx events and the generation of interrupts for those events. rx-frames[-irq] rx-usecs[-irq] tx-frames[-irq] tx-usecs[-irq] The frames parameters specify how many packets are received/transmitted before generating an interrupt. The usecs parameters specify how many microseconds after at least 1 packet is received/transmitted ...


11

I am sorry, but my honest answer is this: If you don't even know how to check if your database is indexed you are so far out of your league that your only option is to hire a professional, given the scope of your project. Not a group of cheapo code slaves, but someone with a lot of experience with sites like yours. We can give you a plethora of tips, but ...


11

Taskset is for binding a process to one or more CPUs; essentially specifying where it can run at initial execution or while it's running. If using RHEL/CentOS on modern server equipment, numactl is recommended over taskset. Cpuset/cset is for CPU shielding and is a framework build around Linux cgroups. Cset was never popular on certain distributions (like ...


11

I have a similar (albeit smaller) setup, with 12x 2TB disks in a RAID6 array, used for the very same purpose (rsnapshot backup server). First, it is perfectly normal for du -hs to take so much time on such a large, and used, filesystem. Moreover du accounts for hardlinks, which cause considerable and bursty CPU load in addition to the obvious IO load. Your ...


10

Try using pidstat -wt The 't' option shows the threads also. It might be a thread who is doing the context switches .


10

Caveat: there may be inaccuracies below. I've been learning about a lot of this stuff as I go along, so take it with a pinch of salt. This is quite long, but you could just read the parameters we were playing with, then skip to the Conclusion at the end. There are a number of layers where you can worry about SQLite write performance: We looked at the ...


10

This is outlined in the document you linked under the "Tuning Procedures" heading. In order to enter the Advanced system options menu for modifying a G6 or newer HP ProLiant for a low-latency application, you need to get to the main BIOS screen by pressing F9 during POST. Once there, type Ctrl-A. This will present an additional menu named "Service Options"...


10

Honestly, I wouldn't be using Ubuntu for this... but there are options that can be applied to any Linux variant. You'll want to increate your network stack buffers: net.core.rmem_default = 10000000 net.core.wmem_default = 10000000 net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 If the application is writing to disk, maybe a scheduler/elevator ...


10

Is it ever sane to use a Virtualized solution when performing I/O heavy workloads? Does a database server regularly pulling 1gb/second random IO count? Have one here. Or a virtual file server delivering up to 600mb/second to a HPC cluster. That one is running off 8 Velicoraptors in a Raid 10, dedicated. What are the best practices around this sort of ...


9

My job is building large (>1m user) commercial VoD systems and unless you can utilise multicast/anycast and don't use a CDN then you just have one option and that's to scale up your storage systems and networking to handle the maximum concurrent IO load you need. Certainly local caching, as you alude to, can help but I always size our streamers to assume ...


8

Let's identify the bottleneck. Since you're on the same machine, we can assume it's either CPU or disk activity. For the one text file, it shouldn't be disk activity, but at 35k connections, you may be generating 35MB of logging every second as well. The examples you're showing don't run access logging, only errors. Your config, however, has much more going ...


8

Look in the /proc/irq/283 directory. There is a smp_affinity_list file which shows which CPUs will get the 283 interrupt. For you this file probably contains "0" (and smp_affinity probably contains "1"). You can write the CPU range to the smp_affinity_list file: echo 0-7 | sudo tee /proc/irq/283/smp_affinity_list Or you can write a bitmask, where each bit ...


8

The connections you listed in the first paragraph are either ESTABLISHED or in process of being cleaned up after they have been used. Established means what the name implies. A connection is established between one of your users and the HAProxy. Usage as intended. The other states you mention in the first paragraph all indicate that an previously ...


8

Have a watch of Shak and Larry's performance tuning videos from Summit, they talk about the tuned profiles in depth. Part 1 - http://www.youtube.com/watch?v=fATEiBJ3pKw Part 2 - http://www.youtube.com/watch?v=km-vLELmWLs One of the biggest intended takeaways is that the profiles are only a recommended starting point, not immutable numbers which are ...


7

There's a lot to cover here. If you want more performance (in order from greatest to least impact): Add another pair of disks and expand to RAID 1+0. This will provide the greatest benefit. Tune your filesystem (noatime, journaling mode, remove write barriers, etc.) and/or move to a higher-performance filesystem like XFS or even ext4. Go back to a ...


7

The short answer is that any tuning is guesswork and only has value when backed up with empiricial data: Try it. Measure it. If you don't like it, tweak it. A longer answer: Increasing the dirty_ratio would probably mean higher write performance ...IO will be blocked for a considerably longer time No. Increasing the dirty ratio means that your system is ...


Only top voted, non community-wiki answers of a minimum length are eligible