Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I have a webserver that I wish to benchmark before I make some optimizations to it to see if they have any effect.

However, I want to know what are the best practices for benchmarking?

For example, a co-worker told me to benchmark the machine with another machine on a local network to eliminate network traffic problems.

However, I was thinking of using an off-site machine to benchmark because I wanted to see if the optimizations would make any difference in a real-world case.

I was arguing that many of the speed tweaks deal with optimizing network connection. For example, in Apache, the KeepAlive value allows browsers to use a single TCP connection to request multiple objects, instead of opening and closing a connection for each resource.

If the test was done on a local network connection, then that tweak wouldn't really make that much of a difference, right? The same with minimizing js/css and removing whitespace/comments from HTML.

On the other hand, I do see the problem with internet traffic making each time the benchmark run from being consistent. I wouldn't really know if the changes in the numbers are from the tweaks or if the servers in between had increased or decreased loads.

Who's right? What is the best practice for benchmarking? Should we do both?

tl;dr - Should I benchmark using a machine on the local network/off-site/or both?

share|improve this question

2 Answers 2

up vote 2 down vote accepted

This all depends very much about your site. Doing both off-line and on-site benchmarks won't hurt. But, what to tweak and how to test?

If you are mostly serving out static content or some very simple dynamic content and doing so at an insane rate, then tweaking things in Apache timeout / kernel-level makes sense.

If you have a dynamic site, then tweaking very soon becomes a different beast. Sure, making all those Apache and kernel level optimizations are still a sensible thing to do, but don't get fooled and smoke-mirrored by those.

When it comes to performance tweaking / testing, always first take care of the slowest parts. With a dynamic site the real bottleneck is most likely 1) the database or 2) the scripts your site is running.

If the slowest part is the database, it doesn't make sense to performance-tune your Apache before you can make your database fast. In addition to that you'll need to be sure you have proper caching techniques in place and you run memcached or similar if possible.

So, that was about what to benchmark and tweak. Now, how to benchmark?

I like to run two kinds of benchmarks. If the numbers are already known -- say, a site is getting a revamp, so you know unique visitors, page loads and similar statistics from the old site --, I run a realistic benchmark with rate similar to current web site. I also benchmark it with a similar rate * 2 to see if it can stand a little bit of future growth.

The other kind of benchmark I like to run is to smoke and burn the site. I'll just torture the hell out of it by running benchmark as fast as possible with insane amounts of simultaneous requests. If it can stand it, great, if it cannot, I find what's the breaking point.

Tools I like: ab, siege, JMeter.

share|improve this answer
    
It is a CMS written in PHP. I have already tuned the MySQL database with MySQL Tuner (mysqltuner.pl) and PHP with APC. I was planning on picking a few pages with dynamic content and running ab against them. Though most of the content is behind a login system so I am kinda limited by that. So I know what to do, but are is it a best practices to just on-site or off-site? What do you do? –  rlorenzo Nov 4 '10 at 18:48
    
I run on-site benchmarking a lot -- after all, that's the real-world environment. If there are other bottlenecks than the software (such as a firewall, server disk I/O), those will get caught. –  Janne Pikkarainen Nov 5 '10 at 7:39

I have used a custom log format with the %T (service time) specified on an ongoing basis. se I replaced %l (identd name) in the format. This can be used to identify slow pages for tuning. Large responses on slow lines can cause false positives, as can network retries.

I used a custom log summary script to identify pages with slowest responses and average response time. These may be good targets for optimization. Comparing reports over time helps identify upcoming problems.

Client side tools can provide good stress benchmarks and verify fixes haven't broken the site. There are factors which can provide benchmarks which do not match what you get in production.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.