I have a webserver that I wish to benchmark before I make some optimizations to it to see if they have any effect.
However, I want to know what are the best practices for benchmarking?
For example, a co-worker told me to benchmark the machine with another machine on a local network to eliminate network traffic problems.
However, I was thinking of using an off-site machine to benchmark because I wanted to see if the optimizations would make any difference in a real-world case.
I was arguing that many of the speed tweaks deal with optimizing network connection. For example, in Apache, the KeepAlive value allows browsers to use a single TCP connection to request multiple objects, instead of opening and closing a connection for each resource.
If the test was done on a local network connection, then that tweak wouldn't really make that much of a difference, right? The same with minimizing js/css and removing whitespace/comments from HTML.
On the other hand, I do see the problem with internet traffic making each time the benchmark run from being consistent. I wouldn't really know if the changes in the numbers are from the tweaks or if the servers in between had increased or decreased loads.
Who's right? What is the best practice for benchmarking? Should we do both?
tl;dr - Should I benchmark using a machine on the local network/off-site/or both?