Effects of HTTP/TCP connection handshakes and server performance

Question

When running apache bench on the same server as the website like:

ab -n 1000 -c 10 localhost:8080/

I am most probably not getting accurate results when compared to users hitting the server from various locations.

I'm trying to understand how or rather why this will effect real world performance since a user in china will have different latency issues when compared to someone in the same state/country.

Say my web server has a maximum thread limit of 100.

Can someone explain in detail how end user latency can/will effect server performance.

I'm assuming here that each request will be computed equally at say 10ms.

What I'm not understand is how external factors can effect overal server performance, specifically internet connections (location, or even device like mobile) and http/tcp handshakes etc.

Answer 1

In general, end user latency doesn't affect server performance. The main difference will be that with higher end user latency, your server will have more connections at a time because each connection takes a bit longer to complete. But the server is still doing about the same amount of work for each connection. As long as you don't hit server limitations, primarily memory, it doesn't matter.

The server won't start doing any heavy work for a connection until it has the entire request. So if it takes longer to set up the connection and get the request, that will just mean the server will wait a bit longer, while basically not doing anything, before doing the real processing.

Typically, the server processes the request and queues the reply in one shot. Client and network latency may then mean it takes a bit longer to empty that queue. But the part of the server that handles this is heavily optimized and the logic for the particular page or object has already run to completion generating the response. So again, there's typically no significant effect on server performance.

However, the client experience can be much worse. This is especially true if the service has a lot of cases where the client has to get information from the server and then connect back for more information. For example, if a web page tells the client to load a bunch of frames and then those frames tell the client to load a bunch of images, there will be many "back and forth" operations (each increased by the network latency) before the client sees a result. But the server does the same amount of work.

Answer 2

Actually that won't be a problem unless you have a real-time operating, multi-multi processor (say, count of 1K Cpu), lots of memory super computer...

In a multi-process system all processes have a time window which is called Quantum Size. Operating systems with multi-process capabilities, which is the case from roughly '80s through '90s to today, swithes back and forth between running processes giving each of 'em that quantum size. This time window is around 20 mili-seconds in our modern operating systems and that switching is being done so quickly with very low switching overhead. Say, if we have one cpu, two processes being switched between, in 1 second, which is equal to 1000 mili-second, we can run them for 900-950-980(maybe) mili-seconds (Difference is gone to process switching). Anyway, as I said, this switching is done so quickly and imagine 50 processes running, we see all the processes are running simultaniously. Actually they're not and that is multi-processing, basics of process scheduling...

When we have multi-threads in processes, O.S. schedules the process first and gives it a quantum, then it schedules the threads in that process. And in that quantum, threads are scheduling, too. When whole quantum ends, O.S. schedules another process (or same according to scheduling algorithm) and threads in that new process are being scheduled, too.

There are two levels of execution enviorenment for threads. One is user level, two is kernel level. The one i mentioned above is user level. Process scheduling, thread scheduling in that quantum size. But, when you go down to kernel level scheduler can schedule different threads from different processes. The quantum is applied to threads directly in kernel level...

After speaking of all that lets begin to understand how an end connection latency may affect server performance:

Your threads must be in kernel level if you want maximum performance, and I know apache threads are not in kernel mode. Apache itself is in user mode, it's a user-end application and its threads runs in user level mode. So you wont get %100 performance from that server in any way... Lets say the threads are runing in kernel mode and you have two cpus. One thread for first cpu, one thread for second. Now two threads really are running simultaniosly. A web worker thread is actually an I/O Bounded thread from O.S. point of view and when it requests for some file it'll be blocked until file is ready. Scheduler will schedule another worker thread to run. When 'that' file is ready, blocked thread will move to ready queue and will be scheduled again. So on so good... What happens if you have 100 worker threads? This question brings another question: When a worker thread is created?

Speaking for web server applications, a worker thread is created when low-level IP connection is made. So, your actual two threads are already running, a new connection established by hardware (they have their own PUs, and interrups main system for data-info transfer), a new worker thread poped up, it' been sent to ready queue for scheduling...

Back to main theme, how an external factor affects system performance. It is all about system limitations. Thread count affects performance wheter system has enough process unit to handle them or not. Basic math, two processors only process two threads at the same time... Network connection badwidth affects performance by "how many connections can it accept". Say a connection data is 10-byte and bandwith is 100-byte per second, you can have 10 connections per second...

Scaling those is up to you. You have to remember just one thing: Your total cpu resource is already processing the threads those are in the ready queue already... So when a new thread pops up it just dont makes things worse for present threads.

Performance can be an issue when the server app. first starts. Quickly it'll hit top-limit. It is kind of acceleration of a car. It will accelerate first, and hit top-speed after some time. You can go top speed until you ran out of gas or put your feet out of gas pedal.