HTTP request duration with HAProxy

Question

Is it possible to monitor the duration of HTTP requests? (from the moment that the request is received by the load balancer, to the time when the response is returned to the client by the load balancer)

I would like to monitor this metric in the frontend, because I want to include the queue time. If I measure the duration of the backend I may get good stats but the real performance can be bad (for example if the request gets queued in the load balancer itself).

For example I would like to monitor a web application during traffic spikes to make sure that the responses are still fast, from the point of view of the user (e.g. < 3 seconds).

Answer 1

HAproxy offers all those informations in the logs .

see Haproxy HTTP Log Format

In particular you are looking for the section number 6 ,

6 TR '/' Tw '/' Tc '/' Tr '/' Ta* 10/0/30/69/109

where

• "TR" is the total time in milliseconds spent waiting for a full HTTP request from the client (not counting body) after the first byte was received. It can be "-1" if the connection was aborted before a complete request could be received or the a bad request was received. It should always be very small because a request generally fits in one single packet. Large times here generally indicate network issues between the client and haproxy or requests being typed by hand. See "Timers" below for more details.

• "Tw" is the total time in milliseconds spent waiting in the various queues. It can be "-1" if the connection was aborted before reaching the queue. See "Timers" below for more details.

• "Tc" is the total time in milliseconds spent waiting for the connection to establish to the final server, including retries. It can be "-1" if the request was aborted before a connection could be established. See "Timers" below for more details.

• "Tr" is the total time in milliseconds spent waiting for the server to send a full HTTP response, not counting data. It can be "-1" if the request was aborted before a complete response could be received. It generally matches the server's processing time for the request, though it may be altered by the amount of data sent by the client to the server. Large times here on "GET" requests generally indicate an overloaded server. See "Timers" below for more details.

• "Ta" is the time the request remained active in haproxy, which is the total time in milliseconds elapsed between the first byte of the request was received and the last byte of response was sent. It covers all possible processing except the handshake (see Th) and idle time (see Ti). There is one exception, if "option logasap" was specified, then the time counting stops at the moment the log is emitted. In this case, a '+' sign is prepended before the value, indicating that the final one will be larger. See "Timers" below for more details.

Exporting those metrics from the logs to a more useful format / backend can be done in many ways, Logstash comes to mind since it can parse those logs and send them to many different backends