Measuring page response time for different servers in web farm (HAProxy?)

Question

I'm building out a small webserver farm (IIS7.5) to serve up an internal website. One of the things I want to measure is what the HTTP response time is between user requests and the page being served.

Given that I'm using HAProxy as a front-end load balancer, it would seem natural that HAProxy would be in an ideal place to collect this information, as it sits between every request/response. However, looking at the generated stats page, there's a lot of good information but I can't see anything about response times.

What would be the best way to go about doing this? Is this a function that HAProxy can do, or would I need some other specialist monitoring software?

AFAIK, haproxy does not fully buffers server response. I'm not sure on it though, but you need to check it. If I am right, than 'response time' would differ a lot depending on client's network speed. — rvs, Mar 23 '11 at 10:45
You've made me think whether I care more about the response time from client->server, or the response time from HAProxy->server. Obviously the first will vary depending on client->HAProxy latency, whereas I actually care about how long it takes the server to process the response once it has received the request. So I guess I'm looking for the HAProxy->Server response time... — growse, Mar 23 '11 at 10:57
Ok, I'm not expert on haproxy, just suggesting you to make sure you've selected right metric. — rvs, Mar 23 '11 at 11:19

Holger Just · Accepted Answer · 2017-03-05T17:52:04.220

HAProxy provides very detailed statistics about all HTTP requests. Way more than you see on the stats page.

You need to setup a syslog server listening to UDP syslog messages (e.g. on localhost). That syslog server needs to be configured to write the log messages received by HAProxy to a file (or wherever you like it). An example HAproxy configuration could look like this:

defaults
  # send the logs to localhost:514 via UDP, using the local0 facility
  log 127.0.0.1 local0 debug
  option httplog

  mode http
  balance roundrobin

listen farm
  bind 10.0.0.1:80

  server iis1 10.0.1.1:80
  server iis2 10.0.1.2:80

See section 8 of the configuration manual for detailed information about the log format and how to set it up.

Thanks - I'll point it at my syslog server and write a script to parse the data. — growse, Mar 23 '11 at 11:27

score 1 · Answer 2 · answered Mar 23 '11 at 12:43

I use option httplog clf for logging and capture a few extra fields, but here is a python regex I wrote for that (which might save you some time):

line_regex=re.compile('ny-lb[0-9]{2} (?P<HTTP_CLIENT_IP>[^ ]+) - - \[[^\]]*\] "(?P<HTTP_METHOD>[^ ]+) (?P<HTTP_URI>[^ ]+) (?P<HTTP_VERSION>\w+/\d+\.\d+)" (?P<HTTP_RESPONSE_CODE>\d{3}) (?P<HTTP_HAPROXY_BYTES_READ>\d+) "[^"]*" "[^"]*" [^ ]+ [^ ]+ "(?P<HTTP_HAPROXY_FRONTEND>[^"]+)" "(?P<HTTP_HAPROXY_BACKEND>[^"]+)" "(?P<HTTP_SERVER>[^"]+)" (?P<HTTP_HAPROXY_TQ>-?\d*) (?P<HTTP_HAPROXY_TW>-?\d*) (?P<HTTP_HAPROXY_TC>-?\d*) (?P<HTTP_HAPROXY_TR>-?\d*) (?P<HTTP_HAPROXY_TT>-?\d*) "(?P<HTTP_HAPROXY_TERM_STATE>[^"]*)" (?P<HTTP_HAPROXY_ACTCONN>-?\d*) (?P<HTTP_HAPROXY_FECONN>-?\d*) (?P<HTTP_HAPROXY_BECONN>-?\d*) (?P<HTTP_HAPROXY_SRV_CONN>-?\d*) (?P<HTTP_HAPROXY_RETRIES>-?\d*) (?P<HTTP_HAPROXY_SRV_QUEUE>-?\d*) (?P<HTTP_HAPROXY_BACKEND_QUEUE>-?\d*) "[^"]*" "[^"]*" "(?P<HTTP_REFERER>[^"]*)" "(?P<HTTP_UA>[^"]*)" "(?P<HTTP_HOST>[^"]*)" "(?P<HTTP_X_FORWARD_FOR>[^"]*)"')

You probably want the HTTP_HAPROXY_TR field:

Tr: server response time (HTTP mode only). It's the time elapsed between the moment the TCP connection was established to the server and the moment the server sent its complete response headers. It purely shows its request processing time, without the network overhead due to the data transmission. It is worth noting that when the client has data to send to the server, for instance during a POST request, the time already runs, and this can distort apparent response time. For this reason, it's generally wise not to trust too much this field for POST requests initiated from clients behind an untrusted network. A value of "-1" here means that the last the response header (empty line) was never seen, most likely because the server timeout stroke before the server managed to process the request.

score 1 · Answer 3 · answered Mar 04 '17 at 21:35

No syslog needed, the CSV export function from the stats page provides all the details.

$ lynx -source -auth=STATSUSER:STATSPASSWORD 'http://haproxy/stats;csv' | cut -d ',' -f 2,8,61

Fields 2,8 and 61 will show backend's name, total sessions and avg response time in ms for the last 1024 reqs.

Measuring page response time for different servers in web farm (HAProxy?)

3 Answers3