2

I'm setting up a system that has hard requirements on the response time. How do I force Apache2 to drop requests in the queue after 200ms if it hasn't been served?

Background

One of our internal services needs to interface with another service. The other service requires that we make all responses within a very short timeframe. For small number of queries, we are able to serve the request within the desired timeframe. However, once the system is loaded with lots of requests, the response time really suffers.

Since the load is going to be pretty much continuous, we get a traffic jam situation where the older queries are being queued up and then serviced, but by then the request has already expired. How do I make sure that we either serve the request within the limited time allotted, or just plain drop the request?

Loading
  • 21
  • 1
  • 3
  • It might help to give a bit of context and explain exactly what you're trying to achieve at a high level, there might be other solutions for the problem. – Drew Khoury Sep 24 '13 at 03:26

2 Answers2

1

Apache httpd has one timeout knob, TimeOut which deals with timing out TCP send and receive actions if the client is not sending data to the server in a timely manner (and a few other things). This is provided at second granularity and defaults to 300 seconds. What this won't do is timeout a request that takes, say 5 seconds to process locally.

The only queue in httpd is the ListenBacklog for requests that sit on the socket listen backlog waiting to be picked up by a currently busy httpd worker thread/child. Their "Time" according to Apache doesn't start until they are picked up by a worker. You could use a small ListenBacklog setting so new requests are refused when your server starts going slow. Actually, if your clients are ending the connection after 200ms this probably doesn't matter as the backlog requests will never get started properly as http requests.

Connections only get onto the backlog when you have MaxClients or ServerLimit / ThreadLimit / TheadsPerChild connections currently in use. You may be able to tune these down to a level where your service is able to survive better.

Otherwise requests are being handled by a httpd worker child/thread and are not producing a http response in < 200ms which I suspect is what you are running into. If the response is all dealt with "in process" in httpd there's not much you can do other than fix whatever triggers the problem. If you have an application running on top of httpd producing the responses, how would it react if httpd cuts off the connection? How does the database, if one is underneath, deal with the cut off connection? Usually they carry on unbeknownst to complete the request until they try and write data back to a socket at the end. Dealing with timeouts is something that needs to be handled in your stack at each level so it works end to end.

At the levels you are talking I think you would need something wholly different to httpd to achieve your requirements. Maybe Mongrel2 with it's queue based work distribution could handle the front end timeouts more easily? Maybe a custom event based http server could handle the timeouts?. As David mentions, even TCP could struggle to deliver at the levels you are asking. The TCP component of the request doesn't even count towards server side application level timeouts you implement unless you have some smarts built in that calculate round trip times (which http doesn't).

Matt
  • 1,559
  • 8
  • 11
0

Sorry, this just isn't sensible. TCP was simply not made to provide a low latency service. Things like delayed ACK and Nagle's algorithm interact to make it unsuitable for applications where latency must be low.

David Schwartz
  • 31,449
  • 2
  • 55
  • 84
  • TCP was certainly made to provide for high latency networks but I'm not sure that precludes it working with low latency requirements. If you control the recv/send ends (and hopefully the network in between) I don't see TCP as the limitation. Most applications and/or stacks can be tuned to prefer low latency/small packets than higher through put that can introduce packet delays. Apache httpd on the other hand... – Matt Sep 24 '13 at 19:01