4

I'm using akka-http for a web application that has a quite short response time target. I use the routing DSLs completeWithFuture method with a chain of CompletableFutures.

I noticed that, when chaining every future using the XXXasync variant of the CompletionStage methods and passing the same executor, the thread used for processing the stage can change arbitrarily, causing a higher response time for some requests in case that all threads of the specified executor are used. So I passed my custom executor to the first CompletableFuture and chained all following stages with normal variant, in order to use the same thread for them.

Problem: One stage does the conversion of the HttpEntity to a String via HttpEntity.toStrict() and that method uses a thread from akka.actor.default-dispatcher. With increasing workload there are more and more requests that exceed the required response time at the beginning of the next stage, despite passing a timeout to toStrict that is way lower than the targeted response time and seeing no timeout exceptions.

Simplified Code:

private Route handleRequest(final HttpRequest request) {
    return completeWithFuture(CompletableFuture.runAsync(() -> preprocessing(), systemDispatcher) // Dispatcher 1
            .thenCompose((preprocessingResult) -> // please ignore that preprocessingResult is not used in that simplified version
            entityToString(request.entity()).thenApply((requestString) -> generateResponse(requestString))));
}

public CompletionStage<String> entityToString(final HttpEntity entity) {
        long start = System.nanoTime();
        return entity.toStrict(bodyReadTimeoutMillis, materializer).thenApplyAsync((final Strict strict) -> {
            System.out.println(start-System.nanoTime()); // varies between <1ms and >500ms
            return strict.getData().utf8String();
        }, systemDispatcher); // Dispatcher 2
    }

So my guess is that the switch from a thread of my custom executor to one of akkas default actor dispatcher thread and back is causing the problem.

Questions: Is there another explanation for the delay in my entityToString method? Is there a way to achieve the same as toStrict, i.e. getting the whole possibly chunked message body as string while avoiding to switch threads multiple times?

Please note that I need the timeout functionality of the toStrict method to abort processing of slow POST requests.

UPDATE

Thinking about it for the last days, I believe that it is not possible to achieve a non-blocking read, that akka guarantees, without switching threads. So the real problem is the noticeably high delay probably caused by the scheduling after toStrict.

I tried to use different dispatchers (see the comments Dispatcher 1 / Dispatcher 2 in the code above) and logged the inhabitants count of Dispatcher 2 in the case the delay exceeds 50ms. I can not find a proper documentation but I assume this is the number of scheduled tasks. I ran apache bench with 10000 requests, 200 concurrent connections and got 55 times a delay exceeding 50ms. The output shows a maximum of 80 inhabitants.

I ran that test on Amazons m3.2xlarge instance (8 vCPU, 30GB RAM, Ubuntu 16.04), no other processes consuming any noticeable amount of cpu). The dispatchers are of type fork-join-executor with parallelism-factor = 1.

Real traffic with a much more variable number of concurrent requests causes an increasing rate of requests that exceed the limit (up to 50%).

The average time for processing a request is below 1ms. What is causing that infrequent delay after toStrict and how to avoid it?

teano
  • 505
  • 5
  • 12
  • 1
    (1) When toStrict collects data into a single ByteString, it may have to wait for data from the network. So, it will depend on how long the client takes to actually send the complete request. (2) If you run 200 concurrent requests on 8 cores it means that every single request will only get a small share of CPU time so the overall execution time will increase by at least that factor. – jrudolph Sep 21 '16 at 12:02
  • In most real world scenario, you will have to apply much higher timeouts if you don't control the clients, because clients accessing your site may be slow with sending data. In any case, it would be better if you could process the incoming data in a streaming fashion, so that 1) you don't need to use toStrict 2) don't need to buffer all incoming data before being able to process it. – jrudolph Sep 21 '16 at 12:05
  • In my current scenario, i'm bidding on real time auctions and responses are not valid after 50ms.Unfortunately sometimes i receive incomplete http POST bodys. Waiting for the missing parts must be aborted in order to free ressources for the processing following request. I tried to process the incoming data with streams, but could not figure out 1) how to answer http requests with streams 2) stop the processing of requests after 50ms 3) apply tcp backpressure in case of to much requests. The request contains a complex JSON object that needs to be parsed before it can be processed – teano Sep 22 '16 at 17:49

1 Answers1

1

You can try get the entity content using akka-streams, and achieve timeout requirement in a different way instead of entity read.

entity.getDataBytes().runFold(ByteString.empty(), ByteString::concat, materializer)
.thenCompose(r -> r.utf8String());
Tiago
  • 691
  • 7
  • 12
  • I tried that, but the stream also uses `akka.actor.default-dispatcher`. `thenCompose` should be `thenApply`, I think. Wichever, it always uses a thread from another threadpool. As no executor is specified in your code, it uses a thread from `ForkJoinPool.commonPool`. Also I wouldn't know how to stop a stream and close the connection, like `toStrict` does, if I could achieve a timeout somehow. Finally, I think entity.getDataBytes() does not _collect all possible parts_ of the message body as described in the `toStrict`documentation. – teano Sep 19 '16 at 07:12
  • Did you try to create the materializer using a context from an actor that is configured to use a custom dispatcher? – Tiago Sep 19 '16 at 13:45
  • Same result. Thinking about it for the last days, I believe that it is not possible to achieve a non-blocking read, that akka guarantees, without switching threads. So the real problem is the noticeably high delay caused by the scheduling after `toStrict`. – teano Sep 20 '16 at 08:14