0

I'm getting some unexpected behaviour in an akka http endpoint when a client sends an "Expect: 100-continue" header as part of a post request where the request causes backpressure in the system. The following (contrived) endpoint is the simplest way that I could find to reproduce this behaviour - I'm not trying to upload a file, I actually came across this when trying to pipe incoming POST data to the stdin of a server process and to return data from the stdout of that process. The issue seems to be the same though.

val route = path("test") {
post {
  extractRequest { httpRequest =>
    val file = new File("test-123")
    def fileWritingFuture = httpRequest.entity.dataBytes.runWith(FileIO.toFile(file))
    def fileSource = Source.fromFuture(fileWritingFuture).map(x => ByteString("")).drop(1).concat(FileIO.fromFile(file))
    complete(HttpEntity.Chunked.fromData(`application/pdf`,  fileSource))
  }
 }
}

When using this from a curl client (which sends the Expect: 100-continue header by default) with a POST payload of about 18kb in size the system sends a 200 OK response with a Connection: close header rather than a 100 continue, presumably as a way to slow down the incoming requests. Curl then responds to this by waiting for 2 seconds before sending the payload (without ever having received a 100 Continue), and the endpoint continues at that point and returns the response from the file. Curl could be behaving incorrectly here, I'm not sure, but I think that in any case it can be expected that clients may behave incorrectly from time to time!

Sending with a much smaller payload leads to the server responding with a 100 continue, and the flow then continues immediately.

Also, as may be expected, omitting the Expect: 100-continue header in curl (by specifying -H "Expect:" on the command line) means that payload is sent and the response is returned immediately.

My question is whether the behaviour on the part of akka http is expected, in particular:

  • should akka http try to close an incoming HTTP POST request which has specified Expect: 100-continue where there is any backpressure in the system, even if, as in this case the backpressure is inevitable because of the way the incoming stream is linked to the outgoing stream?
  • should akka http continue to process the incoming request even after it has responded with a 200 OK to the client?
  • Alternatively, am I just doing something wrong?

This is using akka version 2.4.3 although I was having the same (or very similar) issues with 2.4.2

Thanks

Steve

EDIT: adding curl command lines for clarity

Standard call (curl includes Expect: 100-continue)

curl --trace-ascii curl-trace.txt --data "@post-test1.html" -X POST http://localhost:8080/test

Call with header overwrite (excludes Expect: 100-continue)

curl -H "Expect:" --trace-ascii curl-trace.txt --data "@post-test1.html" -X POST http://localhost:8080/test
Steve Willcock
  • 26,111
  • 4
  • 43
  • 40
  • Correct me if I'm wrong, but isn't a message that is sent with an `Expect: 100-continune` NOT supposed to contain a payload body? In your example you are sending that `Expect` as well as the body when according to the Http 1.1 spec, no payload is sent on the initial request, and then, if the sever responds with the `100-continue` then the payload is sent. – cmbaxter Apr 04 '16 at 14:06
  • You are right, but that is how it is happening at the moment. The curl command is handling all that internally. curl sends the Expect: 100-continue and then waits for a response from the server before sending the body - this is just how curl works internally. Setting -H "Expect:" on the curl command line turns off the sending of the Expect: 100-continue header and so everything is sent at once. – Steve Willcock Apr 04 '16 at 15:30
  • You seem to think that "Connection: close" means to close the connection immediately. It doesn't: it only means that *after this request has ended*, then the connection cannot be reused for other requests and a new one will have to be opened. – Samuel Tardieu Apr 04 '16 at 15:34
  • OK maybe I should have said "end the request" rather than "close the connection". As far as the server is concerned, the request is ended at this point. The server has sent the 200 OK status code and it never goes on to send the 100-continue so the server seems to be done with the request. The fact that it goes on to accept the body anyway when curl sends it across seems like a fault. – Steve Willcock Apr 04 '16 at 15:42
  • @cmbaxter I have added the curl commands to the original question for clarity – Steve Willcock Apr 04 '16 at 15:43

0 Answers0