Error loading large JSON files using Scala Play Framework 2

Question

I'm trying to use Apache Bench to load test a group of large (4MB each) JSON requests. When running with a large file and many concurrent requests I get the following error:

Exception caught in RequestBodyHandler java.nio.channels.ClosedChannelException: null

Here is my ab command:

ab -p large.json -n 1000 -c 10 http://127.0.0.1:9000/json-tests

If I run this with no concurrency and only 10 requests it works fine. Increasing the number of requests or the concurrency causes this error to occur over and over.

My controller currently has no logic in it:

def addJsonTest = Action {
  Ok("OK")
}

Here is the full error:

[error] play - Exception caught in RequestBodyHandler java.nio.channels.ClosedChannelException: null at org.jboss.netty.channel.socket.nio.AbstractNioWorker.setInterestOps(AbstractNioWorker.java:506) [netty-3.9.3.Final.jar:na] at org.jboss.netty.channel.socket.nio.AbstractNioWorker$1.run(AbstractNioWorker.java:455) [netty-3.9.3.Final.jar:na] at org.jboss.netty.channel.socket.ChannelRunnableWrapper.run(ChannelRunnableWrapper.java:40) [netty-3.9.3.Final.jar:na] at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:372) [netty-3.9.3.Final.jar:na] at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:296) [netty-3.9.3.Final.jar:na]

This is just using Play in development mode, is there any setup or configuration for having Play handle multiple large requests?

Thanks!

score 0 · Accepted Answer · edited May 23 '17 at 11:44

You need to do it reactively, with Iterators

  val iteratee = Iteratee.foldM[Array[Byte], Either[Result, String]](Right("start")) { case (str, bytes) =>
      Future.successful(Left(Ok))
  }

  val parser = BodyParser(rh => iteratee)


  def eatDust = Action(parser) { req =>
    Ok
  }

See these links.

https://www.playframework.com/documentation/2.2.x/Iteratees

Play 2.x : Reactive file upload with Iteratees

Error loading large JSON files using Scala Play Framework 2

1 Answers1