2

I'm developing a Http server that processes fairly large size of payloads. Since Netty provide zero-copy, I thought of using zero-coping of the payload using Netty's zero-copy. But it seems Netty only provide transferTo(WritableByteChannel target, long position) but not providing transferFrom() like method to read the content directly in to a file.

Is that just the way it is or are there any workarounds? thanks.

Sudheera
  • 1,727
  • 1
  • 12
  • 28

2 Answers2

4

Netty 4.0.28+ provides splice support when using the native epoll transport. After proper setup, channel.spliceTo(fileDescriptor, 0, data.length); call in your handler will do the trick. Note that this only works on Linux.

Ersin Er
  • 1,025
  • 7
  • 10
  • 1
    HttpObject object you receive in your handler will be of type HttpRequest (and its method will be POST) for the request which contains the headers as well. You can read content length and then call splice with it and the HttpContent object (again a subtype of HttpObject) which you receive in subsequent handler invocations. If the request is chunked it's a little bit different story. Not sure whether Netty's exposed HTTP API is granular enough to use chunked transfer encoding with splice. You need length of each chunk. This is all in theory, would be glad to see if you come up with a solution. – Ersin Er May 21 '15 at 16:13
  • In the handler implementation you given above, there are no objects are receives because it has overridden the channelActive method (not channel read) – Sudheera May 21 '15 at 16:39
  • 1
    Well, the example in the answer does not use the HTTP codec. You can check the HTTP codec examples here: https://github.com/netty/netty/tree/4.0/example/src/main/java/io/netty/example/http/ – Ersin Er May 21 '15 at 17:00
  • The thing is I want only the payload of the Httprequest splice in to a file and the Header part should be read in to memory. I think doing this is impossible in Netty. what do you think? – Sudheera May 22 '15 at 03:56
2

A Netty channel does not provide an operation that zero-copies the inbound traffic into a file. Alternatively, you can unregister your Netty channel from its event loop, do the zero-copy into the file, and then re-register it to the event loop. However, it is relatively an expensive operation, so its gain might be canceled by unregistration/reregistration overhead.

trustin
  • 12,231
  • 6
  • 42
  • 52
  • thank you for the response. As a workaround is there a way to intercept the bytestream in order to transfer it to a fileChannel. ? – Sudheera May 20 '15 at 06:11
  • 1
    You can always do that in your handler's `channelRead()` method, but it's not zero-copy as you know. – trustin May 21 '15 at 08:09