2

I have a client that sends chunked data. My server is expected to read that data. On the server i am using Tomcat 7.0.42 and expecting this data to be loaded via an existing servlet.

I was looking up google to see if i can get any examples that read chunked data, unfortunately i haven't stumbled upon any.

I found few references of ChunkedInputStream provided by Apache Http Client or ChunkedInputFilter provided by Tomcat. But i couldn't find any decent examples on how best to use these.

If any of you guys have any experience with reading/parsing chunked data, please share pointers around those.

Java version used - 1.7.0.45

In my existing servlet code, i have been handling simple request via post using NIO. But now if a client has set transfer encoding to chunked, i need to specifically handle that. So i am having a forking code in place. Something like below,

inputStream = httpServletRequest.getInputStream();

if ("chunked".equals(getRequestHeader(httpServletRequest, "Transfer-Encoding"))) {
   // Need to process chunked data
} else {
   // normal request data
   if (inputStream != null) {
     int contentLength = httpServletRequest.getContentLength()
     if (contentLength <= 0) {
        return new byte[0];
     }
     ReadableByteChannel channel = Channels.newChannel(inputStream);
     byte[] postData = new byte[contentLength];
     ByteBuffer buf = ByteBuffer.allocateDirect(contentLength);
     int numRead = 0;
     int counter = 0;
     while (numRead >= 0) {
        buf.rewind();
        numRead = channel.read(buf);
        buf.rewind();
        for (int i = 0; i < numRead; i++) {
           postData[counter++] = buf.get();
        }
     }
     return postData;
  }
}

So if you observe, the normal request case is based on the "content-length" being available, while for chunked encoding, that is not present. And hence an alternative process to handle chunked data.

Thanks,

Vicky

Victor
  • 1,207
  • 2
  • 13
  • 21
  • 2
    Why do you insist on layering NIO code on top of `java.io` code? There's nothing efficient about doing that: on the contrary. Just use the streams. And are you sure you have to do anything? I would have expected HttpServletRequest to handle chunking out of the box. – user207421 Jan 10 '14 at 07:04

2 Answers2

1

See HTTP 1/1 Chunked Transfer Coding.

You're servlet will be served with chunks of variable size. You'll get the size of each chunk in it's first line. The protocol is quiet simple so you could implement it by yourself.

Markus Malkusch
  • 7,738
  • 2
  • 38
  • 67
  • I know the format and the purpose. Just that, i was looking if there is anything out of the box available like my initial search threw up ChunkedInputStream or ChunkedInputFilter as references. Additionally i want to use NIO api's to do this, since i am using AsyncContext. – Victor Jan 09 '14 at 12:07
  • Apache's `ChunkedInputStream` seems to be a perfect choice. Did you try it? – Markus Malkusch Jan 09 '14 at 12:10
  • Nope, i yet have to try it out. Just wanted to understand from folks what is the best way to do this. If there are any better alternatives. Like ChunkedInputFilter is part of tomcat which i am already using, but i haven't found much documentation around it. – Victor Jan 09 '14 at 12:22
-3

Following NIO based code worked for me,

    ReadableByteChannel channel = Channels.newChannel(chunkedInputStream);

    // content length is not known upfront, hence a initial size
    int bufferLength = 2048;

    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    ByteBuffer byteBuffer = ByteBuffer.allocate(bufferLength);

    int numRead = 0;
    while (numRead >= 0) {
        byteBuffer.rewind();
        //Read bytes from the channel
        numRead = channel.read(byteBuffer);
        byteBuffer.rewind();

        if (numRead > 0) {
            byte[] dataBytes = byteBuffer.array();
            baos.write(dataBytes, 0, dataBytes.length);
        }

        byteBuffer.clear();
    }

    return baos.toByteArray();
Victor
  • 1,207
  • 2
  • 13
  • 21