Need only the stream for response while using Apache Async http client

Question

I am using Apache Async Http Client to download huge files from Azure Storage.

This is the sample code I am using ->

 CloseableHttpAsyncClient httpclient = HttpAsyncClients.createDefault();
  httpclient.execute(request, new FutureCallback<org.apache.http.HttpResponse>() {
     @Override
     public void completed(final org.apache.http.HttpResponse httpResponse) {
     }

     @Override
     public void failed(final Exception e) {
        future.completeExceptionally(e);
     }

     @Override
     public void cancelled() {
        future.completeExceptionally(new Exception("Request cancelled"));
     }
  });

But this is storing the file in a local buffer before invoking completed callback .

I tried using AsyncByteConsumer ->

AsyncByteConsumer<org.apache.http.HttpResponse>
consumer = new AsyncByteConsumer<org.apache.http.HttpResponse>() {
     @Override
     protected void onByteReceived(ByteBuffer buf, IOControl ioctrl) throws IOException {

     }

     @Override
     protected void onResponseReceived(org.apache.http.HttpResponse response) throws HttpException, IOException {
     }

     @Override
     protected org.apache.http.HttpResponse buildResult(HttpContext context) throws Exception {
        return null;
     }
  };

This also did not work for me. I am getting the following error -> java.lang.IllegalStateException: Content has not been provided

All I want is to get the stream for the response which I will pass on to my clients so that they can download the file directly using the stream.

EDIT 1 ->

So I extended AbstractAsyncResponseConsumer to write my own consumer ->

public abstract  class MyConsumer extends AbstractAsyncResponseConsumer<HttpResponse> {

private volatile HttpResponse response;
private volatile SimpleInputBuffer buf;

public MyConsumer() {
    super();
}

@Override
protected void onResponseReceived(HttpResponse response) {
    this.response = response;
}

@Override
protected void onContentReceived(ContentDecoder decoder, IOControl ioctrl) throws IOException {
    Asserts.notNull(this.buf, "Content buffer");
    System.out.println("onContentReceived");
    buf.consumeContent(decoder);
}

protected abstract void onEntitySet(HttpResponse httpResponse);
@Override
protected void onEntityEnclosed(HttpEntity entity, ContentType contentType) throws IOException {
    System.out.println("onEntityEnclosed");
    long len = entity.getContentLength();
    if (len > Integer.MAX_VALUE) {
        throw new ContentTooLongException("Entity content is too long: " + len);
    }
    if (len < 0) {
        len = 4096;
    }
    this.buf = new SimpleInputBuffer((int) len, new HeapByteBufferAllocator());
    this.response.setEntity(new ContentBufferEntity(entity, this.buf));
    onEntitySet(this.response);


}

@Override
protected HttpResponse buildResult(HttpContext context) throws Exception {
    System.out.println("buildResult");
    return response;

}

@Override
protected void releaseResources() {
    } 
 }

Here is the code I am using to execute a http request

CompletableFuture<HttpResponse> makeRequest() {
            HttpAsyncRequestProducer producer3 = HttpAsyncMethods.create(request);
            CompletableFuture<HttpResponse> future = new CompletableFuture<>();
            httpclient.execute(producer3, new MyConsumer() {
                                   @Override
                                   protected void onEntitySet(HttpResponse httpResponse) {
                                       future.complete(httpResponse);
                                   }
                               },
                               new FutureCallback<HttpResponse>() {
                @Override
                    public void completed(HttpResponse result) {
                        System.out.println("Completed" + result);
                    }

                    @Override
                    public void failed(Exception ex) {

                    }

                    @Override
                    public void cancelled() {

                    }
                });
}


     makeRequest().thenAccept((HttpResponse httpResponse) -> {
         try {
             System.out.println(IOUtils.toString(httpResponse.getEntity().getContent()));
         } catch (IOException e) {
             e.printStackTrace();
         }
     });

I am getting this output ->

onEntityEnclosed .

java.io.IOException: Underlying input stream returned zero bytes .

I am completing the response future as soon as I am getting the onResponseReceived callback which returns the status and headers of the response.

What I believe should happen is that onContentReceived callback will be invoked in a separate thread which will write the buffer data to the stream and my caller thread can read it in a separate thread.

score -1 · Answer 1 · answered May 03 '18 at 12:33

I am not sure why you are having those problems but the following code snippet works for me (using HttpAsyncClient 4.1.3)

CloseableHttpAsyncClient httpClient = HttpAsyncClients.createDefault();
httpClient.start();
final Future<Void> future = httpClient.execute(
        HttpAsyncMethods.createGet("http://httpbin.org/"),
        new AsyncByteConsumer<Void>() {

            @Override
            protected void onByteReceived(final ByteBuffer buf, final IOControl ioctrl) throws IOException {
                System.out.println(buf.remaining());
            }

            @Override
            protected void onResponseReceived(final HttpResponse response) throws HttpException, IOException {
                System.out.println(response.getStatusLine());
            }

            @Override
            protected Void buildResult(final HttpContext context) throws Exception {
                return null;
            }

        },
        null);
future.get();
httpClient.close();

console >

HTTP/1.1 200 OK
1060
8192
3877

I dont need to read the data from the stream. I want to pass the stream to my client which will read it. — lostintranslation, May 04 '18 at 06:40

Need only the stream for response while using Apache Async http client

1 Answers1