How to use AsynchronousSocketChannel#read in a loop or recursively?

Question

I found a related question, but it wasn't particularly helpful as it didn't provide a full example.

The problem: how to use AsynchronousSocketChannel for reading data of unknown length using a buffer of a fixed size

First attempt (reads once):

final int bufferSize = 1024;
final SocketAddress address = /*ip:port*/;
final ThreadFactory threadFactory = Executors.defaultThreadFactory();
final ExecutorService executor = Executors.newCachedThreadPool(threadFactory);
final AsynchronousChannelGroup asyncChannelGroup = AsynchronousChannelGroup.withCachedThreadPool(executor, 5);
final AsynchronousSocketChannel client = AsynchronousSocketChannel.open(asyncChannelGroup);
client.connect(address).get(5, TimeUnit.SECONDS);//block until the connection is established

//write the request
Integer bytesWritten = client.write(StandardCharsets.US_ASCII.encode("a custom request in a binary format")).get();

//read the response
final ByteBuffer readTo = ByteBuffer.allocate(bufferSize);
final StringBuilder responseBuilder = new StringBuilder();
client.read(readTo, readTo, new CompletionHandler<Integer, ByteBuffer>() {
        public void completed(Integer bytesRead, ByteBuffer buffer) {
            buffer.flip();
            responseBuilder.append(StandardCharsets.US_ASCII.decode(buffer).toString());
            try {
                client.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
        
        public void failed(Throwable exc, ByteBuffer attachment) {
            try {
                client.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }
);
asyncChannelGroup.awaitTermination(5, TimeUnit.SECONDS);
asyncChannelGroup.shutdown();
System.out.println(responseBuilder.toString());

What changes would I need to make to cleanly implement continuously reading into the buffer while bytesRead != -1 (i.e. end of stream is reached)?

score 1 · Accepted Answer · edited Jun 20 '20 at 09:12

Here's a simplified version of what I ended up doing (using Guava's ListenableFuture):

class SomeUtilClass {
public interface Processor<T> {
    boolean process(Integer byteCount, ByteBuffer buffer);
    T result();
}
public static <T> ListenableFuture<T> read(
    final AsynchronousSocketChannel delegate,
    final Processor<T> processor,
    ByteBuffer buffer
) {
    final SettableFuture<T> resultFuture = SettableFuture.create();
    delegate.read(buffer, buffer, new Handler<T, Integer, ByteBuffer>(resultFuture) {
        public void completed(Integer bytesRead, ByteBuffer buffer) {
            buffer.flip();
            if(processor.process(bytesRead, buffer)) {
                buffer.clear();
                delegate.read(buffer, buffer, this);
            } else {
                resultFuture.set(processor.result());
            }
        }
    });
    return resultFuture;
}
}

Further improvements included using a Commons Pool of ByteBuffers

score 0 · Answer 2 · answered Nov 10 '12 at 22:35

In my mind, the simplest way would be to split this code into its own method, then having the CompletionHandler recursively call that method when bytesRead != -1. That way you can separate out the responsibilities of the code and avoid the necessity of either "busy waiting" or using Thread.sleep() while your asynchronous read is running. You could, of course, also add a case when bytesRead == -1 to do something with the data that has been read in.

score 0 · Answer 3 · answered Nov 10 '12 at 23:12

I wouldn't do anything prolonged from the callback methods failed and completed as they run on threads that aren't under your control.

I understand you want to continue listening for new bytes in the socket even when stream has reached its end (bytesRead == -1). Put the read method in a while(true) loop. Inside it, listen on a synchronized field set by the failed and completed methods. Let's call it myBytesRead.

In order to enable stopping the endless read, replace the while(true) with some other synchronized condition.

private static final BYTES_READ_INIT_VALUE = Integer.MIN_VALUE;
private static final BYTES_READ_COMPLETED_VALUE = -1;
private static final BYTES_READ_FAILED_VALUE = -2;
private Integer myBytesRead = BYTES_READ_INIT_VALUE;

private void setMyBytesRead(final Integer bytesRead) {
    synchronized(myBytesRead) {
        this.myBytesRead = bytesRead;
    }
}

private Integer getMyBytesRead() {
    synchronized(myBytesRead) {
        return myBytesRead;
    }
}

...

// in your method
while (true) {
    final int lastBytesRead = getMyBytesRead();
    if (lastBytesRead == BYTES_READ_FAILED_VALUE) {
        // log failure and retry?
    } else if (lastBytesRead != BYTES_READ_COMPLETED_VALUE) {
        // Thread.sleep( a while ); to avoid exhausting CPU
        continue;
    }

    // else lastBytesRead == BYTES_READ_COMPLETED_VALUE and you can start a new read operation
    client.read(readTo, readTo, new CompletionHandler<Integer, ByteBuffer>() {
            public void completed(Integer bytesRead, ByteBuffer buffer) {
                setMyBytesRead(bytesRead);
                buffer.flip();
                responseBuilder.append(StandardCharsets.US_ASCII.decode(buffer).toString());
                try {
                    client.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }

            public void failed(Throwable exc, ByteBuffer attachment) {
                try {
                    setMyBytesRead(BYTES_READ_FAILED_VALUE);
                    client.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
    );
}

Andrey · Answer 4 · 2012-11-14T20:14:42.240

My initial attempt:

package com.example;

import java.nio.ByteBuffer;
import java.nio.channels.AsynchronousSocketChannel;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;

public class LoopingReader implements Callable<String> {
    final AsynchronousSocketChannel client;
    final String responseTerminator;
    final StringBuilder responseBuilder;

    LoopingReader(
        AsynchronousSocketChannel client,
        String responseTerminator
    ) {
        this.client = client;
        this.responseTerminator = responseTerminator;

        responseBuilder = new StringBuilder();
    }

    public String call() {
        boolean doLoop;
        do {
            int bytesRead = executeIteration();//blocking
            boolean didReachEndOfStream = bytesRead == -1;
            boolean didEncounterResponseTerminator = responseBuilder.indexOf(responseTerminator) != -1;

            doLoop =  !didReachEndOfStream && !didEncounterResponseTerminator;
        } while(doLoop);
        return responseBuilder.toString();
    }

    int executeIteration() {
        final ByteBuffer buffer = ByteBuffer.allocate(256);//use pool here
        final int bytesRead;
        try {
            bytesRead = client.read(buffer).get();
        } catch (InterruptedException | ExecutionException e) {
            throw new IllegalStateException("Failed to read", e);
        }
        decodeAndAppend(buffer);
        return bytesRead;
    }

    void decodeAndAppend(ByteBuffer buffer) {
        buffer.flip();
        responseBuilder.append(StandardCharsets.US_ASCII.decode(buffer));
    }
}

How to use AsynchronousSocketChannel#read in a loop or recursively?

4 Answers4