My code throws an OutOfMemoryError when running the following line:
int numBytes = socketChannel.write(_send_buffer);
where socketChannel is an instance of java.nio.channels.SocketChannel
and _send_buffer is an instance of java.nio.ByteBuffer
The code arrives at this point via a non-blocking selector write operation, and throws this on the first attempt to write when the capacity of _send_buffer is large. I have no issues with the code when _send_buffer is less than 20Mb, but when attempting to test this with larger buffers (e.g. > 100Mb) it fails.
According to the docs for java.nio.channels.SocketChannel.write():
An attempt is made to write up to r bytes to the channel, where r is the number of bytes remaining in the buffer, that is, src.remaining(), at the moment this method is invoked. Suppose that a byte sequence of length n is written, where 0 <= n <= r. This byte sequence will be transferred from the buffer starting at index p, where p is the buffer's position at the moment this method is invoked; the index of the last byte written will be p + n - 1. Upon return the buffer's position will be equal to p + n; its limit will not have changed. Unless otherwise specified, a write operation will return only after writing all of the r requested bytes. Some types of channels, depending upon their state, may write only some of the bytes or possibly none at all. A socket channel in non-blocking mode, for example, cannot write any more bytes than are free in the socket's output buffer.
My channels should be setup to be non-blocking, so I would think the write operation should only attempt to write up to the capacity of the socket's output buffer. As I did not previously specify this I tried setting it to 1024 bytes via the setOption method with the SO_SNDBUF option. i.e:
socketChannel.setOption(SO_SNDBUF, 1024);
Though I am still getting the OutOfMemoryError. Here is the full error message:
2021-04-22 11:52:44.260 11591-11733/jp.oist.abcvlib.serverLearning I/.serverLearnin: Clamp target GC heap from 195MB to 192MB
2021-04-22 11:52:44.260 11591-11733/jp.oist.abcvlib.serverLearning I/.serverLearnin: Alloc concurrent copying GC freed 2508(64KB) AllocSpace objects, 0(0B) LOS objects, 10% free, 171MB/192MB, paused 27us total 12.714ms
2021-04-22 11:52:44.261 11591-11733/jp.oist.abcvlib.serverLearning W/.serverLearnin: Throwing OutOfMemoryError "Failed to allocate a 49915610 byte allocation with 21279560 free bytes and 20MB until OOM, target footprint 201326592, growth limit 201326592" (VmSize 5585608 kB)
2021-04-22 11:52:44.261 11591-11733/jp.oist.abcvlib.serverLearning I/.serverLearnin: Starting a blocking GC Alloc
2021-04-22 11:52:44.261 11591-11733/jp.oist.abcvlib.serverLearning I/.serverLearnin: Starting a blocking GC Alloc
Now I can inline debug and stop at the write line and nothing crashes, so I believe there is no problem handling the memory requirement for the _send_buffer itself, but when attempting to write, something in the background is creating another allocation that's too much to handle.
Maybe I'm thinking about this wrong, and need to limit my _send_buffer size to something smaller, but I'd think there should be a way to limit the allocation made by the write command no? Or at least some way to allocate more of the Android memory to my app. I'm using a Pixel 3a, which according to the specs it should have 4GB of RAM. Now I realize that has to be shared with the rest of the system, but this is a bare bones test device (no games, personal apps, etc. are installed) so I'd assume I should have access to a fairly large chunk of that 4GB. As I'm crashing with a growth limit of 201,326,592 (according to the logcat above), it seems strange to me that I'm crashing at 0.2 / 4.0 = 5% of the spec'd memory.
Any tips in the right direction about a fundamental flaw in my approach, or recommendations for avoiding the OutOfMemoryError would be much appreciated!
Edit 1:
Adding some code context as requested by comments. Note this is not a runnable example as the code base is quite large and I am not allowed to share it all due to company policies. Just note that the _send_buffer is has nothing to do with the sendbuffer of the socketChannel itself (i.e. what is referenced by getSendBufferSize, it is just a ByteBuffer that I use to bundle together everything before sending it via the channel. As I can't share all the code related to generating the contents of _send_buffer just note it is a ByteBuffer than can be very large (> 100Mb). If this is fundamentally a problem, then please point this out and why.
So with the above in mind, the NIO related code is pasted below. Note this is very prototype alpha code, so I apologize for the overload of comments and log statements.
SocketConnectionManager.java
(Essentially a Runnable in charge of the Selector)
Note the sendMsgToServer method is overridden (without modification) and called from the main Android activity (not shown). The byte[] episode arg is what gets wrapped into a ByteBuffer within SocketMessage.java (next section) which later gets put into the _send_buffer instance within the write method of SocketMessage.java.
package jp.oist.abcvlib.util;
import android.util.Log;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.SocketOption;
import java.nio.channels.CancelledKeyException;
import java.nio.channels.ClosedSelectorException;
import java.nio.channels.IllegalBlockingModeException;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;
import java.util.Set;
import static java.net.StandardSocketOptions.SO_SNDBUF;
public class SocketConnectionManager implements Runnable{
private SocketChannel sc;
private Selector selector;
private SocketListener socketListener;
private final String TAG = "SocketConnectionManager";
private SocketMessage socketMessage;
private final String serverIp;
private final int serverPort;
public SocketConnectionManager(SocketListener socketListener, String serverIp, int serverPort){
this.socketListener = socketListener;
this.serverIp = serverIp;
this.serverPort = serverPort;
}
@Override
public void run() {
try {
selector = Selector.open();
start_connection(serverIp, serverPort);
do {
int eventCount = selector.select(0);
Set<SelectionKey> events = selector.selectedKeys(); // events is int representing how many keys have changed state
if (eventCount != 0){
Set<SelectionKey> selectedKeys = selector.selectedKeys();
for (SelectionKey selectedKey : selectedKeys){
try{
SocketMessage socketMessage = (SocketMessage) selectedKey.attachment();
socketMessage.process_events(selectedKey);
}catch (ClassCastException e){
Log.e(TAG,"Error", e);
Log.e(TAG, "selectedKey attachment not a SocketMessage type");
}
}
}
} while (selector.isOpen()); //todo remember to close the selector somewhere
} catch (IOException e) {
Log.e(TAG,"Error", e);
}
}
private void start_connection(String serverIp, int serverPort){
try {
InetSocketAddress inetSocketAddress = new InetSocketAddress(serverIp, serverPort);
sc = SocketChannel.open();
sc.configureBlocking(false);
sc.setOption(SO_SNDBUF, 1024);
socketMessage = new SocketMessage(socketListener, sc, selector);
Log.v(TAG, "registering with selector to connect");
int ops = SelectionKey.OP_CONNECT;
sc.register(selector, ops, socketMessage);
Log.d(TAG, "Initializing connection with " + inetSocketAddress);
boolean connected = sc.connect(inetSocketAddress);
Log.v(TAG, "socketChannel.isConnected ? : " + sc.isConnected());
} catch (IOException | ClosedSelectorException | IllegalBlockingModeException
| CancelledKeyException | IllegalArgumentException e) {
Log.e(TAG, "Initial socket connect and registration:", e);
}
}
public void sendMsgToServer(byte[] episode){
boolean writeSuccess = socketMessage.addEpisodeToWriteBuffer(episode);
}
/**
* Should be called prior to exiting app to ensure zombie threads don't remain in memory.
*/
public void close(){
try {
Log.v(TAG, "Closing connection: " + sc.getRemoteAddress());
selector.close();
sc.close();
} catch (IOException e) {
Log.e(TAG,"Error", e);
}
}
}
SocketMessage.java
This is greatly inspired from the example Python code given here, in particular the libclient.py and app-client.py. This is because the server is running python code and clients are running Java. So if you want the reasoning behind why things are the way they are, reference the RealPython socket tutorial. I essentially used the app-server.py as a template for my code, and translated (with modifications) to Java for the clients.
package jp.oist.abcvlib.util;
import android.util.Log;
import org.json.JSONException;
import org.json.JSONObject;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.channels.ClosedChannelException;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;
import java.nio.charset.StandardCharsets;
import java.text.DecimalFormat;
import java.util.Vector;
public class SocketMessage {
private final SocketChannel sc;
private final Selector selector;
private final ByteBuffer _recv_buffer;
private ByteBuffer _send_buffer;
private int _jsonheader_len = 0;
private JSONObject jsonHeaderRead; // Will tell Java at which points in msgContent each model lies (e.g. model1 is from 0 to 1018, model2 is from 1019 to 2034, etc.)
private byte[] jsonHeaderBytes;
private ByteBuffer msgContent; // Should contain ALL model files. Parse to individual files after reading
private final Vector<ByteBuffer> writeBufferVector = new Vector<>(); // List of episodes
private final String TAG = "SocketConnectionManager";
private JSONObject jsonHeaderWrite;
private boolean msgReadComplete = false;
private SocketListener socketListener;
private long socketWriteTimeStart;
private long socketReadTimeStart;
public SocketMessage(SocketListener socketListener, SocketChannel sc, Selector selector){
this.socketListener = socketListener;
this.sc = sc;
this.selector = selector;
this._recv_buffer = ByteBuffer.allocate(1024);
this._send_buffer = ByteBuffer.allocate(1024);
}
public void process_events(SelectionKey selectionKey){
SocketChannel sc = (SocketChannel) selectionKey.channel();
// Log.i(TAG, "process_events");
try{
if (selectionKey.isConnectable()){
sc.finishConnect();
Log.d(TAG, "Finished connecting to " + ((SocketChannel) selectionKey.channel()).getRemoteAddress());
Log.v(TAG, "socketChannel.isConnected ? : " + sc.isConnected());
}
if (selectionKey.isWritable()){
// Log.i(TAG, "write event");
write(selectionKey);
}
if (selectionKey.isReadable()){
// Log.i(TAG, "read event");
read(selectionKey);
// int ops = SelectionKey.OP_WRITE;
// sc.register(selectionKey.selector(), ops, selectionKey.attachment());
}
} catch (ClassCastException | IOException | JSONException e){
Log.e(TAG,"Error", e);
}
}
private void read(SelectionKey selectionKey) throws IOException, JSONException {
SocketChannel socketChannel = (SocketChannel) selectionKey.channel();
while(!msgReadComplete){
// At this point the _recv_buffer should have been cleared (pointer 0 limit=cap, no mark)
int bitsRead = socketChannel.read(_recv_buffer);
if (bitsRead > 0 || _recv_buffer.position() > 0){
if (bitsRead > 0){
// Log.v(TAG, "Read " + bitsRead + " bytes from " + socketChannel.getRemoteAddress());
}
// If you have not determined the length of the header via the 2 byte short protoheader,
// try to determine it, though there is no gaurantee it will have enough bytes. So it may
// pass through this if statement multiple times. Only after it has been read will
// _jsonheader_len have a non-zero length;
if (this._jsonheader_len == 0){
socketReadTimeStart = System.nanoTime();
process_protoheader();
}
// _jsonheader_len will only be larger than 0 if set properly (finished being set).
// jsonHeaderRead will be null until the buffer gathering it has filled and converted it to
// a JSONobject.
else if (this.jsonHeaderRead == null){
process_jsonheader();
}
else if (!msgReadComplete){
process_msgContent(selectionKey);
} else {
Log.e(TAG, "bitsRead but don't know what to do with them");
}
}
}
}
private void write(SelectionKey selectionKey) throws IOException, JSONException {
if (!writeBufferVector.isEmpty()){
SocketChannel socketChannel = (SocketChannel) selectionKey.channel();
Log.v(TAG, "writeBufferVector contains data");
if (jsonHeaderWrite == null){
int numBytesToWrite = writeBufferVector.get(0).limit();
// Create JSONHeader containing length of episode in Bytes
Log.v(TAG, "generating jsonheader");
jsonHeaderWrite = generate_jsonheader(numBytesToWrite);
byte[] jsonBytes = jsonHeaderWrite.toString().getBytes(StandardCharsets.UTF_8);
// Encode length of JSONHeader to first two bytes and write to socketChannel
int jsonLength = jsonBytes.length;
// Add up length of protoHeader, JSONheader and episode bytes
int totalNumBytesToWrite = Integer.BYTES + jsonLength + numBytesToWrite;
// Create new buffer that compiles protoHeader, JsonHeader, and Episode
_send_buffer = ByteBuffer.allocate(totalNumBytesToWrite);
Log.v(TAG, "Assembling _send_buffer");
// Assemble all bytes and flip to prepare to read
_send_buffer.putInt(jsonLength);
_send_buffer.put(jsonBytes);
_send_buffer.put(writeBufferVector.get(0));
_send_buffer.flip();
Log.d(TAG, "Writing to server ...");
// Write Bytes to socketChannel //todo shouldn't be while as should be non-blocking
if (_send_buffer.remaining() > 0){
int numBytes = socketChannel.write(_send_buffer); // todo memory dump error here!
int percentDone = (int) Math.ceil((((double) _send_buffer.limit() - (double) _send_buffer.remaining())
/ (double) _send_buffer.limit()) * 100);
int total = _send_buffer.limit() / 1000000;
// Log.d(TAG, "Sent " + percentDone + "% of " + total + "Mb to " + socketChannel.getRemoteAddress());
}
} else{
// Write Bytes to socketChannel
if (_send_buffer.remaining() > 0){
socketChannel.write(_send_buffer);
}
}
if (_send_buffer.remaining() == 0){
int total = _send_buffer.limit() / 1000000;
double timeTaken = (System.nanoTime() - socketWriteTimeStart) * 10e-10;
DecimalFormat df = new DecimalFormat();
df.setMaximumFractionDigits(2);
Log.i(TAG, "Sent " + total + "Mb in " + df.format(timeTaken) + "s");
// Remove episode from buffer so as to not write it again.
writeBufferVector.remove(0);
// Clear sending buffer
_send_buffer.clear();
// make null so as to catch the initial if statement to write a new one.
jsonHeaderWrite = null;
// Set socket to read now that writing has finished.
Log.d(TAG, "Reading from server ...");
int ops = SelectionKey.OP_READ;
sc.register(selectionKey.selector(), ops, selectionKey.attachment());
}
}
}
private JSONObject generate_jsonheader(int numBytesToWrite) throws JSONException {
JSONObject jsonHeader = new JSONObject();
jsonHeader.put("byteorder", ByteOrder.nativeOrder().toString());
jsonHeader.put("content-length", numBytesToWrite);
jsonHeader.put("content-type", "flatbuffer"); // todo Change to flatbuffer later
jsonHeader.put("content-encoding", "flatbuffer"); //Change to flatbuffer later
return jsonHeader;
}
/**
* recv_buffer may contain 0, 1, or several bytes. If it has more than hdrlen, then process
* the first two bytes to obtain the length of the jsonheader. Else exit this function and
* read from the buffer again until it fills past length hdrlen.
*/
private void process_protoheader() {
Log.v(TAG, "processing protoheader");
int hdrlen = 2;
if (_recv_buffer.position() >= hdrlen){
_recv_buffer.flip(); //pos at 0 and limit set to bitsRead
_jsonheader_len = _recv_buffer.getShort(); // Read 2 bytes converts to short and move pos to 2
// allocate new ByteBuffer to store full jsonheader
jsonHeaderBytes = new byte[_jsonheader_len];
_recv_buffer.compact();
Log.v(TAG, "finished processing protoheader");
}
}
/**
* As with the process_protoheader we will check if _recv_buffer contains enough bytes to
* generate the jsonHeader objects, and if not, leave it alone and read more from socket.
*/
private void process_jsonheader() throws JSONException {
Log.v(TAG, "processing jsonheader");
// If you have enough bytes in the _recv_buffer to write out the jsonHeader
if (_jsonheader_len - _recv_buffer.position() < 0){
_recv_buffer.flip();
_recv_buffer.get(jsonHeaderBytes);
// jsonheaderBuffer should now be full and ready to convert to a JSONobject
jsonHeaderRead = new JSONObject(new String(jsonHeaderBytes));
Log.d(TAG, "JSONheader from server: " + jsonHeaderRead.toString());
try{
int msgLength = (int) jsonHeaderRead.get("content-length");
msgContent = ByteBuffer.allocate(msgLength);
}catch (JSONException e) {
Log.e(TAG, "Couldn't get content-length from jsonHeader sent from server", e);
}
}
// Else return to selector and read more bytes into the _recv_buffer
// If there are any bytes left over (part of the msg) then move them to the front of the buffer
// to prepare for another read from the socket
_recv_buffer.compact();
}
/**
* Here a bit different as it may take multiple full _recv_buffers to fill the msgContent.
* So check if msgContent.remaining is larger than 0 and if so, dump everything from _recv_buffer to it
* @param selectionKey : Used to reference the instance and selector
* @throws ClosedChannelException :
*/
private void process_msgContent(SelectionKey selectionKey) throws IOException {
if (msgContent.remaining() > 0){
_recv_buffer.flip(); //pos at 0 and limit set to bitsRead set ready to read
msgContent.put(_recv_buffer);
_recv_buffer.clear();
}
if (msgContent.remaining() == 0){
// msgContent should now be full and ready to convert to a various model files.
socketListener.onServerReadSuccess(jsonHeaderRead, msgContent);
// Clear for next round of communication
_recv_buffer.clear();
_jsonheader_len = 0;
jsonHeaderRead = null;
msgContent.clear();
int totalBytes = msgContent.capacity() / 1000000;
double timeTaken = (System.nanoTime() - socketReadTimeStart) * 10e-10;
DecimalFormat df = new DecimalFormat();
df.setMaximumFractionDigits(2);
Log.i(TAG, "Entire message containing " + totalBytes + "Mb recv'd in " + df.format(timeTaken) + "s");
msgReadComplete = true;
// Set socket to write now that reading has finished.
int ops = SelectionKey.OP_WRITE;
sc.register(selectionKey.selector(), ops, selectionKey.attachment());
}
}
//todo should send this to the mainactivity listener so it can be customized/overridden
private void onNewMessageFromServer(){
// Take info from JSONheader to parse msgContent into individual model files
// After parsing all models notify MainActivity that models have been updated
}
// todo should be able deal with ByteBuffer from FlatBuffer rather than byte[]
public boolean addEpisodeToWriteBuffer(byte[] episode){
boolean success = false;
try{
ByteBuffer bb = ByteBuffer.wrap(episode);
success = writeBufferVector.add(bb);
Log.v(TAG, "Added data to writeBuffer");
int ops = SelectionKey.OP_WRITE;
socketWriteTimeStart = System.nanoTime();
sc.register(selector, ops, this);
// I want this to trigger the selector that this channel is writeReady.
} catch (NullPointerException | ClosedChannelException e){
Log.e(TAG,"Error", e);
Log.e(TAG, "SocketConnectionManager.data not initialized yet");
}
return success;
}
}
