How do I prevent OutOfMemory Error when allocating new byte[] for File transfer?

Question

I am trying to read a file in order to send it via Wear App, but I get an OutOfMemory exception.

File file = new File(filePath);
final FileInputStream fileInputStream = new FileInputStream(file);
byte fileContent[] = new byte[(int) file.length()]; //***BOMBS HERE***
fileInputStream.read(fileContent);
fileInputStream.close();
Asset programDataAsset = Asset.createFromBytes(fileContent);

The exception states the following:

java.lang.OutOfMemoryError: Failed to allocate a 31150467 byte allocation with 2097152 free bytes and 16MB until OOM
       at com.rithmio.coach.wear.TransferService.sendAssetToMobile(TransferService.java:110)
       at com.rithmio.coach.wear.TransferService.onHandleIntent(TransferService.java:84)
       at com.rithmio.coach.wear.TransferService$1.run(TransferService.java:60)
       at java.lang.Thread.run(Thread.java:818)

possible duplicate of [Sending big file using FileInputStream/ObjectOutputStream](http://stackoverflow.com/questions/10819516/sending-big-file-using-fileinputstream-objectoutputstream) — Alex S. Diaz, Aug 18 '15 at 00:05
The solution is to read and write in chunks of a fixed size. — MeetTitan, Aug 18 '15 at 00:08
What if he just increases the heap? It's not like he's trying to send 1 GB of data, 30 MB isn't that much. — Fred Porciúncula, Aug 18 '15 at 00:09
I usually find that if your program needs to increase the jvm memory allocation, you might be better off with another solution. After all, it's a portability issue — especially for wearables. Who knows how much memory the consumer will really have? — MeetTitan, Aug 18 '15 at 00:11
And increasing the memory for one specific file isn't a general solution. Sooner or later there will be a file that doesn't fit. — user207421, Aug 18 '15 at 00:20
Not to mention this is likely on a mobile device with limited heap space — user3727843, Aug 18 '15 at 00:21
How much memory can your application actually use? You are copying the entire file into memory, which is not considerate to every other application running as well as being extremely impractical. — Jason, Aug 18 '15 at 00:53
You're using the wrong API. Copying entire files into memory doesn't scale, and is almost always unnecessary. There are three other ways to create an `Asset`. [`Asset.createFromFd()](https://developers.google.com/android/reference/com/google/android/gms/wearable/Asset.html#createFromFd(android.os.ParcelFileDescriptor)) seems the most suitable to me, but I'm no Android expert. — user207421, Aug 18 '15 at 01:12

MeetTitan · Accepted Answer · 2015-10-06T09:47:24.030

Let's solve this using the ChannelApi. We also do not have to worry about chunks like I said in my comment. Google thought ahead and made a convenience method for sending files. public abstract PendingResult sendFile (GoogleApiClient client, Uri uri)

private String pickBestNodeId(List<Node> nodes) {
    String bestNodeId = null;
    // Find a nearby node or pick one arbitrarily
    for (Node node : nodes) {
        if (node.isNearby()) {
            return node.getId();
         }
         bestNodeId = node.getId();
    }
    return bestNodeId;
}

public boolean send(File f) {
    GoogleApiClient mGoogleApiClient = new GoogleApiClient.Builder(this)
    // Request access only to the Wearable API
        .addApi(Wearable.API)
        .build();
    mGoogleApiClient.blockingConnect();
    Channel channel = openChannel(mGoogleApiClient, pickBestNodeId(Wearable.NodeApi.getConnectedNodes(mGoogleApiClient).await()), "/your/arbitrary/application/specific/path/").await(); //NOTE THE PATH IS ARBITRARY, IT CAN BE WHATEVER YOU WANT. IT DOES NOT POINT TO ANYTHING, AND CAN EVEN BE LEFT WITH WHAT I HAVE.
    boolean didSend = channel.sendFile(mGoogleApiClient, f.toURI()).await().isSuccess();
    channel.close(mGoogleApiClient);
    mGoogleApiClient.disconnect();
    return didSend;
}

NOTE: this uses blocking methods, and should not be run in the ui thread.

If you would like the calls to be nonblocking, you should omit my usage of PendingResult.await(), and set the result callback of PendingResult. The callback can be set via setResultCallback(ResultCallback callback).

score 0 · Answer 2 · answered Sep 12 '15 at 18:27

I recommend that you use a ByteArrayOutputStream. Then to read from an InputStream, use this code:

ByteArrayOutputStream stream = new ByteArrayOutputStream();
int b;
while((b = input_stream.read()) != -1) {
    stream.write(b);
}

Then, you can iterate through the bytes with for(byte b : stream.toByteArray()) {...}

user3727843 · Answer 3 · 2015-08-18T00:35:58.617

-1

You should use a buffer rather than attempting to store the entire file as an array. Read in parts of the file into a byte array and then write them out to the stream. This also will prevent OOM errors on devices with less memory.

int fileLength = (int) file.length();
while(fileLength > 0){
    byte[] arr;
    if(fileLength > 1024){
        arr = new byte[1024];
    }else{
        arr = new byte[fileLength];
    fileInputStream.read(arr);
    // write to outputStream/file transfer
    fileLength -= arr.length;
}

edited Aug 18 '15 at 00:35

answered Aug 18 '15 at 00:07

user3727843

574
1
4
13

Do you have an example perhaps? – Kamilski81 Aug 18 '15 at 00:20
Ive edited my answer. To be more specific you would need to give more details about the file transfer – user3727843 Aug 18 '15 at 00:26
This code is not a correct read loop. You shouldn't try to guess the file length. It might change while you're reading. You must read while the return value is positive, and you must store that return value into a variable. Otherwise it is impossible to use the data that was read correctly. At present you're assuming every read returned 1024 byes. It isn't specified to do that, and it certainly won't do that the last time unless the file length is a multiple of 1024. – user207421 Aug 18 '15 at 00:31
A handler will be open on the file so it shouldnt change. But if it did it would cause an issue just the same as the OP's original code would so theres no reason to assume the file will change. – user3727843 Aug 18 '15 at 00:33
You can't assume it *won't* change, and there is no need for the assumption at all. It's poor practice, and easily avoidable. – user207421 Aug 18 '15 at 00:34
Yes but reading a file inherently has race conditions. So although your point is technically valid it has little significance in most cases especially assuming the coder knows the source and all operations on a file. – user3727843 Aug 18 '15 at 00:41
1

The point is that a currently written copy loop doesn't suffer from this problem. Your new code still suffers from this, and it still suffers from the second problem I mentioned, and sprays byte arrays all over the place. The correct way to do this has been posted here hundreds times. It's four lines of code plus two declarations. – user207421 Aug 18 '15 at 01:03
Okay then suggest an edit or enlighten the rest of us with a link rather than commenting that you know how to do something others of us dont :) – user3727843 Aug 18 '15 at 17:33

How do I prevent OutOfMemory Error when allocating new byte[] for File transfer?

3 Answers3