1

To pass large amounts of data over binder, we create a pipe, then pass the read end of the pipe over the binder as a ParcelFileDescriptor, and start a thread to write data to the write end of the pipe. It basically looks like this:

  public void writeToParcel(Parcel out, int flags) {
    ParcelFileDescriptor[] fds;
    try {
      fds = ParcelFileDescriptor.createPipe();
    } catch (IOException e) {
      throw new RuntimeException(e);
    }
    out.writeParcelable(fds[0], 0);
    byte[] bytes = ...; // Marshall object data to bytes
    write(bytes, fds[1]); // Starts a thread to write the data
  }

The receiving end reads the data from the read end of the pipe. It looks like this:

ParcelFileDescriptor readFd = in.readFileDescriptor();

FileInputStream fis = new ParcelFileDescriptor.AutoCloseInputStream(readFd);
ByteArrayOutputStream out = new ByteArrayOutputStream();

byte[] b = new byte[16 * 1024];
int n;

try {
  while ((n = fis.read(b)) != -1) {
    out.write(b, 0, n);
  }
} catch (IOException e) {
  throw new RuntimeException(e);
} finally {
  try {
    Log.i(TAG, "Closing read file descriptor..."); // I see this
    fis.close();
    Log.i(TAG, "Closed read file descriptor"); // And I see this
  } catch (IOException e) {
    e.printStackTrace();
  }
}

This works, but when strictmode is enabled, we crash with this:

 01-03 14:26:48.099 E/StrictMode(25346): A resource was acquired at attached stack trace but never released. See java.io.Closeable for information on avoiding resource leaks. 
 01-03 14:26:48.099 E/StrictMode(25346): java.lang.Throwable: Explicit termination method 'close' not called 
 01-03 14:26:48.099 E/StrictMode(25346):    at dalvik.system.CloseGuard.open(CloseGuard.java:223) 
 01-03 14:26:48.099 E/StrictMode(25346):    at android.os.ParcelFileDescriptor.<init>(ParcelFileDescriptor.java:192) 
 01-03 14:26:48.099 E/StrictMode(25346):    at android.os.ParcelFileDescriptor.<init>(ParcelFileDescriptor.java:181) 
 01-03 14:26:48.099 E/StrictMode(25346):    at android.os.ParcelFileDescriptor.createPipe(ParcelFileDescriptor.java:425) 
 01-03 14:26:48.099 E/StrictMode(25346):    at com.clover.sdk.FdParcelable.writeToParcel(FdParcelable.java:118)

Line 118 is the creation of the pipe (ParcelFileDescriptor.createPipe()).

So it seems the sender needs to close the read end as well as the write end. My problem is that I don't know when I can close the read end, as I don't know when the reader will finish reading.

What am I missing?

Jeffrey Blattman
  • 22,176
  • 9
  • 79
  • 134

1 Answers1

1

  1. As soon as you're finished writing close the output stream.

  2. The consumer is always responsible to close their input stream as soon as they're done reading. This is not your responsibility (unless you act as the consumer as well).

What you're describing is analogous to me opening a FileOutputStream (I'm consuming the API) and expecting the runtime to close it for me just because I didn't close it explicitly when I was done using it.

So send the FD in a Parcel and whoever uses it is responsible for closing it. They could use something like this:

val fd = parcel.readFileDescriptor()
val input = ParcelFileDescriptor.AutoCloseInputStream(fd)
// Use input. When #close() is called it will also close the FD.

See: https://developer.android.com/reference/android/os/ParcelFileDescriptor.AutoCloseInputStream

You can hide this implementation in your client SDK library. In any case, document it well for the consumers.

Eugen Pechanec
  • 37,669
  • 7
  • 103
  • 124
  • Thanks. Yes I'm doing that. I'm still getting the strictmode exception on the service side. I'll update my answer to show the receiver-side code. – Jeffrey Blattman Jan 03 '19 at 21:49
  • Also, I'm looking at the impl of `ParcelFileDescriptor`. The closeguard is triggered in the finalizer of that class. It doesn't do any checks w/ regard to the other side closing. It is simply a check if someone has explicitly called `close()` on that instance of `ParcelFileDescriptor`. – Jeffrey Blattman Jan 03 '19 at 21:58
  • So the reference on server side is garbage collected while the actual file descriptor is still in use and valid on the client side. You'll need to keep a strong reference to the FD passed in Parcel. And we're back to the problem of not knowing when the read is finished. You would need to periodically check on the server side if the underlying FD is valid. And that's just not good enough solution, at least in my book. – Eugen Pechanec Jan 03 '19 at 22:11
  • It should be safe to drop FD references when the service is unbound... actually, at that point you should close all FDs anyway. So maintain a set of delivered FDs, close and remove them all in `onUnbind`, and perhaps filter the set by `getFileDescriptor().valid()` in each `onBind`. – Eugen Pechanec Jan 03 '19 at 22:29
  • I did a test where I just closed the read FD on the sender side after the write to the write FD returns. It seems to work and doesn't cause the strictmode violation. Here's some debugging info: https://pastebin.com/hbRm0D4m. You can see that the receiver-side close (when it's done reading) happens at the same millisecond as the sender-side close of the read FD. I tested with data size from ~1k to ~100k. I do worry though that this works because of some specific timing. Needs more thought. – Jeffrey Blattman Jan 03 '19 at 22:51
  • One theory is that the send write doesn't return until the receiver finishes reading. In which case it's okay to close the send side read FD at that point. – Jeffrey Blattman Jan 03 '19 at 22:54
  • 1
    This is what I found: According to a [c pipe example](http://pubs.opengroup.org/onlinepubs/009696899/functions/pipe.html) a process should close unnecessary FDs. That means that your service should close the writing FD after writing data, close the reading FD *after writing it to parcel and delivering that parcel to other process*, and your client should close the reading FD it got after reading. Can you check further? – Eugen Pechanec Jan 03 '19 at 23:25
  • The sticky part is "after delivering it to the other process". I don't really know when the receiver receives it. I think my test is working because I am not closing it until the sender finishes writing, and that seems to be tied to the receiver reading, which obviously means that receiver had received the FD. If you want to put what you found out in the answer I will accept it. – Jeffrey Blattman Jan 03 '19 at 23:36
  • 1
    @JeffreyBlattman "I don't really know when the receiver receives it" - if you use blocking Binder calls (the default), the receiver will always receive the FD by the time the call returns. You can use `Parcelable.PARCELABLE_WRITE_RETURN_VALUE` to make it even more explicit. – user1643723 Jan 04 '19 at 15:56