Problems with implementing DocumentsProvider for the cloud on Android

Question

I made custom Documents provider implementation where files are actually stored on a server.

I followed the documentation so the provider works most of the time, but I have issues with certain applications, namely:

1) Gmail attaching a file from my provider: Originally my public ParcelFileDescriptor openDocument was something like this:

ParcelFileDescriptor[] pipe=null;
pipe=ParcelFileDescriptor.createPipe();
OutputStream out = new ParcelFileDescriptor.AutoCloseOutputStream(pipe[1]);
new TransferThread(file_id,out).start();
return pipe[0];

but this never worked with Gmail I got error broken pipe. It works if I first download file in the same method (blocking).

Original worked with most of the other applications

2) Blackberry Hub - I cannot get it working in anyway as it seems to be calling the provider methods on the main thread

I would give up but I see the dropbox provider worked with all the apps including BB hub.

Even worse :) it seems that dropbox can show own UI while downloading for example.

Any idea how this can be done?

Please note I checked the Documentation and several examples many times but still cannot get the provider to work with all the apps, I know that some of the apps may do wrong usage but dropbox proves it is possible to meet all of the apps

Answer 1

Your Gmail issue most likely comes from ContentProvider lifecycle.

ContentProvider is actually just IPC Binder, very similar to ones, commonly returned from Service#onBind. But instead of binding to your app, clients obtain it indirectly via ContentResolver each time they make a request. There is usually no explicit unbinding, Android system caches the provider for a short time after each IPC request completes.

Unfortunately, hidden nature of implicit ContentProvider binding means that there is no way to immediately release a provider. Worse yet, there is no way to handle errors in buggy providers — if your ContentProvider crashes before returning Cursor or ParcelFileDescriptor, the calling app will immediately go down with you! Google obviously knows about that, so they created another API for interacting with mis-trusted third-party ContentProviders — ContentProviderClient. Note, that the ContentProviderClient contains both means to handle remote exceptions and process death and a way to explicitly close ContentProvider.

Now imagine the hypothetical Gmail ContentProvider workflow:

ParcelFileDescriptor fd = null;

try (ContentProviderClient c = resolver.acquireUnstableContentProviderClient(...)) {
    fd = c.openFile(...)
} catch (Exception ohThoseBuggyProviders) {
    ...
}

// here ContentProvider is already closed

if (fd != null) {
    // use the received descriptor to create email attachment
    ...
}

But what if your ContentProvider wants to live a bit longer to read the rest of file from server in background thread? Well, the system will likely kill your process anyway, because it does not know, that you want that. Your process dies, Gmail gets the "broken pipe" error.

This is why you should not create new threads or use ContentProvider#openPipeHelper (why does that method even exist?), just do all your work in the calling thread.

---

The answer to second part of your question also lies in ContentProvider internals. When your provider is called from the main thread of calling app, your code is not executed on the main thread of your process — it is executed in the Binder thread pool as usual. But to make programmers' lives easier, Android takes several steps to make that less obvious:

1. The priority of your thread is set to priority of thread, that makes the call (including raising to UI priority, if you get called from UI thread).
2. Android passes current Strict Mode settings (the stuff that makes apps crash when they do network on main thread) from the calling app to your thread. After the call completes, all Strict Mode violations are collected and written to Parcel sent back to the calling app.

Even if ContentProvider ops are executed in binder pool, they behave almost as if there were no bounds between processes — including bad-bad-bad stuff happening when someone tries to download files from UI thread.

You should be able to get rid of that obnoxious "help" by using android.os.StrictMode, but that won't do, if the files in question are too large (ANR may still happen in the calling process). Instead of downloading files to pipe return from openDocument a ParcelFileDescriptor for socket.

Why does Dropbox not suffer from this issue? Because Dropbox Core is written in C++, and Android Strict Mode is currently Java-only construct, it does not hook into into the native code. If you write to disk or download from network in the main thread using C library calls, your app won't receive any repercussions (besides ANR, which is triggered on the UI thread independently from Strict Mode).