Android - accessing files in native C/C++ code with Google Scoped Storage API

Question

I need to open files by file name in Android apps within native C/C++ code. The native code are 3rd party libraries that I would prefer not to modify, but they often require file name as an argument to read/write files. With the Google's "scoped storage" API and disabling native access to files in Android 10 or later, it's a real problem.

One well known solution is to get a file descriptor and use "proc/self/fd/FD_NUMER" trick, like:

       ParcelFileDescriptor mParcelFileDescriptor = null;

       String getFileNameThatICanUseInNativeCode(Context context, DocumentFile doc) { 
           try {
                Uri uri = doc.getUri();
                mParcelFileDescriptor =
                        context.getContentResolver().openFileDescriptor(uri, "r");
                if (mParcelFileDescriptor != null) {
                    int fd = mParcelFileDescriptor.getFd();
                    return "/proc/self/fd/" + fd;
                }
            }
            catch (FileNotFoundException fne) {
                return "";
            }
        }

        // Don't forget to close mParcelFileDescriptor when done!

Passing this to native C/C++ code works, but only if the file is in phone main storage. If the user tries to open a file that is on external SD card inserted into the phone slot, it does not work - there is no read permission for the file opened this way. I can only grab the file descriptor int number and use fdopen(fd). But this will require modifying the source code of 3rd party libraries (open source or licensed), and a big headache, whenever the original source of these libraries is updated.

Is there any better solution to that problem? And no, I don't want to hear the solution with adding

android:requestLegacyExternalStorage="true"

to AndroidManifest.xml application section - Google threatens to disable that in the next version of Android in 2020, so a permanent solution is needed. Another easy but dumb solution is copying the entire (maybe huge) file that the user tries to open into private app directory. Dumb and useless...

Answer 1

Update May 19, 2020: just discovered: the READ_EXTERNAL_STORAGE permission lets you read files, but not list directory contents, when running on Android 11 and targeting API 30 (or higher in the future). I submitted it as a bug and just got "This is working as intended" reply (https://issuetracker.google.com/issues/156660903). If anyone cares, please comment there, and also at their "survey": https://google.qualtrics.com/jfe/form/SV_9HOzzyeCIEw0ij3?Source=scoped-storage I have no idea how to proceed developing apps on Android with all these limitations.

Update May 17 2020: Google has finally conceded and is permitting READ_EXTERNAL_STORAGE permission in Android 11 and beyond. After spending months in converting my apps to Storage Access Framework (SAF), I now take a week or two to convert it back, at least the reading files part, to regular file access... Thank you, Google! Thank you sarcastically, for the lost time and effort, and thank you sincerely, for at least partially understanding our point!

So, my previous answer listed below won't be needed anymore, sigh of relief!

After wasting another good day of my life on the Android "Scoped Storage" B.S., I found a solution that works, without modifying the source of 3rd party native libraries, provided that one has the source of these libraries and can build them.

My (very unsatisfactory) solution is: add the following option to C/C++ compile command:

-include "[some/path/]idiocy_fopen_fd.h"

and the idiocy_fopen_fd.h is as follows, as you can see, every call to regular fopen() is replaced by the idiocy_fopen_fd() code, which checks if the file name starts with "/proc/self/fd/", and if so, extracts the file descriptor number and calls fdopen() instead of fopen()... If someone has a better solution, preferably which would work also when you don't have the source code of the 3rd party libs, please share.

#ifndef fopen_fd

#include <stdio.h>
#include <string.h>
#include <unistd.h> // for dup()

#ifdef __cplusplus
extern "C" {
#endif

inline FILE* idiocy_fopen_fd(const char* fname, const char * mode) {
  if (strstr(fname, "/proc/self/fd/") == fname) {
    int fd = atoi(fname + 14);
    if (fd != 0) {
      // Why dup(fd) below: if we called fdopen() on the
      // original fd value, and the native code closes
      // and tries re-open that file, the second fdopen(fd)
      // would fail, return NULL - after closing the
      // original fd received from Android, it's no longer valid.
      FILE *fp = fdopen(dup(fd), mode);
      // Why rewind(fp): if the native code closes and 
      // opens again the file, the file read/write position
      // would not change, because with dup(fd) it's still
      // the same file...
      rewind(fp);
      return fp;
    }
  }
  return fopen(fname, mode);
}
// Note that the above leaves the original file descriptor
// opened when finished - close parcelFileDescriptor in
// Java/Kotlin when your native code returns!

#ifdef __cplusplus
}
#endif

#define fopen idiocy_fopen_fd

#endif

Answer 2

Hi @gregko I managed to get the real path instead of the fd path and send it to my native activity which eases the burden a bit

  String getFileNameThatICanUseInNativeCode(Uri uri) throws FileNotFoundException {
      mParcelFileDescriptor =
              getApplicationContext().getContentResolver().openFileDescriptor(uri, "r");
      if (mParcelFileDescriptor != null) {
        int fd = mParcelFileDescriptor.getFd();
        File file = new File("/proc/self/fd/" + fd);
        String path = null;
        try {
          if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP) {
            path = Os.readlink(file.getAbsolutePath()).toString();
          }
        } catch (ErrnoException e) {
          e.printStackTrace();
        }

        return path;
      }
      else{
        return null;
      }
  }

With this code I basically do a full loop and then I get /storage/CC12-FF343/Games/game.zip instead of /proc/....