Why is DocumentFile so slow, and what should I use instead?

Question

I want to create something like "PDF Viewer app". Application will search for all *.pdf files in location chosen by user. User can choose this folder by this function:

Intent intent = new Intent(Intent.ACTION_OPEN_DOCUMENT_TREE);
startActivityForResult(intent, REQUEST_CODE);

Then I get DocumentFile (folder):

@Override
public void onActivityResult(int requestCode, int resultCode, Intent data) {
    if (resultCode == getActivity().RESULT_OK && requestCode == REQUEST_CODE) {
        Uri uriTree = data.getData();
        DocumentFile documentFile = DocumentFile.fromTreeUri(getActivity(), uriTree);
        //rest of code here
    }
}

Why I chose this method of selecting folder? Because I want to make possible to choose Secondary Storage (you know, in Android >= 5.0, you can't access Secondary Storage with Java.io.file).

Ok, so I get folder with all *.pdf as DocumentFile. Then I call:

for(DocumentFile file: documentFile.listFiles()){
    String fileNameToDisplay = file.getName();
}

And this is VERY SLOW. It takes almost 30 seconds when there are ~600 files in chosen folder. To prove it, I chose directory from External Storage (not secondary storage), and then I tried two solutions: DocumentFile and File. File version looks like it:

File f = new File(Environment.getExternalStorageDirectory()+"/pdffiles");
    for(File file: f.listFiles()){
        String fileNameToDisplay = file.getName();
    }
}

Second version works about 500x faster. There is almost no time in displaying all files on List View.

Why is DocumentFile so slow?

`Why is DocumentFile so slow?`. Wrong question. You should ask 'What should i use instead of DocumentFile?'. Well only if you want something faster ;-). — greenapps, Feb 12 '17 at 12:07
The listFiles() of DocumentFile is already slower then the one of File. But extremely slow is DocumentFile.getName(). Soooo slooow. — greenapps, Feb 12 '17 at 12:15
Yeah, I should have asked like this, but I can't change it right now :( — jdi, Feb 12 '17 at 14:12

score 12 · Answer 1 · answered Feb 20 '21 at 21:10

If you read the source code of TreeDocumentFile, you will find that each call to listFiles() and getName() invokes ContentResolver#query() under the hood. Like CommonsWare said, this would perform hundreds of queries, which is very inefficient.

Here is the source code of listFiles():

@Override
public DocumentFile[] listFiles() {
    final ContentResolver resolver = mContext.getContentResolver();
    final Uri childrenUri = DocumentsContract.buildChildDocumentsUriUsingTree(mUri,
            DocumentsContract.getDocumentId(mUri));
    final ArrayList<Uri> results = new ArrayList<>();
    Cursor c = null;
    try {
        c = resolver.query(childrenUri, new String[] {
                DocumentsContract.Document.COLUMN_DOCUMENT_ID }, null, null, null);
        while (c.moveToNext()) {
            final String documentId = c.getString(0);
            final Uri documentUri = DocumentsContract.buildDocumentUriUsingTree(mUri,
                    documentId);
            results.add(documentUri);
        }
    } catch (Exception e) {
        Log.w(TAG, "Failed query: " + e);
    } finally {
        closeQuietly(c);
    }
    final Uri[] result = results.toArray(new Uri[results.size()]);
    final DocumentFile[] resultFiles = new DocumentFile[result.length];
    for (int i = 0; i < result.length; i++) {
        resultFiles[i] = new TreeDocumentFile(this, mContext, result[i]);
    }
    return resultFiles;
}

In this function call, listFiles() made a query that only selects the document ID column. However, in your case you also want the file name for each file. Therefore, you can add the column COLUMN_DISPLAY_NAME to the query. This would retrieve the filename and document ID (which later you will convert it into Uri) in a single query and is much more efficient. There are also many other columns available such as file type, file size, and last modified time, which you may want to retrieve them as well.

c = resolver.query(mUri, new String[] {
        DocumentsContract.Document.COLUMN_DOCUMENT_ID,
        DocumentsContract.Document.COLUMN_DISPLAY_NAME
    }, null, null, null);

Within the while loop, retrieve the filename by

final String filename = c.getString(1);

The above modified code is able to instantly retrieve the Uri and filename of a directory with 1000+ files.

In summary, my recommendation is to avoid using DocumentFile if you are working with more than just a few files. Instead use ContentResolver#query() to retrieve the Uri and other information by selecting multiple columns in the query. For file operations, use the static methods in the DocumentsContract class by passing the appropriate Uri's.

By the way, it seems that the sortOrder parameter of ContentResolver#query() gets completely ignored in the above code snippet when tested on Android 11 and Android 9. I would manually sort the results instead of relying on the query order.

This should be the accepted answer. In short: do not use DocumentFile. — gxcare, Aug 22 '21 at 11:52

score 3 · Accepted Answer · answered Feb 12 '17 at 12:16

3

Why is DocumentFile so slow?

For ~600 files you are performing ~600 requests of a ContentProvider to get the display name, which means ~600 IPC transactions.

Instead, use MediaStore to query for all indexed media with the application/pdf MIME type.

answered Feb 12 '17 at 12:16

CommonsWare

986,068
189
2,389
2,491

score 2 · Answer 3 · answered Dec 05 '21 at 17:27

2

In case someone comes up here still looking for a solution,
I built a wrapper over this with some pretty good performance.

You can check the wrapper & performance info. here: https://github.com/ItzNotABug/DocumentFileCompat

answered Dec 05 '21 at 17:27

Darshan

4,020
2
18
49

score 1 · Answer 4 · answered Feb 12 '17 at 12:19

1

To obtain a speed little less than the one from File use DocumentsContract instead of DocumentFile to list the content of trees obtained with Intent.ACTION_OPEN_DOCUMENT_TREE.

answered Feb 12 '17 at 12:19

greenapps

11,154
2
16
19

Thanks for answer! But how can i use DocumentsContract to list subdirectories/files? – jdi Feb 23 '17 at 22:16
I think a little googling will do. Less then two months ago listing code was posted here. – greenapps Feb 24 '17 at 08:14
https://stackoverflow.com/a/68789978/9687246 solution – Lokik Soni Mar 11 '23 at 10:39

score 1 · Answer 5 · answered Apr 07 '17 at 05:39

To use DocumentsContract to obtain children documents, see https://developer.android.com/reference/android/provider/DocumentsContract.html#buildChildDocumentsUriUsingTree(android.net.Uri, java.lang.String).

The Uri returned from ACTION_OPEN_DOCUMENT_TREE is a tree document URI. Use the above method to build the Uri to query all children documents.

The root document ID can be obtained using https://developer.android.com/reference/android/provider/DocumentsContract.html#getTreeDocumentId(android.net.Uri) with the Uri returned from ACTION_OPEN_TREE_DOCUMENT.

Hexise · Answer 6 · 2023-06-02T18:46:09.557

Thanks to the answer from @sincostan, I can create a replacement method for DocumentFile.findFile(), which should be much faster than original one(reduced lots of ContentResolver#query() from loop and DocumentFile#getName()). You can CONTINUE to use DocumentFile class, just use this utility method to replace DocumentFile.findFile().

This utility class also provide a sample filter method to filter whatever files you want based on file names, you can customize it based on your needs.

package androidx.documentfile.provider;

import android.content.ContentResolver;
import android.content.Context;
import android.database.Cursor;
import android.net.Uri;
import android.provider.DocumentsContract;
import android.text.TextUtils;
import android.util.Log;

import java.util.ArrayList;
import java.util.List;

public class DocumentUtils {

    public interface IFileFilter {

        boolean accept(String name);

    }

    private static TreeDocumentFile findFile(Context context, TreeDocumentFile file, String name) {
        final ContentResolver resolver = context.getContentResolver();
        final Uri childrenUri = DocumentsContract.buildChildDocumentsUriUsingTree(file.getUri(), DocumentsContract.getDocumentId(file.getUri()));

        Cursor c = null;
        try {
            c = resolver.query(childrenUri, new String[] {DocumentsContract.Document.COLUMN_DOCUMENT_ID, DocumentsContract.Document.COLUMN_DISPLAY_NAME}, null, null, null);
            while (c.moveToNext()) {
                final String documentName = c.getString(1);
                if (TextUtils.equals(name, documentName)) {
                    final String documentId = c.getString(0);
                    final Uri documentUri = DocumentsContract.buildDocumentUriUsingTree(file.getUri(), documentId);
                    return new TreeDocumentFile(file, context, documentUri);
                }
            }
        } catch (Exception e) {
            Log.w("DocumentUtils", "Failed query: " + e);
        } finally {
            if (c != null) {
                c.close();
            }
        }
        return null;
    }

    public static DocumentFile findFile(Context context, DocumentFile documentFile, String name) {
        if (documentFile instanceof TreeDocumentFile)
            return findFile(context, (TreeDocumentFile)documentFile, name);
        return documentFile.findFile(name);
    }

    private static List<DocumentFile> filterFiles(Context context, TreeDocumentFile file, IFileFilter filter) {
        ContentResolver resolver = context.getContentResolver();
        Uri childrenUri = DocumentsContract.buildChildDocumentsUriUsingTree(file.getUri(), DocumentsContract.getDocumentId(file.getUri()));
        List<DocumentFile> filtered = new ArrayList<>();
        Cursor c = null;
        try {
            c = resolver.query(childrenUri, new String[] {DocumentsContract.Document.COLUMN_DOCUMENT_ID, DocumentsContract.Document.COLUMN_DISPLAY_NAME}, null, null, null);
            while (c.moveToNext()) {
                String documentName = c.getString(1);
                String documentId = c.getString(0);
                Uri documentUri = DocumentsContract.buildDocumentUriUsingTree(file.getUri(), documentId);
                TreeDocumentFile child = new TreeDocumentFile(file, context, documentUri);
                if (child.isDirectory())
                    filtered.addAll(filterFiles(context, child, filter));
                else if (filter.accept(documentName))
                    filtered.add(child);
            }
        } catch (Exception e) {
            Log.w("DocumentUtils", "Failed query: " + e);
        } finally {
            if (c != null) {
                c.close();
            }
        }
        return filtered;
    }

    public static List<DocumentFile> filterFiles(Context context, DocumentFile documentFile, IFileFilter filter) {
        if (documentFile instanceof TreeDocumentFile)
            return filterFiles(context, (TreeDocumentFile)documentFile, filter);
        List<DocumentFile> filtered = new ArrayList<>();
        DocumentFile[] files = documentFile.listFiles();
        if (files != null) {
            for (DocumentFile file : files) {
                if (filter.accept(file.getName()))
                    filtered.add(file);
            }
        }
        return filtered;
    }

}

Use the utility class is easy, just change

parentDocumentFile.findFile("name");

to

DocumentUtils.findFile(context, parentDocumentFile, "name");

If you want to filter files based on MIME, such as filter all files with pdf extension, just use following code:

List<DocumentFile> filtered = DocumentUtils.filterFiles(context, parentFile, new IFileFilter() {
    @Override
    public boolean accept(String name) {
        return name != null && name.toLowerCase().endsWith(".pdf");
    }
});

Why is DocumentFile so slow, and what should I use instead?

6 Answers6

Linked

Related