Optimise thousands of SQLite queries android

Question

I have this task where I need to query all details about every contact and back it up.

The first 770 contacts take up 5-6 seconds to load while after 770 contacts, each contact takes more than a second to query all data. I understand that this is a complex query but that is just too slow. Is there any way to optimize this?

Code

// This contacts array typically contains more than a thousand items in it, even more
contacts.forEach {
val lookupKey = it.lookupKey
queryContacts(lookupKey)
}

fun queryContacts(lookupKey: String) {
    val contact = Contact()
    val contactsCursor = resolver.query(CONTENT_URI, arrayOf(_ID, DISPLAY_NAME, HAS_PHONE_NUMBER), CONTACTS_ALL_DETAILS_SELECTION, arrayOf(lookupKey), null)

    if (contactsCursor.count > 0) {

        while (contactsCursor.moveToNext()) {

            val contactId = contactsCursor.getString(contactsCursor.getColumnIndex(_ID))
            val name = contactsCursor.getString(contactsCursor.getColumnIndex(DISPLAY_NAME))
            contact.name = name

            val hasPhoneNumber = Integer.parseInt(contactsCursor.getString(contactsCursor.getColumnIndex(HAS_PHONE_NUMBER)))

            // This is to read all phone numbers associated with the contact
            val phoneNumbers = arrayListOf<String>()
            if (hasPhoneNumber > 0) {
                val phoneCursor = contentResolver.query(PHONE_CONTENT_URI, null, "$PHONE_CONTACT_ID = ?", arrayOf(contactId), null)
                while (phoneCursor.moveToNext()) {
                    phoneNumbers.add(phoneCursor.getString(phoneCursor.getColumnIndex(NUMBER)))
                }
                phoneCursor.close()
            }
            contact.phoneNumbers.addAll(phoneNumbers)

            // Read every email id associated with the contact
            val emailCursor = contentResolver.query(EMAIL_CONTENT_URI, null, "$EMAIL_CONTACT_ID = ?", arrayOf(contactId), null)

            val emailIds = arrayListOf<String>()
            while (emailCursor.moveToNext()) {

                val email = emailCursor.getString(emailCursor.getColumnIndex(DATA))
                emailIds.add(email)
            }
            emailCursor.close()
            contact.emails.addAll(emailIds)

            val columns = arrayOf(
                    ContactsContract.CommonDataKinds.Event.START_DATE,
                    ContactsContract.CommonDataKinds.Event.TYPE,
                    ContactsContract.CommonDataKinds.Event.MIMETYPE)

            val where = ContactsContract.CommonDataKinds.Event.TYPE + "=" + ContactsContract.CommonDataKinds.Event.TYPE_BIRTHDAY +
                    " and " + ContactsContract.CommonDataKinds.Event.MIMETYPE + " = '" + ContactsContract.CommonDataKinds.Event.CONTENT_ITEM_TYPE + "' and " + ContactsContract.Data.CONTACT_ID + " = " + contactId

            val selectionArgs = arrayOf<String>()
            val sortOrder = ContactsContract.Contacts.DISPLAY_NAME

            val birthdayCur = contentResolver.query(ContactsContract.Data.CONTENT_URI, columns, where, selectionArgs, sortOrder);
            val birthayList = arrayListOf<String>()
            if (birthdayCur.count > 0) {
                while (birthdayCur.moveToNext()) {
                    val birthday = birthdayCur.getString(birthdayCur.getColumnIndex(ContactsContract.CommonDataKinds.Event.START_DATE))
                    birthayList.add(birthday)
                }
            }
            birthdayCur.close()
            contact.dates.addAll(birthayList)
        }

    }

    contactsToBackup.add(contact)
    contactsCursor.close()

}

As you can see, the whole complex query thing going on gets called more than 1000 times. How can I improve this?

Whether this can be optimized depend in part on where and how the backup is stored. Example: if the back-up of the contact data is stored in a MySQL database on a webserver, you could just load the entire SQLite file to the server and let the server handle it. Another option is to simply add a flag column to the contact table of your SQLite database which indicates whether the contact data has been changed and needs to be backed-up. — Barns, May 19 '18 at 18:36
No that part of my service is optimized. I need to optimize the query speed from the Contacts Provider exposed by Android. In here, I am reading the contacts on our phones. — Sriram R, May 20 '18 at 03:33

marmor · Accepted Answer · 2018-09-27T06:47:52.010

you don't really need to optimise thousands of SQLite queries what you do need is migrate those thousands of queries into one big query.

I'm assuming this part:

contacts.forEach {
val lookupKey = it.lookupKey
queryContacts(lookupKey)
}

actually runs over all contacts on the device, if so, you can do this:

Map<Long, Contact> contacts = new HashMap<>();

String[] projection = {Data.CONTACT_ID, Data.DISPLAY_NAME, Data.MIMETYPE, Data.DATA1, Data.DATA2, Data.DATA3};
String selection = Data.MIMETYPE + " IN ('" + Phone.CONTENT_ITEM_TYPE + "', '" + Email.CONTENT_ITEM_TYPE + "', '" + Event.CONTENT_ITEM_TYPE + "')";
Cursor cur = cr.query(Data.CONTENT_URI, projection, selection, null, null);

while (cur != null && cur.moveToNext()) {
    long id = cur.getLong(0);
    String name = cur.getString(1);
    String mime = cur.getString(2); // email / phone / event
    String data = cur.getString(3); // the actual info, e.g. +1-212-555-1234
    int type = cur.getInt(4);

    Log.d(TAG, "got " + id + ", " + name + ", " + data);

    Contact contact;
    if (contacts.containsKey(id)) {
        contact = contacts.get(id);
    } else {
        contact = new Contact();
        contact.name = name;
        contacts.put(id, infos);
    }

    switch (mime) {
        case Phone.CONTENT_ITEM_TYPE: 
            contact.addPhone(data, type); // you'll need to add this method
            break;
        case Email.CONTENT_ITEM_TYPE: 
            contact.addEmail(data, type); // you'll need to add this method
            break;
        case Event.CONTENT_ITEM_TYPE: 
            contact.addBirthday(data, type); // you'll need to add this method
            break;
    }
}

Optimise thousands of SQLite queries android

1 Answers1

Linked