I been testing this solution for couple of days and it seems OK but i think i need to test it much more. If you using this method, do your own testing and above all, PLEASE LET ME KNOW IF I MISSED ANYTHING and don't be hurry to downgrade. Thx!
- I built an App class that extents Application and implements the
ActivityLifecycleCallbacks. In which i create a ContactSync class for the
first time and activate it everytime the app goes to foregound
- In ContactSync class, i am using Kotlin
withContext(Dispatchers.IO) to suspend any code for easier flow
- I use .get() to get all the contacts from firestore related to current user
- at the .get() addOnSuccessListener, i add all the contacts to a HashMap with the normalized phone number as key and name + firestore id as values (using internal class)
- While making the HashMap i also make sure there are no duplicates on firestore with smae phone number and if so delete them (using batch)
- i then retrieve all the contacts from android phone. I sort them by NORMALIZED_NUMBER first and DISPLAY_NAME (will explain later)
- I am now creating a
batchArray with index and count to avoid exceeding the 500 limit
- I start scanning through the contacts cursor,
- I first get the normalized number, if not available (null) i create it my self using a function i made (it might be that a null value is only returned for phone numbers not in correct format, not sure)
- I then compare the normalized number with previous cursor value. If the same i ignore it to avoid duplicates in firestore (remember the cursor is sorted by NORMALIZED_NUMBER)
- I then check if the normalized number already in HashMap.
- If in HashMap: i compare the name in HashMap to the cursor name. if different, i conclude the name was changed and i update the firestore contact in batch array (remember to increment counter and if exceeds 500 increase index). I then remove the normalized number from the HashMap to avoid its deletion later
- If not in HashMap: i conclude the contact is new and i add it to firestore via batch
- I iterate through all the cursor until completed.
- When cursor complete i close it
- Any remaining records found in HashMap are ones that were not found on firestore hence deleted. I iterate and delete them using batch
- sync is done on the phone side
Now, since making the actual sync needs access to all users, i user firebase functions in node. I create 2 functions:
- function that fires when new user is created (signed via phone)
- function that fires when new contact document is created.
Both functions compare the users to the normalized number in document and if matching, writing the uid of that user to the firestore document "friend_uid" field.
Note you might have errors in these functions if you try to use them in free firebase plan. I suggest changing to Blaze plan and limit the charging to couple of dollars. By changing to Blaze, google also gives you free extras and avoid actual payment
By that, the sync is completed. The sync takes only couple of seconds
To display all the contacts which are users to the app, query all user contacts with "friend_uid" that are not null.
Some extra notes:
- The .get() will retrieve all the contacts every time a sync is made. That might be a lot of reads if user has couple of hundreds contacts. To minimize, i use
.get(Source.DEFAULT) when launching the app and .get(Source.CACHE) for the other times. Since these documents name and number only modified by user, i believe it will not be a problem most of the times (still testing)
- To minimize the sync process as much as possible, i initiate it only if any contact changed its timestamp. I save the last timestamp to SharedPreferences and compare it. I found it mostly saves sync when app re-opened fast.
- I also save the last user logged in. If any change in user, i re-initialize the current user contacts
Some source code (still testing, please let me know if any error):
private fun getContacts(): Cursor? {
val projection = arrayOf(
ContactsContract.CommonDataKinds.Phone._ID,
ContactsContract.CommonDataKinds.Phone.NUMBER,
ContactsContract.CommonDataKinds.Phone.NORMALIZED_NUMBER,
ContactsContract.CommonDataKinds.Phone.DISPLAY_NAME,
ContactsContract.CommonDataKinds.Phone.CONTACT_LAST_UPDATED_TIMESTAMP)
//sort by NORMALIZED_NUMBER to detect duplicates and then by name to keep order and avoiding name change
val sortOrder = ContactsContract.CommonDataKinds.Phone.NORMALIZED_NUMBER + " ASC, " +
ContactsContract.CommonDataKinds.Phone.DISPLAY_NAME + " ASC"
return mContentResolver.query(
ContactsContract.CommonDataKinds.Phone.CONTENT_URI,
projection,
null,
null,
sortOrder)
}
private suspend fun syncContactsAsync() = withContext(Dispatchers.IO) {
if (isAnythingChanged() || mFirstRun) {
if (getValues() == Result.SUCCESS) {
myPrintln("values retrieved success")
} else {
myPrintln("values retrieved failed. Aborting.")
return@withContext
}
val cursor: Cursor? = getContacts()
if (cursor == null) {
myPrintln("cursor cannot be null")
mFireContactHashMap.clear()
return@withContext
}
if (cursor.count == 0) {
cursor.close()
mFireContactHashMap.clear()
myPrintln("cursor empty")
return@withContext
}
var contactName: String?
var internalContact: InternalContact?
val batchArray = mutableListOf(FirebaseFirestore.getInstance().batch())
var batchIndex = 0
var batchCount = 0
var normalizedNumber:String?
var prevNumber = ""
var firestoreId: String
while (cursor.moveToNext()) {
normalizedNumber = cursor.getString(COLUMN_UPDATED_NORMALIZED_NUMBER)
if (normalizedNumber == null) {
normalizedNumber = cursor.getString(COLUMN_UPDATED_PHONE_NUMBER)
normalizedNumber = Phone.getParsedPhoneNumber(mDeviceCountryIso,normalizedNumber,mContext)
}
//cursor sorted by normalized numbers so if same as previous, do not check
if (normalizedNumber != prevNumber) {
prevNumber = normalizedNumber
contactName = cursor.getString(COLUMN_UPDATED_DISPLAY_NAME)
internalContact = mFireContactHashMap[normalizedNumber]
//if phone number exists on firestore
if (internalContact != null) {
//if name changed, update in firestore
if (internalContact.name != contactName) {
myPrintln("updating $normalizedNumber from name: ${internalContact.name} to: $contactName")
batchArray[batchIndex].update(
mFireContactRef.document(internalContact.id),
FireContact.COLUMN_NAME,
contactName)
batchCount++
}
//remove to avoid deletions
mFireContactHashMap.remove(normalizedNumber)
} else {
//New item. Insert
if (normalizedNumber != mUserPhoneNumber) {
myPrintln("adding $normalizedNumber / $contactName")
firestoreId = mFireContactRef.document().id
batchArray[batchIndex].set(mFireContactRef.document(firestoreId),
FireContact(firestoreId, -1, contactName,
cursor.getString(COLUMN_UPDATED_PHONE_NUMBER),
normalizedNumber))
batchCount++
}
}
if (BATCH_HALF_MAX < batchCount ) {
batchArray += FirebaseFirestore.getInstance().batch()
batchCount = 0
batchIndex++
}
}
}
cursor.close()
//Remaining contacts not found on cursor so assumed deleted. Delete from firestore
mFireContactHashMap.forEach { (key, value) ->
myPrintln("deleting ${value.name} / $key")
batchArray[batchIndex].delete(mFireContactRef.document(value.id))
batchCount++
if (BATCH_HALF_MAX < batchCount ) {
batchArray += FirebaseFirestore.getInstance().batch()
batchCount = 0
batchIndex++
}
}
//execute all batches
if ((batchCount > 0) || (batchIndex > 0)) {
myPrintln("committing changes...")
batchArray.forEach { batch ->
batch.commit()
}
} else {
myPrintln("no records to commit")
}
myPrintln("end sync")
mFireContactHashMap.clear()
mPreferenceManager.edit().putLong(PREF_LAST_TIMESTAMP,mLastContactUpdated).apply()
mFirstRun = false
} else {
myPrintln("no change in contacts")
}
}
private suspend fun putAllUserContactsToHashMap() : Result {
var result = Result.FAILED
val batchArray = mutableListOf(FirebaseFirestore.getInstance().batch())
var batchIndex = 0
var batchCount = 0
mFireContactHashMap.clear()
var source = Source.CACHE
if (mFirstRun) {
source = Source.DEFAULT
myPrintln("get contacts via Source.DEFAULT")
} else {
myPrintln("get contacts via Source.CACHE")
}
mFireContactRef.whereEqualTo( FireContact.COLUMN_USER_ID,mUid ).get(source)
.addOnSuccessListener {documents ->
var fireContact : FireContact
for (doc in documents) {
fireContact = doc.toObject(FireContact::class.java)
if (!mFireContactHashMap.containsKey(fireContact.paPho)) {
mFireContactHashMap[fireContact.paPho] = InternalContact(fireContact.na, doc.id)
} else {
myPrintln("duplicate will be removed from firestore: ${fireContact.paPho} / ${fireContact.na} / ${doc.id}")
batchArray[batchIndex].delete(mFireContactRef.document(doc.id))
batchCount++
if (BATCH_HALF_MAX < batchCount) {
batchArray += FirebaseFirestore.getInstance().batch()
batchCount = 0
batchIndex++
}
}
}
result = Result.SUCCESS
}.addOnFailureListener { exception ->
myPrintln("Error getting documents: $exception")
}.await()
//execute all batches
if ((batchCount > 0) || (batchIndex > 0)) {
myPrintln("committing duplicate delete... ")
batchArray.forEach { batch ->
batch.commit()
}
} else {
myPrintln("no duplicates to delete")
}
return result
}
