How to bypass the Firebase cache to refresh data (in Android app)?

Question

On an Android application which must works offline most of the time I need, when it's online, to do some synchronous operations for i.e. :

User myUser =  MyclientFacade.getUser();
If (myUser.getScore > 10) {
    DoSomething() 
}

Where User is a POJO filled by Firebase;

The problem occurs when the Firebase cache is activated

Firebase.getDefaultConfig().setPersistenceEnabled(true);

and the user is already in cache and the data are updated on Firebase DB by a third party (or even by another device). Indeed when I query Firebase to get the User I obtain first the data from the cache and later a second change event with the latest data from the Firebase server, but it's too late!

Let's see the synchronous method MyclientFacade.getUser() :

Public User  getUser()  {
  Firebase ref = myFireBaseroot.child("User").child(uid);
  ref.keepSynced(true);
  /* try {
    Thread.sleep(3000);
 } catch (InterruptedException e) {
    e.printStackTrace();
 }*/
final CountDownLatch signal = new CountDownLatch(1);

ref.addListenerForSingleValueEvent(new ValueEventListener() {
//ref.addValueEventListener(new ValueEventListener() {
    @Override
    public void onDataChange(DataSnapshot dataSnapshot) {
       this.user = dataSnapshot.getValue(User.class);
       signal.countDown();
    }
    @Override
    public void onCancelled(FirebaseError firebaseError) {
       signal.countDown();
    }
});
signal.await(5, TimeUnit.SECONDS);
ref.keepSynced(false);
return this.user;
}

I obtain the same behavior if I use addValueEventListener or addListenerForSingleValueEvent mixed with ref.keepSynced:

Let's say my user's score value in cache is 5 and from Firebase DB is 11.

When I call getUser I will obtain the score of 5 (Firebase ask cache first) so I will not call the doSomething() method.

If I uncomment the Thread.sleep() code from my example, the Firebase cache will have enough time to be updated and my getUser will return the correct score value (11).

So how can I directly ask the latest value directly from server side and bypass the cache?

Answer 1

This was a problem that was causing me a lot of stress in my application too.

I tried everything, from changing .addListenerForSingleValueEvent() to .addValueEventListener() to trying to creatively use .keepSynced() to using a delay (the Thread.sleep() method you have described above) and nothing really worked consistently (even the Thread.sleep() method, which wasn't really acceptable in a production app didn't give me consistent results).

So what I did was this: after creating a Query object and calling .keepSynced() on it, I then proceed to write a mock/token object in the node I'm querying and THEN in that operation's completion listener, I do the data retrieval I want to do, after deleting the mock object.

Something like:

 MockObject mock = new MockObject();
    mock.setIdentifier("delete!");

    final Query query = firebase.child("node1").child("node2");

    query.keepSynced(true);

    firebase.child("node1").child("node2").child("refreshMock")
            .setValue(mock, new CompletionListener() {

                @Override
                public void onComplete(FirebaseError error, Firebase afb) {

                    query.addListenerForSingleValueEvent(new ValueEventListener() {

                        public void onDataChange(DataSnapshot data) {

                            // get identifier and recognise that this data
                            // shouldn't be used
                            // it's also probably a good idea to remove the
                            // mock object
                            // using the removeValue() method in its
                            // speficic node so
                            // that the database won't be saddled with a new
                            // one in every
                            // read operation

                        }

                        public void onCancelled(FirebaseError error) {
                        }

                    });

                }

            });
}

This has worked consistently so far for me! (well, for a day or so, so take this with a grain of salt). It seems like doing a write operation before reading somehow bypasses the cache, which makes sense. So the data comes back fresh.

The only downside is the extra write operation before the read operation, which may cause a small delay (obviously use a small object) but if that's the price for always fresh data, I'll take it!

Hope this helps!

Answer 2

A workaround I discovered is using Firebase's runTransaction() method. This appears to always retrieve data from the server.

String firebaseUrl = "/some/user/datapath/";
final Firebase firebaseClient = new Firebase(firebaseUrl);

    // Use runTransaction to bypass cached DataSnapshot
    firebaseClient.runTransaction(new Transaction.Handler() {
        @Override
        public Transaction.Result doTransaction(MutableData mutableData) {
            // Return passed in data
            return Transaction.success(mutableData);
        }

        @Override
        public void onComplete(FirebaseError firebaseError, boolean success, DataSnapshot dataSnapshot) {
            if (firebaseError != null || !success || dataSnapshot == null) {
              System.out.println("Failed to get DataSnapshot");
            } else {
              System.out.println("Successfully get DataSnapshot");
              //handle data here
            }
        }
    });

Answer 3

My Solution was to call Database.database().isPersistenceEnabled = true on load up, then call .keepSynced(true) on any nodes that I needed refreshed.

The issue here is that if you query your node right after .keepSynced(true) then you're likely to get the cache rather than the fresh data. Slightly lame but functional work around: delay your query of the node for a second or so in order to give Firebase some time to get the new data. You'll get the cache instead if the user is offline.

Oh, and if it's a node that you don't want to keep up to date in the background forever, remember to call .keepSynced(false) when you're done.

Answer 4

Just add this code on onCreate method on your application class. (change the database reference)

Example:

public class MyApplication extendes Application{

 @Override
    public void onCreate() {
        super.onCreate();
            DatabaseReference scoresRef = FirebaseDatabase.getInstance().getReference("scores");
            scoresRef.keepSynced(true);
      }
}

Works well to me.

Reference: https://firebase.google.com/docs/database/android/offline-capabilities

Answer 5

I tried both accepted solution and I tried transaction. Transaction solution is cleaner and nicer, but success OnComplete is called only when db is online, so you can't load from cache. But you can abort transaction, then onComplete will be called when offline (with cached data) aswell.

I previously created function which worked only if database got connection lomng enough to do synch. I fixed issue by adding timeout. I will work on this and test if this works. Maybe in the future, when I get free time, I will create android lib and publish it, but by then it is the code in kotlin:

/**
     * @param databaseReference reference to parent database node
     * @param callback callback with mutable list which returns list of objects and boolean if data is from cache
     * @param timeOutInMillis if not set it will wait all the time to get data online. If set - when timeout occurs it will send data from cache if exists
     */
    fun readChildrenOnlineElseLocal(databaseReference: DatabaseReference, callback: ((mutableList: MutableList<@kotlin.UnsafeVariance T>, isDataFromCache: Boolean) -> Unit), timeOutInMillis: Long? = null) {

        var countDownTimer: CountDownTimer? = null

        val transactionHandlerAbort = object : Transaction.Handler { //for cache load
            override fun onComplete(p0: DatabaseError?, p1: Boolean, data: DataSnapshot?) {
                val listOfObjects = ArrayList<T>()
                data?.let {
                    data.children.forEach {
                        val child = it.getValue(aClass)
                        child?.let {
                            listOfObjects.add(child)
                        }
                    }
                }
                callback.invoke(listOfObjects, true)
                removeListener()
            }

            override fun doTransaction(p0: MutableData?): Transaction.Result {
                return Transaction.abort()
            }
        }

        val transactionHandlerSuccess = object : Transaction.Handler { //for online load
            override fun onComplete(p0: DatabaseError?, p1: Boolean, data: DataSnapshot?) {
                countDownTimer?.cancel()
                val listOfObjects = ArrayList<T>()
                data?.let {
                    data.children.forEach {
                        val child = it.getValue(aClass)
                        child?.let {
                            listOfObjects.add(child)
                        }
                    }
                }
                callback.invoke(listOfObjects, false)
                removeListener()
            }

            override fun doTransaction(p0: MutableData?): Transaction.Result {
                return Transaction.success(p0)
            }
        }

If you want to make it faster for offline (to don't wait stupidly with timeout when obviously database is not connected) then check if database is connected before using function above:

DatabaseReference connectedRef = FirebaseDatabase.getInstance().getReference(".info/connected");
connectedRef.addValueEventListener(new ValueEventListener() {
  @Override
  public void onDataChange(DataSnapshot snapshot) {
    boolean connected = snapshot.getValue(Boolean.class);
    if (connected) {
      System.out.println("connected");
    } else {
      System.out.println("not connected");
    }
  }

  @Override
  public void onCancelled(DatabaseError error) {
    System.err.println("Listener was cancelled");
  }
});

Answer 6

Use this line

FirebaseDatabase.getInstance().purgeOutstandingWrites();

Answer 7

On Flutter, here's the way to force get latest snapshot from realtimeDb:

Future<DataSnapshot> _latestRtdbSnapshot(DatabaseReference dbRef) async {
  // ! **NASTY HACK**: Write/delete a child of the query to force an update.
  await dbRef.keepSynced(true);
  await dbRef.child('refreshMock').remove();
  final res = await dbRef.once();
  dbRef.keepSynced(false);
  return res;
}

Answer 8

I know its late but for someone who is facing the same issue i finally found the perfect solution. You can check if the data is coming from the cached memory by using this method:

if (documentSnapShot.metadata.isFromCache){
   // data is coming from the cached memory
} else {
   // data is coming from the server
}

Here's a complete code snippet in Kotlin:

gameRoomRef.addSnapshotListener { documentSnapshot, fireStoreException ->

            fireStoreException?.let {

                return@addSnapshotListener
            }

            documentSnapshot?.let {

                if (it.metadata.isFromCache) 
                   // data is from cached memory
                else 
                   // data is from server
               
            }