I am experiencing inconsistencies when trying to load all entities of a given kind, using Objectify.
The setup
I am running a Cloud Endpoints backend locally, and using the following method to populate Datastore:
public static void dsUp() {
Store store1 = new Store("H&M", new GeoPt(new Float(56.157702), new Float(10.206938)));
Store store2 = new Store("Marc Jacobs", new GeoPt(new Float(56.158284), new Float(10.208618)));
Store store3 = new Store("Weekday", new GeoPt(new Float(56.158522), new Float(10.207547)));
ofy().save().entities(store1, store2, store3).now(); // Synchronous save to auto generate id
}
After setting up the database, the Datastore Viewer shows all three objects were created:
The problem
I started noticing that my client app (iOS) sometimes only received two stores:
I revisited my getStores() method on the backend, and changed it so that it loads all Stores 4 times, logging each try, and finally returning the results from the first try.
The method (after changing it to multiple load tries), looks like this:
public static List<Store> getStores() {
List<Store> result = ofy().load().type(Store.class).list();
LOGGER.warning("Getting Stores, try 1: " + result.toString());
LOGGER.warning("Getting stores, try 2: " + ofy().load().type(Store.class).list());
LOGGER.warning("Getting stores, try 3: " + ofy().load().type(Store.class).list());
LOGGER.warning("Getting stores, try 4: " + ofy().load().type(Store.class).list());
return result; // Returning the results from try 1
}
Usually, every try returns all three stores. However, sometimes the first try, and even the second try will return only two stores. By the third and fourth try, all stores are returned.
The logged output from one of the method calls looks like this. Notice that this time both try 1 and 2 returned only two stores.
I can not see any pattern for which Store is left out between the database resets. Sometimes it is one, sometimes another. However, it seems like within the same call to getStores(), if try 1 and 2 both fails, it seems to be missing the same store (like in the picture above).
What have I tried?
My initial thought was that I must have set up the Objectify filter wrong. However, it seems to be in order as far as I can tell. In my webapp/WEB-INF/web.xml file, I have the following filter and filter-mapping:
<filter>
<filter-name>ObjectifyFilter</filter-name>
<filter-class>com.googlecode.objectify.ObjectifyFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>ObjectifyFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
I am not very familiar with Objectify's caching yet, but I tried the following:
Adding the following line as the first thing in my getStores() method on the backend, before loading any entities:
ofy().clear(); // Clear the cache
I have also tried using STRONG consistency when loading the entities, as follows:
List<Store> result = ofy().consistency(ReadPolicy.Consistency.STRONG).load().type(Store.class).list();
Other info that might be relevant
I can recreate the inconsistency and it happens about 1/5 times. However, as far as I can tell, it only happens after setting the database up from scratch, using the dsUp() method as mentioned earlier.
Between tries I use the following method to clear the database:
public static void dsDown() {
List<Key<Store>> allStoreKeys = ofy().load().type(Store.class).keys().list();
ofy().delete().keys(allStoreKeys).now();
}
As far as the Datastore Viewer is concerned, the dsDown() method works as intended and leaves the database empty. As far as I understand, deleting entities directly in the Database Viewer could cause a problem where the cache isn't deleted. However I have read that deleting the entities using Objectify like in the dsDown() method circumvents this problem.
If there is any additional info I should provide, please let me know.
Thanks!
EDIT (with my implementation of solution from @saiyr):
As @saiyr notes, the Datastore simulates eventual consistency when running locally, which caused the inconsistency. The following screenshots show how I added the flag to my backend module's build.gradle file.
NB: Note that if you turn the simulated eventual consistency off completely (setting the flag to 0) can cause an error with Objectify transactions. This is why I set the flag to 1 instead. See this stack overflow post for more information on this.
You want to add the flag to the backend project's build.gradle file:
