Why does app engine bill me less when the below code is wrapped in a transaction?

Question

I have verified this multiple times using appstats. When the below code is NOT wrapped in a transaction, JDO performs two datastore reads and one write, 3 RPC's, at a cost of 240. Not just the first time, every time, even though it is accessing the same record every time hence should be pulling it from cache. However, when I wrap the code in a transaction as above, the code makes 4 RPC's: begin transaction, get, put, and commit -- of these, only the Get is billed as a datastore read, so the overall cost is 70.

If it's pulling it from cache, why would it only bill for a read? It would seem that it would bill for a write, not a read. Could app engine be billing me the same amount for non-transactional cache reads as it does for datastore reads? why?

This is the code WITH transaction:

PersistenceManager pm = PMF.getManager();
Transaction tx = pm.currentTransaction();
String responsetext = "";
try {
  tx.begin();
  Key userkey = obtainUserKeyFromCookie();
  User u = pm.getObjectById(User.class, userkey);
  Key mapkey = obtainMapKeyFromQueryString();
  // this is NOT a java.util.Map, just FYI
  Map currentmap = pm.getObjectById(Map.class, mapkey);
  Text mapData = currentmap.getMapData(); // mapData is JSON stored in the entity
  Text newMapData = parseModifyAndReturn(mapData); // transform the map
  currentmap.setMapData(newMapData); // mutate the Map object
  tx.commit();
  responsetext = "OK";
} catch (JDOCanRetryException jdoe) {
  // log jdoe
  responsetext = "RETRY";
} catch (Exception e) {
  // log e
  responsetext = "ERROR";
} finally {
  if (tx.isActive()) {
    tx.rollback();
  }
  pm.close();
}
resp.getWriter().println(responsetext);

This is the code WITHOUT the transaction:

PersistenceManager pm = PMF.getManager();
String responsetext = "";
try {
  Key userkey = obtainUserKeyFromCookie();
  User u = pm.getObjectById(User.class, userkey);
  Key mapkey = obtainMapKeyFromQueryString();
  // this is NOT a java.util.Map, just FYI
  Map currentmap = pm.getObjectById(Map.class, mapkey);
  Text mapData = currentmap.getMapData(); // mapData is JSON stored in the entity
  Text newMapData = parseModifyAndReturn(mapData); // transform the map
  currentmap.setMapData(newMapData); // mutate the Map object
  responsetext = "OK";
} catch (Exception e) {
  // log e
  responsetext = "ERROR";
} finally {
  pm.close();
}
resp.getWriter().println(responsetext);

As already stated on your other issue, looking at the log tells you what comes from the cache and what doesn't. DataNucleus also has a persistence property "datanucleus.findObject.validateWhenCached" amply documented that extends things beyond the JDO spec. — DataNucleus, Dec 03 '12 at 16:31
@DataNucleusgging did not see anything about the cache in the log, even when I set all log levels to fine in the app engine logging.properties file. Some stuff about state changes and rolling forward unapplied jobs but nothing about the cache. — eeeeaaii, Dec 11 '12 at 10:11

score 2 · Answer 1 · answered Dec 03 '12 at 15:17

With the transaction, the PersistenceManager can know that the caches are valid throughout the processing of that code. Without the transaction, it cannot (it doesn't know whether some other action has come in behind its back and changed things) and so must validate the cache's contents against the DB tables. Each time it checks, it needs to create a transaction to do so; that's a feature of the DB interface itself, where any action that's not in a transaction (with a few DB-specific exceptions) will have a transaction automatically added.

In your case, you should have a transaction anyway, because you want to have a consistent view of the database while you do your processing. Without that, the mapData could be modified by another operation while you're in the middle of working on it and those modifications would be silently lost. That Would Be Bad. (Well, probably.) Transactions are the cure.

^{(You should also look into using AOP for managing the transaction wrapping; that's enormously easier than writing all that transaction management code yourself each time. OTOH, it can add a lot of complexity to deployment until you get things right, so I could understand not following this piece of advice…)}

Sure, I get it that xactions are good, but I'm just trying to understand the behavior. Interestingly, when I set datanucleus.findObject.validateWhenCached = false (as noted by user Datanucleus above) it comes down to to one read (instead of 2) and one write. At any rate, the PM checking the datastore to make sure the object exists is a reasonable explanation for why there are more reads when I'm NOT in a transaction -- but it doesn't explain why, when I DO wrap the code in a transaction, it bills me for a single DS READ when it seems like it should be billing me for a single DS WRITE. — eeeeaaii, Dec 05 '12 at 05:11
I thought about your answer again and it doesn't make sense. It would make sense if I was accessing the same object multiple times in the request, but I'm not. In the request, I access two different items -- the user object and the map object. Unless you are saying that it checks against the datastore when I access fields of the JDO object. — eeeeaaii, Dec 09 '12 at 11:35

Why does app engine bill me less when the below code is wrapped in a transaction?

1 Answers1