0

We are developing an application that makes use of deep object models that are being stored in Gemfire. Clients will access a REST service to perform cache operations instead of accessing the cache via the Gemfire client. We are also planning to have a database underneath Gemfire as the system of record making use of asynchronous write-behind to do the DB updates and inserts.

In this scenario it seems to be a non-trivial problem to guarantee that an insert or update into Gemfire will result in a successfull insert or update in the DB without setting up elaborate server-side validation (essentially the constraints to the Gemfire operation would have to match the DB operation constraints). We cannot bubble the DB insert or update success/failure back to the client without making the DB call synchronous to the Gemfire operation. This would obviously defeat the purpose of using Gemfire for low latency client operations.

We are curious as to how other Gemfire adopters who are using write-behind have solved the problem of keeping the DB in sync with the Gemfire data fabric?

Tom
  • 3,006
  • 6
  • 33
  • 54

3 Answers3

2

We generally recommend to implement validations in GemFire in a CacheWriter since by definition it's going to be called before any modification occurs on the cache.

http://gemfire.docs.gopivotal.com/javadocs/japi/com/gemstone/gemfire/cache/CacheWriter.html

That said, a pattern that I saw on a customer was to receive data on region "A" and implement basic validations there, accepting that data. This data will then be copied to the DB using write-behind as you describe, batched through AsyncEventListener where in the try/catch of the insertion, if any error occur they store that data in a another region that has another Listener for the write-behind, non-batched, so you can actually see which record failed and decide what to do accordingly.

Something like:

data -> cacheWriter basic checks -> region A -> AEL (batch 500~N events) -> DB If error occurs copy to region B -> Listener persist individual records on DB -> Do some action on the failed record.

In their case, they had an interface to manually clear pending records on region B.

Hope that helps, if not maybe you can provide more details about your use case...

Cheers.

William Markito
  • 65
  • 1
  • 6
0

You can register an AsyncEventListener and update your DB with batch updates. For more info: http://pubs.vmware.com/vfabric53/topic/com.vmware.vfabric.gemfire.7.0/developing/events/implementing_write_behind_event_handler.html

user2373241
  • 83
  • 6
  • This doesn't answer my question. I'm aware of AsyncEventListener and plan on using this. My question is more about validation during write behind operations. – Tom Nov 23 '13 at 00:31
0

Gemfire ver8 and above solves your problem of accessing Cache objects REST service.

Gemfire REST

vkg
  • 351
  • 4
  • 10