Concurrent threads in GemFire CacheWriter

Question

We are currently using Cassandra as NoSQL Database and GemFire as In memory Database. We have been using the GemFire CacheWriter to insert the records in Cassandra. I would like your feedback on whether it’s a good engineering practice to use Concurrent threads in CacheWriter to insert/Update records. Your feedback on this would be appreciated.

public class GenericWriter<K, V> extends CacheWriterAdapter<K, V> implements Declarable {

    private static Logger log = LoggerFactory.getLogger(GenericWriter.class);

    @Autowired
    private CassandraOperations cassandraOperations;

    ExecutorService executor = null;

    @Override
    public void beforeCreate(EntryEvent<K, V> e) {

    executor = Executors.newSingleThreadExecutor();

        executor.submit(() -> {
            if (eventOperation.equals("CREATE") || eventOperation.equalsIgnoreCase("PUTALL_CREATE")) {
                try {
                    cassandraOperations.insert(e.getNewValue());
                } catch (CassandraConnectionFailureException | CassandraWriteTimeoutException
                        | CassandraInternalException cassException) {
                } catch (Exception ex) {
                    log.error("Exception in GenericCacheWriter->" + ExceptionUtils.getStackTrace(ex));
                    throw ex;
                }
            }
        });
        executor.shutdown();
    }

    @Override
    public void init(Properties arg0) {
        // TODO Auto-generated method stub

    }
}

Both answers are correct. Can anyone tell me way to accept both — Vigneshwaran, Aug 24 '17 at 14:39

score 3 · Answer 1 · answered Aug 24 '17 at 07:55

The CacheWriter handler is called synchronously, so the application does not continue until the handler returns. Therefore, is not recommended to execute long-running operations inside this listener. If a long-running operation is needed, consider processing the operation asynchronously through an AsyncEventListener instead.

Using an ExecutorService to delegate the execution to a different thread is possible but it is an anti-pattern, as it no longer implements the fail-fast property, and the handling of the event is no longer synchronous, so its timing would not be guaranteed relative to the application's completion of the event.

You can read more about this topic in the Geode Wiki, specifically in CacheWrite and CacheListener Best Practices.

Hope this helps.

Best regards.

Agreed with this approach; using an AEQ + AsyncEventListener is a much more appropriate solution. This is what is known as "write-behind". See docs for further details... http://gemfire.docs.pivotal.io/geode/developing/events/implementing_write_behind_event_handler.html — John Blum, Aug 25 '17 at 17:49

score 1 · Accepted Answer · answered Aug 24 '17 at 13:37

Yes, it's a fine pattern but remove the Executor and partition your data such that all updates into GemFire go to one and only one node. Partition Cassandra the same way. Put a write lock around the Cassandra update. Use this only when your throughput is low.

If you need high throughput, use the AsyncEventListener and guarantee eventual consistency to your users. If you must use Executors in the AEL, use them in a way so as to throw an exception in the main thread. If the update fails after a number of tries, you write the failed entry to a different region with an expiration of a few seconds or a minute. When that expires, retry the operation. Keep doing this until the succeeds and then and only then, delete the expired entry.

You will need to track version numbers and what you are updating watching old values/ new values if order of updates is important to you or not.

Concurrent threads in GemFire CacheWriter

2 Answers2