Synchronize a variable throughout all the threads of a Storm Topology

Question

I have a storm topology to write some data from Kafka queue to Cassandra DB. The program is a multithreaded one. To facilitate the cassandra db insertion, I have this as my DBUtils:

public DBUtils() {
    if(session == null) {
        session = CassandraUtil.getInstance().getSession();
        LOG.info("Started a new session for dbUtils-Monitoring.....");
    }
    synchronized(session) {
        testMapper = new MappingManager(session).mapper(TestVO.class);
    }
}

So, I have used synchronized to create a single dbUtils instance throughout all the running threads. But when I checked the logs, it seems like the session is initializing multiple times. The dbUtils in the storm topology was initialized in prepare method only and it has been used across prepare/execute/clean up methods. So, not sure how to use synchronized block if the variable I want to use throughout all the threads is used in multiple places. My question is then how to make the initialization of session/dbUtils variable only once throughout all the threads.

So actually you want a threadsafe [Singleton](http://stackoverflow.com/documentation/java/130/singletons#t=201608121142368276512)? — Fildor, Aug 12 '16 at 11:42
Yes! I have used 2 solutions till now: one is java.util.concurrent and other is synchronization. They both worked fine with plain java code, but they are not working in case of Storm. Does Storm follow some different multithreading mechanism? — tanmayghosh2507, Aug 12 '16 at 11:58
Actually, I don't know storm. So I cannot really say anything about this. But your snippet is **not** a threadsafe singleton. So even if storm does nothing "fancy" about synchronization, your code will create issues when used in a concurrent context. — Fildor, Aug 12 '16 at 12:05

score 0 · Answer 1 · answered Aug 13 '16 at 10:01

As Storm is a distributed system, you cannot have a single shared variable over all your parallel running bolts. You could only share a variable over executors within a single worker JVM.

For this, you need to create a static variable and use shared/static object instance to synchronize its initialization.

Synchronize a variable throughout all the threads of a Storm Topology

1 Answers1