1

I've a use case where I build a graph from data retrieved from multiple data stores. Each of those data stores has its own client library that builds the sub-graph representing that particular data stores data.

Current implementation Since, I didn't have any concurrency requirements, I build a single Graph object in the service layer and pass it to each of those client libraries and they will use the same Graph instance

New implementation - to meet the SLA

To meet the SLA, i want to pull the data from these data stores concurrently.

  1. In this scenario, can each of the client libraries build the sub-graph using the same Graph instance passed from the service layer?
  2. Or is there a better way to handle this?

EDIT

How the object being used

  1. Client sends a REST request to pull person data
  2. Person data is stored in 3 different data stores
  3. Service layer creates an instance of com.tinkerpop.blueprints.impls.tg.TinkerGraph and shares it among 3 different threads retreiving the data from 3 different store simultaneously on 3 different threads. Each thread also responsible for adding the data pulled to the shared Graph instance.
Aravind Yarram
  • 78,777
  • 46
  • 231
  • 327

1 Answers1

2

It depends on the Graph interface implementation. For example, if you are using TinkerGraph it makes no guarantees of thread-safety. Neo4jGraph uses a ThreadLocal variable to manage transactions which means that you can share the same Neo4jGraph instance with multiple threads without any trouble. Blueprints also allows for multi-threaded transactions via ThreadedTransactionalGraph:

https://github.com/tinkerpop/blueprints/wiki/Graph-Transactions#threadedtransactionalgraph-and-multi-threads-for-one-transaction

but you have to find a Graph that supports that function.

stephen mallette
  • 45,298
  • 5
  • 67
  • 135
  • plus one. I am using using the "com.tinkerpop.blueprints.impls.tg.TinkerGraph" obj to represent the graph in memory. Please see my EDITs section on the question. Is this object thread-safe or should I encapsulate it in ThreadedGraph? – Aravind Yarram Feb 03 '15 at 19:30
  • I would be cautious on writes to `TinkerGraph` from different threads. I can't say what the problems would be, but I would not make guarantees that you would not have problems doing that. I would probably spawn the three separate threads to just get the data from the different stores, but then construct your `TinkerGraph` in a single thread from the gathered data. – stephen mallette Feb 03 '15 at 21:45
  • I had this as a backup plan if "com.tinkerpop.blueprints.impls.tg.TinkerGraph" isn't thread-safe. Thanks for confirming about the thread-safety guarantee. The backup plan is now the only choice :-) – Aravind Yarram Feb 04 '15 at 02:08
  • What does it take to add transaction support to TinkerGraph? I can submit a patch. I need this feature. Also, what is the specific thread-safety concern here? – Aravind Yarram Jul 02 '15 at 03:39
  • Please note that Blueprints is largely in maintenance mode at this point and so any development there is mostly focused on bug fixes. All focus is on TP3: http://bit.ly/1LJVBut At this point, I'd suggest that you fork TinkerGraph if you require that feature in TP2. In TP3, TinkerGraph has similar limitations as it is meant to be "simple" and for lightweight operations. – stephen mallette Jul 02 '15 at 12:10
  • I'm not sure TinkerGraph should have full transaction support, but having guarantees around thread-safety would be nice. You might want to bring this topic up on the dev mailing list if you would like to submit a patch of some sort for TP3: http://bit.ly/1f4HSkQ – stephen mallette Jul 02 '15 at 12:10
  • will post it on the mailing list. – Aravind Yarram Jul 02 '15 at 13:30