0

According to the docs (https://docs.wso2.org/display/CEP310/Clustered+Deployment), you can run Siddhi in distributed mode.

As fas as i understand, all nodes will share the streams definitions and operate over a shared context (streams, queries, events...).

But i can't put it to work:

  1. Working with 2.0.0-wso2v4 version
  2. I've coded a simple program that starts two siddhiManagers with distributed processing enabled.
  3. The program creates a stream and a query in siddhiManager1
  4. Adds a stream callback that print events in the two managers (previous stream)
  5. Send an event in siddhiManager1
  6. Prints streams in siddhiManager1 and siddhiManager2

The result is:

  • I can see Hazelcast working, each manager see the other one.
  • siddhiManager1 has one stream, one query and prints one event.
  • siddhiManager2 does not have anything.

I've tested by using siddhi-distribution (fat-jar), but also using siddhi-api, siddhi-core and siddhi-query jars.

Siddhi-distribution fat-jar has some xml inside related to Hazelcast, but as fas as i can see these configs are not being loaded and in the source code (github) i don't see anything special about these files (siddhiManager).

Any idea about how can i run siddhi in distributed cache mode deployment? what am i doing wrong?

user2451444
  • 5
  • 3

1 Answers1

1

Siddhi does not replicate artifacts (queries, stream definitions) through Hazelcast. It shares only events/state of the engine. So in order to get this working, you'll have to first separately synchronize these queries/stream definitions across all Siddhi instances in your distributed setup. Once you have those in place, you'll be able to get it working in distributed mode.

Note that the above explanation is assuming that you are using just the Siddhi library without WSO2 CEP product. In WSO2 CEP, there is a mechanism to synchronize these deployment artifacts among the nodes in a distributed setup.

Rajeev Sampath
  • 2,739
  • 1
  • 16
  • 19
  • Thanks for your answer Rajeev, but then i don't understand the "distributed" deployment mode in siddhi. Let me please ask you a few questions: – user2451444 Apr 07 '14 at 12:20
  • 1) "Only Events/state of the engine is shared when distributed"... so which benefits have this solution? what's the real purpose of distributed mode? or, in other words, why or when i should use this kind of deployment? – user2451444 Apr 07 '14 at 12:22
  • 2) By "events" do you mean stream events? In my tests, neither streams, queries or stream events are seen/shared between engines. – user2451444 Apr 07 '14 at 12:23
  • 3) I can sinchronize queries/stream definitions, but if i have a time window how does it would work in distributed mode? each engine would have its own window? or is there a global window? – user2451444 Apr 07 '14 at 12:24
  • 1. I think the modified answer addresses this question. You can sync the artifacts in wso2 cep. – Rajeev Sampath Apr 07 '14 at 14:59
  • 2. State of the engine will get replicated if you set up the system properly i.e. if node A and node B to have the same state, you'll need them to have same set of artifacts. – Rajeev Sampath Apr 07 '14 at 15:03
  • 3. Nodes will have synchronized copies of the time window, synchronized using hazelcast. So to the user it appears as a global window. – Rajeev Sampath Apr 07 '14 at 15:20