1

I am looking for Kafka Streams complex event processing (CEP). I did not find Kafka streams native CEP. However, I found drools fusion. I have a few questions?

  1. Could Drools Fusion be integrated with Kafka streams? What's the advantage of such solution? What needs attention?

  2. Why there isn't any Kafka Streams official CEP library? Is it because of the existence of the KSQL?

bad_coder
  • 11,289
  • 20
  • 44
  • 72
Paul Yang
  • 11
  • 1
  • The Processor API is as complex as necessary... Anything that interfaces with Java can interact with it (including KSQL)... not clear what you're trying to ask – OneCricketeer Aug 09 '20 at 06:07
  • @OneCricketeer Thanks. I just would like to a kafka streams CEP solution that can solve CEP problems with Kafka streams with configuration of drools syntax. As far as possible do not repeat the invention of wheels. Kafka streams Processor API and drools CEP integration can meet my needs? Or is the integration of the two insignificant? Or rather, what is the normal way to solve the Kafka streams cep? – Paul Yang Aug 09 '20 at 08:09
  • 1) I don't know Drools or CEP 2) Sounds like you are looking for Apache Camel, perhaps 3) Like I said, if Drools can interact with the JVM, then it is "possible" – OneCricketeer Aug 10 '20 at 04:01
  • I just would like to a CEP solution of KafkaStreams. – Paul Yang Aug 10 '20 at 12:13
  • Again, don't know what that exactly means, but looking at [FlinkCEP](https://ci.apache.org/projects/flink/flink-docs-stable/dev/libs/cep.html), Kafka Streams can already do similar actions – OneCricketeer Aug 10 '20 at 15:51
  • The CEP I imagine is that when a program is upgraded, it just needs to be configured, like a Drools rules engine and CEP. Do you mean Kafka Streams have such a function? How KafkaStreams achieved this. – Paul Yang Aug 11 '20 at 07:53
  • What do you mean "upgraded"? Kafka clients have full backwards compatibility down to version 0.10 – OneCricketeer Aug 11 '20 at 15:46

1 Answers1

1

There is no fundamental reason why there is no CEP in Kafka Streams. It was just not in the focus of the community to add so far.

There is an independent CEP project for Kafka Streams thought: https://github.com/fhussonnois/kafkastreams-cep

Last, Apache Kafka is an open source project and if you think adding CEP support out-of-the-box is a necessary feature, feel free to contribute it.

Matthias J. Sax
  • 59,682
  • 7
  • 117
  • 137