Stack Overflow
Stack Overflow

Questions tagged [flink-cep]

FlinkCEP - complex event processing for Apache Flink

368 questions
0
votes
2 answers

Flink Avro 1.8.1 NoSuchMethodError when running on cluster

We run Flink 1.3.0 CEP jobs and rely on Avro 1.8.1 (LogicalType does not exist in Avro 1.7.7) to serialize complex events (as POJOs). It's working when run from within the IDE (IntelliJ), but when we package the jar-file and deploy it to the cluster…
wipatrick
  • 3
  • 5
0
votes
1 answer

flink Table SQL Api

I want to know can we write query using two tables ( join) in flink Table and SQL api. I am new to flik, I want to create two table from two different data set and query them and produce other dataset. my query would be like select... from table1,…
Sanjay Rabari
  • 2,091
  • 1
  • 17
  • 32
0
votes
1 answer

flink sliding window will consider duplicate message?

suppose I am using event time processing and in sliding windows time window 10 and sliding factor 5 seconds. so now for example some message arrive at 8 seconds event time so it will fall in first window-1 1 to 10. now it slides 5 seconds and…
Sanjay Rabari
  • 2,091
  • 1
  • 17
  • 32
0
votes
1 answer

Match or Join event and rule from two kafka topics in one datastream

i want to do join between two kafka topics in one Datastream. In fact the two datastream must have the same id to make the join. Event are the data coming from the sensors and rule contains rules that will be checked with the CEP(coming from User…
Tarek Khal
  • 1
0
votes
0 answers

java.io.NotSerializableException using Apache Flink with Lagom

I am writing Flink CEP program inside the Lagom's Microservice Implementation. My FLINK CEP program run perfectly fine in simple scala application. But when i use this code inside the Lagom service implementation i am receiving the following…
Madiha Khalid
  • 414
  • 3
  • 15
0
votes
0 answers

Apache Flink CEP initialize a input Data Stream before listening to RMQSource

I am using Flink 1.2 CEP to detect missing Heart beat event from a device. I read Heart beat events from RabbitMQ source and using following Pattern to detect missing heart beat of device keyed by serial number by timeout function. This pattern…
Abirami
  • 87
  • 7
0
votes
1 answer

Lagom insert direct stream of events (records) to Cassandra

I am new in Lagom and Persistence Entities Database concepts. I am building Streaming Analytics Engine. My each analysis will work as independent micro-service and for each individual micro-service according to its design philosophy the results will…
Madiha Khalid
  • 414
  • 3
  • 15
0
votes
2 answers

Does Flink JobManager on Yarn need Zookeeper for HA setup

Flink documentation says "When running a highly available YARN cluster, we don’t run multiple JobManager (ApplicationMaster) instances, but only one, which is restarted by YARN on failures.". Then down below "high-availability: zookeeper". I don't…
razvan
  • 559
  • 7
  • 23
0
votes
1 answer

Flink 1.2 does not start in HA Cluster mode

I've installed Flink 1.2 in HA cluster mode 2 JobManagers 1 TaskManager locally and it kept refusing to actually start in this mode showing "Starting cluster." message instead of "Starting HA cluster with 2 masters and 1 peers in ZooKeeper…
razvan
  • 559
  • 7
  • 23
0
votes
1 answer

Flink state backend for TaskManager

I have a Flink v1.2 setup with 1 JobManager, 2 TaskManagers each in it's own VM. I configured the state backend to filesystem and pointed it to a local location in the case of each of the above hosts (state.backend.fs.checkpointdir:…
razvan
  • 559
  • 7
  • 23
0
votes
2 answers

how do do subtype() to tuple in flink cep?

Pattern< Tuple3< String, String, String >, ? > pattern = Pattern. > begin( "start" ) .next( "3" ).where( new FilterFunction< Tuple3< String, String, String > >() { @Override …
陶加涛
  • 11
  • 4
0
votes
1 answer

How to use "every" pattern operator in Flink CEP

In Esper CEP engine event patterns, there's Every operator which indicates that the pattern sub-expression should restart when the sub-expression qualified by the every keyword evaluates to true or false. According to this document, We can use every…
Sina Nourian
  • 128
  • 10
0
votes
1 answer

Linking two events in Flink based on a third event

If two events stream into Flink, can they be logically joined using the information in a third event that follows them (using either the DataStream API or CEP)? E.g., can the third event in the example below be used to link the first two events…
rbinnun
  • 1,169
  • 2
  • 10
  • 18
0
votes
1 answer

Flink emit event when earlier matching event in pair not found

I have two streams of events: one that emits an event to signal the start of an item's lifetime, and another stream that emits an event to signal the end of an item's lifetime. (The streams can be joined on an itemId.) How can I emit a new event in…
epb
  • 454
  • 4
  • 15
0
votes
1 answer

#Apache-flink: use case for Data Management

I am trying to build data management (DM) solution involving high volume data ingestion, pass through some data domain rules, substitution (enrichment), flag the erroneous data before sending it off to downstream system. The rules checking & value…
Cal Ranjan
  • 1
1 2 3
…
24
25