Stack Overflow
Stack Overflow

Questions tagged [flink-cep]

FlinkCEP - complex event processing for Apache Flink

368 questions
2
votes
2 answers

Apache Flink union operator giving wrong response

I am applying union operator on two DataStreams of Generic record type. package com.gslab.com.dataSets; import java.io.File; import java.util.ArrayList; import java.util.List; import org.apache.avro.Schema; import…
Ranjit Shinde
  • 71
  • 1
  • 5
2
votes
2 answers

Flink Eclipse JDT Compiler Issue

I am trying to run this program : [Flink CEP Monitoring][1] [1]: https://github.com/tillrohrmann/cep-monitoring on Amazon EC2 with openjdk1.8 & Flink 1.0.2 installed on it. But when I am trying to run this program,it is throwing the following…
AKSHAY SHINGOTE
  • 407
  • 1
  • 8
  • 22
1
vote
0 answers

io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: Deadline expired before operation could complete

In the dataproc cluster I am submitting the Flink job which reads the data from Pubsub. When I am executing the flink job I am getting the below error. We are using the flink version 1.12.5 and dataproc image version is 2.0.45-rocky8 The program…
Nagesh B Viswanadham
  • 27
  • 3
1
vote
1 answer

Flink CEP sql restrict output

I have a use case where I have 2 input topics in kafka. Topic schema: eventName, ingestion_time(will be used as watermark), orderType, orderCountry Data for first topic: {"eventName": "orderCreated", "userId":123, "ingestionTime": "1665042169543",…
user9068199
  • 55
  • 4
1
vote
0 answers

How to get all outputtags from SingleOutputStreamOperator and process them separately for all of different DataStream

I am new to Flink i am doing of pattern evaluation using Flink CEP, as shown below，In the ProcessFunction, it is judged to store different types of AlertEvent in different OutputTags. There are many event types, and the number of events types grows…
rayz_z
  • 11
  • 1
1
vote
1 answer

Apache Flink CEP , pattern not matching

I am new to Flink CEP and trying to test basic things - In below code my expectation is all the input should matched in patter and should print as matched result. But somehow nothing is matching ('matechedStream.print()') any idea about the reason…
MHegde
  • 329
  • 3
  • 14
1
vote
0 answers

Building event based alerting pipeline with Flink

We have a kafka source of events(~5000/sec), and another stream of rules(~5000k/day) created by users for alerting purpose. A rule expires after a day in our system. We need to match each event against 5000k rules and send alert if rule matches. Ex.…
Jack
  • 165
  • 2
  • 10
1
vote
2 answers

Dynamic Job Creation and Submission to Flink

Hi I am planning to use flink as a backend for my feature where we will show a UI to user to graphically create event patterns for eg: Multiple login failures from the same Ip address. We will create the flink pattern programmatically using the…
JDForLife
  • 91
  • 2
  • 10
1
vote
0 answers

How can I work with Apache Flink to detect fraud with data taken from diferent topics from kafka?

I'm new with apache flink and I need to implement a fraud detector with fixed time. Basically what I need is to take data from 3 topics from kafka (UserAuthentication,PasswordChanged and SafeTransaction). when I get events of userAuthentication and…
Anderson Jimenez
  • 45
  • 4
1
vote
1 answer

Flink Timer Timestamp

Do timers in Flink get fired if they are set to a timestamp in the past? Since the current timestamp is greater than that of the timer service, would it get fired immediately or never get fired? Also, we are trying to sort/order input events based…
user2456976
  • 1,248
  • 2
  • 11
  • 19
1
vote
1 answer

Deactivate onTimer on KeyedProcessFunction in Flink

Let's take this as example: I have been called onTimer function in a KeyedProcessFunction based in this concept: (when a == "start" -> ctx.timerService().registerProcessingTimeTimer(some time in long)), but then a new record arrive with this concept…
Alter
  • 903
  • 1
  • 11
  • 27
1
vote
2 answers

How to process DataStream with different Data Types together

let's say that I have a function that process a DataStream and sent the return to DB, but I need to read from another source and when process this new DataStream I will need to find into the states that I could generate before store the…
Alter
  • 903
  • 1
  • 11
  • 27
1
vote
1 answer

Apache Flink updating sql dynamically without restarting

I have query regarding behaviour of Flink. Below is my code snippet. As you can see, some service is supplying list of sql criterias(say about 10k sqls) that Flink is going to execute one by one. My issue is, whenever sql gets updated, how do I…
ParagM
  • 63
  • 1
  • 7
1
vote
2 answers

High Flink network buffer usage, which causes Kafka lagging

Our Flink Jobs contains a filter, key by session id and then session window with 30mins gap. The session window will need to accumulate all the event for the session, and process them using ProcessWindowFunction. We are using Flink 1.9, 128…
lucky_start_izumi
  • 2,511
  • 13
  • 41
  • 61
1
vote
0 answers

Flink: Using MATCH_RECOGNIZE in a VIEW

I'd like to create a view that contains a MATCH_RECOGNIZE statement. This runs into the following error: org.apache.flink.table.api.SqlParserException: SQL parse failed. Encountered "MATCH_RECOGNIZE" at line 3, column 79. Was expecting one of: …
Daniel Lenz
  • 3,334
  • 17
  • 36
1 2 3
…
24 25