Stack Overflow
Stack Overflow

Questions tagged [flink-cep]

FlinkCEP - complex event processing for Apache Flink

368 questions
2
votes
1 answer

Apache Flink - Matching multiple patterns

We have a requirement where multiple events will be matched through the given pattern. Pattern 1 followed by within 10 mins Pattern 2 followed by within 10 mins Pattern 3. Pattern 1: 10 failed logins from the same username and same device within 10…
JDForLife
  • 91
  • 2
  • 10
2
votes
1 answer

Apache Flink delay processing of certain events

I have a requirement to delay processing of some of the events. eg. I have three events (published on Kafka): A (id: 1, retryAt: now) B (id: 2, retryAt: 10 minutes later) C (id: 3, retryAt: now) I need to process record A and C immediately while…
Hamid
  • 717
  • 7
  • 15
2
votes
1 answer

Flink Elasticsearch sink success handler

I use Flink Elasticsearch sink to bulk insert the records to ES. I want to do an operation after the record is successfully synced to Elasticsearch. There is a failureHandler by which we can retry failures. Is there a successHandler in flink…
Vicky
  • 2,999
  • 2
  • 21
  • 36
2
votes
1 answer

If data source doesn't emit data for a while, does the watermark of Flink operator keep advancing?

Just curious, if the data source doesn't emit data for a while, whether Flink operator still receive watermark or not? the Low Watermark in the image will keep advancing or stay still?
Grant
  • 500
  • 1
  • 5
  • 18
2
votes
1 answer

About States and what is better for Flink

Lets assume that I have a job with max.parallelism=4 and a RichFlatMapFunction which is working with MapState. What is the best way to create the MapStateDescriptor? into the RichFlatMapFunction which means that for each instance of this class I…
Alter
  • 903
  • 1
  • 11
  • 27
2
votes
1 answer

Having consumer issues when RocksDB in flink

I have a job which consumes from RabbitMQ, I was using FS State Backend but it seems that the sizes of states became bigger and then I decide to move my states to RocksDB. The issue is that during the first hours running the job is fine, event after…
Alter
  • 903
  • 1
  • 11
  • 27
2
votes
1 answer

What is the best way to keep a HashMap into the state in Flink

I have a Flink job, and one of the statefull operators needs to keep into the state a class that contains a HashMap as attribute, because this hasMap keeps different affinities for a user, example: public class Affinity { public String id; public…
Alter
  • 903
  • 1
  • 11
  • 27
2
votes
1 answer

How to handle exceptions in Apache flink KeyedBroadCastProcessFunction

I am new to Flink i am doing of pattern evaluation using Flink KeyedBroadCastProcessFunction some thing in similar lines to (https://flink.apache.org/2019/06/26/broadcast-state.html) and i am using JAVA to develop my code but i am not getting how…
user13906258
  • 161
  • 1
  • 13
2
votes
1 answer

How to know the status of a flink job from Java?

I have a job running, and I'm interested in to use only one recover retry, because in the meantime that this flink restart is not triggered I have a Thread that try to solve the problem, then when the problem was solved flink will restart, but…
Alter
  • 903
  • 1
  • 11
  • 27
2
votes
1 answer

NullPointer Exception while trying to access or read ReadOnly ctx in processElement method in KeyedBroadCastProcessFunction in Apache Flink

I have some interesting scenario i am working on pattern matching in flink evaluating the incoming patterns using keyedbroadcastprocessfunction, when i am running the program in IDE i am getting null pointer exception in processElements function…
YRK
  • 153
  • 1
  • 1
  • 22
2
votes
1 answer

Flink Process Function is not returning the data to Sideoutputstream

I am trying to validate JSONObject with set of rules if the json matches with set of rules is it will return the matched rule and the JSONObject if not it will return a JSONObject to Sideoutput all this is processed in a ProcessFuntion, i am getting…
YRK
  • 153
  • 1
  • 1
  • 22
2
votes
1 answer

what is the difference between parallelism and parallel computing in Flink?

I have confusion in the number of tasks that can work in parallel in Flink, Can someone explain to me: what is the number of parallelism in a distributed system? and its relation to Flink terminology In Flink, is it the same as we say 2 parallelism…
Maher Marwani
  • 43
  • 1
  • 5
2
votes
1 answer

How to handle transient/application failures in Apache Flink?

My Flink processor listens to Kafka and the business logic in processor involves calling external REST services and there are possibilities that the services may be down. I would like to replay the tuple back into the processor and Is there anyway…
Raghavendar
  • 159
  • 1
  • 11
2
votes
1 answer

Apache Flink job is not scheduled on multiple TaskManagers in Kubernetes (replicas)

I have a simple Flink job that reads from an ActiveMQ source & sink to a database & print. I deploy the job in the Kubernetes with 2 TaskManagers, each having Task Slots of 10 (taskmanager.numberOfTaskSlots: 10). I configured parallelism more than…
vvra
  • 2,832
  • 5
  • 38
  • 82
2
votes
1 answer

Apache Flink: how to use SourceFunction to execute a task at specified interval?

I need my flink job to pull records from a database at specified interval and archive it after processing. I have implemented SourceFunction to fetch the required records from database and added the SourceFunction as the source for…
tweeper
  • 352
  • 1
  • 4
  • 16
1 2
3
…
24 25