Stack Overflow
Stack Overflow

Questions tagged [flink-cep]

FlinkCEP - complex event processing for Apache Flink

368 questions
1
vote
2 answers

Checkpointing issues in Flink 1.10.1 using RocksDB state backend

We are experiencing a very difficult-to-observe problem with our Flink job. The Job is reasonably simple, it: Reads messages from Kinesis using the Flink Kinesis connector Keys the messages and distributes them to ~30 different CEP operators, plus…
Jamalarm
  • 71
  • 4
1
vote
0 answers

Why Apache Flink SQL validator is giving NPE for this CEP SQL?

Here is my Flink CEP MATCH_RECOGNIZE sql. SELECT E.* FROM MyEvents MATCH_RECOGNIZE ( ORDER BY procTime MEASURES A.id as id, A.name as name AFTER MATCH SKIP TO NEXT ROW PATTERN (A) DEFINE A AS source='XYZ' and name IN ('EVENT_SRC1',…
ParagM
  • 63
  • 1
  • 7
1
vote
1 answer

How to use Apache Flink CEP SQL for getting events from already matched pattern?

My requirement is to generate trigger based on 2 events(EVT_A & EVT_B independent of order). Here is expectation 1. EVT_A arrived. --> No action 2. EVT_B arrived --> Should Trigger 3. EVT_B arrived --> should Trigger since A was received…
ParagM
  • 63
  • 1
  • 7
1
vote
3 answers

hashCode() and equals() method for custom classes in flink

my doubt is about if custom classes in Flink with Java needs to override or not hashCode() and equals() methods because I have read in this page that hashCode() MUST never be implemented in distributed systems and Apache Flink is one of…
Alter
  • 903
  • 1
  • 11
  • 27
1
vote
1 answer

About StateTtlConfig

I'm configuring my StateTtlConfig for MapState and my interest is the objects into the state has for example 3 hours of life and then they should disappear from state and passed to the GC to be cleaned up and release some memory and the checkpoints…
Alter
  • 903
  • 1
  • 11
  • 27
1
vote
1 answer

How to stop Apache Flink CEP Pattern?

please help me, i've two questions: I read from Apache Kafka json-messages,(then I have steps: deserialization to POJO, filter, keyBy ....) Which is better to use: KeyedProcessFunction (with state, timers, if-else logic blocks) or Flink CEP…
Viacheslav Ryabchikov
  • 21
  • 5
1
vote
2 answers

Apache Flink - How Checkpoint/Savepoint works If we run duplicate jobs (Multi Tenancy)

I have multiple Kafka topics (multi tenancy) and I run the same job run multiple times based on the number of topics with each job consuming messages from one topic. I have configured file system as state backend. Assume there are 3 jobs running.…
Raghavendar
  • 159
  • 1
  • 11
1
vote
1 answer

FLINK ,trigger event based on JSON dynamic input data ( like map object data)

I would like to know if FLINK can support my requirement, I have gone through with lot of articles but not sure if my case can be solved or not Case: I have two input source. a)Event b)ControlSet Event sample data is: event 1- { "id" :100 …
Ashutosh
  • 33
  • 8
1
vote
1 answer

Apache Flink in Kubernetes

Could anyone please let me know how I can setup Flink in my Serverless platform (FaaS) to perform event driven operations? I looked at Flink functions and it seems to be promising. Could anyone clarify on the below? What I need to install in my…
testbg testbg
  • 193
  • 2
  • 11
1
vote
1 answer

Partition the whole dataStream in flink at the start of source and maintain the partition till sink

I am consuming trail logs from a Queue (Apache Pulsar). I use 5 keyedPrcoessFunction and finally sink the payload to Postgres Db. I need ordering per customerId for each of the keyedProcessFunction. Right now I achieve this…
Nischal Kumar
  • 492
  • 7
  • 15
1
vote
0 answers

Unacked messages in RabbitMQ from Flink

I have flink running with RabbitMQ as source, something in my configuration could be causing unacked messages in rabbitmq and then everything is going to crash eventually. Here is my code: Checkpoint config: Configuration conf = new…
Alter
  • 903
  • 1
  • 11
  • 27
1
vote
1 answer

How to detect babbling patterns using the flink cep library

How to detect babbling patterns using the flink cep library ? Example: Let say devices have some problem so, continuously it publishes values like on, off. How to detect patterns using CEP, If a problem exists for 30 mins. Some sample data I…
MadProgrammer
  • 513
  • 5
  • 18
1
vote
1 answer

How can I update a value on MapState without remove the previous value?

My question this time is: working with a MapState, it is safe to use mapstate.put(key, value) to modify the current value for the key into the mapState or I need to do mapState.remove(key) and after that do mapstate.put(key, value) again or there is…
Alter
  • 903
  • 1
  • 11
  • 27
1
vote
1 answer

MapState does not store the previous session with EventTimeSessionWindows in Flink java

I need to compare the previous session to averages from different sessions for the same user. I'm using MapState to keep the previous session, but somehow the mapstate never contains any previous keys, so every session is new. here's my…
Alter
  • 903
  • 1
  • 11
  • 27
1
vote
1 answer

Detect absence of a certain event

In the documentation of FlinkCEP, I found that I can enforce that a particular event doesn't occur between two other events using notFollowedBy or notNext. However, I was wondering If I could detect the absence of a certain event after a time X.…
Humberto
  • 328
  • 3
  • 12
1 2 3
…
24 25