Stack Overflow
Stack Overflow

Questions tagged [flink-cep]

FlinkCEP - complex event processing for Apache Flink

368 questions
0
votes
1 answer

how to auto generate watermark in flink sql table?

I am testing flink cep sql and my watermark is define as row time, my table is a kafka table. As watermark depends on minimal of all the kafka partitions, so every new message has to wait kafka partition align, then cep trigger results. My kafka…
slo
  • 23
  • 5
0
votes
1 answer

Apache Flink - Matching with fields having different values in successive patterns

Consider the use case where we need to find the pattern for a attack like 10 failed logons from the same device and same username followed by a success logon from different device but same username. This should happen within 10 mins. Let us say we…
JDForLife
  • 91
  • 2
  • 10
0
votes
1 answer

Apache Flink - Matching Fields with the same value

We have a use case where we need to find the pattern for brute force like 10 failed logons from the same device and same username followed by a success logon from the same username and same device. This should happen within 10 mins. Let us say we…
JDForLife
  • 91
  • 2
  • 10
0
votes
1 answer

flink cep sql Event Not triggering

I use CEP Pattern in Flink SQL which is working as expected connecting to Kafka broker. But when i connecting to cluster based cloud kafka setup, the Flink CEP is not triggering. Here is my sql: create table agent_action_detail ( agent_id…
slo
  • 23
  • 5
0
votes
1 answer

Send an alert when an event does not occur in a certain time window using siddhi

I am trying to trigger an alert when an event does not occur within a time window after an event came. I know there are a lot of similar questions out there, but nothing seems to match what I am looking for. Let's say I have a time window of 20…
Madhuri Jain
  • 1
  • 2
0
votes
1 answer

Can you access isBackPressured from Apache Flink operator?

I have written a java program for Apache Flink. I want to implement some calculations periodically so I used timers to trigger recalculations. For example, every 10s the implementation spends around 1s (soft threshold) to update the state. Because…
Bnx
  • 1
0
votes
1 answer

Is Flink's Match_Recognize function suitable for capturing this type of pattern?

I'm trying to catch events in pattern described below: Start event = SalePackageA event (Customer A purchasing PackageA) 2-nd event = PackageUsage event (Customer A uses PackageA) 3-rd event = PackageUsage event (Customer A uses PackageA) 4-th…
deeplay
  • 376
  • 3
  • 20
0
votes
1 answer

How to process an already available state based on an event comes in a different stream in flink

We are working on deriving the status of accounts based on the activity on it. We calculate and keep the expiryOn date(which says the tentative, future date on which account expires) based on the user activity on the account. We have a manual date…
Kranthi
  • 37
  • 6
0
votes
1 answer

Share same key with multiple taskmanagers

I have a flink application running on 400 taskmanagers. Within 1hour window time i get same keys more frequently, let's say out of 1500 unique keys 50 keys will occurs more often. This is making like few taskmangers alone process more amount than…
tony stark
  • 13
  • 3
0
votes
1 answer

Apache Flink pattern detection does not find any match

I am trying to catch pattern with Apache Flink CEP (Complex Event Processing) Library. I started with the following structure, and I expect to see 2 matches for ids [1,2] and [3,4]. However I see no result. public class StreamingJob { private…
turkogluc
  • 148
  • 1
  • 10
0
votes
2 answers

Configure RocksDB in flink 1.13

I have read about EmbeddedRocksDBStateBackend in Flink 1.13 version but has size limitations, so I want to keep the current configuration of my previous Flink version 1.11, but the point is that this way of configuring the RocksDB is deprecated (new…
Alter
  • 903
  • 1
  • 11
  • 27
0
votes
1 answer

Why is Flink Cep constantly waiting for a new entry to catch patterns?

I'm making a Flink CEP application that reads data through Kafka. When I try to catch the patterns, the sink operation does not occur when there is no data after it. For example, I expect A-> B-> C as a pattern. And from the kafka comes A, B, C…
Furkan
  • 1
  • 1
0
votes
1 answer

How to use Flink to implement aggregate operations on non-keyed data streams?

I want to implement the following case : 1. select count(1) from user; 2. select count(distinct uid) from user; However, aggregation operations cannot be used for non-keyed streams. A non-keyed stream cannot use keyed state,I don't know what…
lafeier
  • 1
0
votes
0 answers

Whenever I am using tumblingEventWindow in flink for 30 seconds and using a custom trigger but the trigger is not fired at the registered event time

I am using a flink for a tumblingEventOperation and creating a custom trigger to perform operation on it. My tumbling window code- DataStream processedWindowedStreamData = filteredData.assignTimestampsAndWatermarks(new…
Niraj Kumar
  • 51
  • 1
  • 3
0
votes
1 answer

FlinkCEP pattern detection doesn't happen in real time

I'm still new to Flink CEP library and yet I don't understand the pattern detection behavior. Considering the example below, I have a Flink app that consumes data from a kafka topic, data is produced periodically, I want to use Flink CEP pattern to…
Yassin Rebai
  • 3
  • 1
1 2 3
…
24 25