Stack Overflow
Stack Overflow

Questions tagged [flink-cep]

FlinkCEP - complex event processing for Apache Flink

368 questions
0
votes
1 answer

Flink streaming window capacity boundaries

I am a beginner on Flink streaming. I am looking at processing events at around 5000 incoming per second, and need to look up an event window of the past 3 days. My question is: where does Flink store its Window data? Would I be limited by the size…
Sonali Gupta
  • 95
  • 1
  • 7
0
votes
1 answer

Global Windows in Flink using Custom Triggers vs Flink CEP Pattern API

I am new to the Apache Flink Api and I am trying to understand the different windows it offers. I have a stream of events such as: device_id,trigger_id,event_time,messageId 1,START,1520433909396,1 1,TRACKING,1520433914398,2 …
user390517
0
votes
1 answer

handling events that do not generate alerts in Apache Flink

I'm using Flink CEP and I need to handle even events that do not generate alerts. Please how can I do it? I'm consuming events from rabbitMq and have defined some patterns. Now what I need to do is to send all the events received in another Queue to…
Said Idrissi
  • 1
  • 2
0
votes
1 answer

How to attach schema to a Flink DataStream - on the fly?

I am dealing with a stream of database mutations, i.e., a change log stream. I want to able to transform the values using a SQL query. I am having difficulty putting together the following three concepts RowTypeInfo, Row, and DataStream. NOTE: I…
user758988
  • 67
  • 1
  • 7
0
votes
1 answer

Flink CEP with files and no streams?

I am trying to use CEP of Flink for log files (as batch job), but not for streams (as realtime). Is that possible ? If yes, do you know examples Scala codes about that ?
Eras Rasmuson
  • 11
  • 1
  • 5
0
votes
1 answer

apache flink aggregation of transaction

I've been trying to figure out how to write a flink program that receives events, from 3 kafka's topics, sum them up for the today, yesterday, and the day before yesterday. so the first question is, how can i sum the transaction for 3 different…
TheEliteOne
  • 105
  • 1
  • 11
0
votes
1 answer

CEP cannot detect multi event with same timestamp?

When I used CEP in Flink, I found a very interesting thing is there is only one event for a time. For example, the DataStream that I used comes from a WindowStream, because it may update the occurrences of a same event, it will generates something…
Leyla Lee
  • 466
  • 5
  • 19
0
votes
0 answers

not able to write a datastream into sqlite3 within a map function in Flink

I have a Flink program which is generating data using a custom generator and at the same time, I want to save that data into SQLite3 database. One way to do this is to parse the data into required format within the map function and then insert data…
Amarjit Dhillon
  • 2,718
  • 3
  • 23
  • 62
0
votes
2 answers

How to apply different operators on a same DataStream based on the event

Context: There is a ds: DataStream[Event] , and there are 3 types of Event: A, B, C. Question: How to monitor occurrences of different Events in different interval of time? i.e. Suppose there is only one event in a second, and it is in order. ds =…
Leyla Lee
  • 466
  • 5
  • 19
0
votes
1 answer

finding the Flink CEP detection latency for each tuple

I have a simple pattern as shown below Pattern pattern = Pattern.begin("s1") .where(new SimpleCondition() { @Override public boolean filter(Event event)…
Amarjit Dhillon
  • 2,718
  • 3
  • 23
  • 62
0
votes
2 answers

Flink CEP greedy matching

I'm having a bit of a battle with the Flink CEP greedy operator. Given the below java code: final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); List strings =…
Stephan Kotze
  • 21
  • 3
0
votes
1 answer

Apache Flink: How to apply patterns from a source onto another datastream?

I've a datastream of events, and another datastream of patterns. The patterns are provided by users at runtime, and they need to come via a Kafka topic. I need to apply each of the pattern on the event stream using Flink-CEP. Is there a way to get a…
wittyameta
  • 375
  • 1
  • 3
  • 16
0
votes
1 answer

Single FLINK CEP Pattern exist at a Time

Is it possible to have single pattern for a key at a time. For e.g. I want to check if I get 5 events which matches a where condition within 1 min window and if I get 10 such events with in a min, I should get 2 alerts but i am getting 6 alerts. It…
Anand Shaw
  • 229
  • 5
  • 13
0
votes
2 answers

How to deal with super size window in FLINK?

Now I have a requirement that aggregate someone's operations in last 3 month. Then put result into ML model to get anomaly. Taking into account the huge flow of the system, it's a very big window. How can I deal with this scenario?
Brutal_JL
  • 2,839
  • 2
  • 21
  • 27
0
votes
1 answer

How to I relate the metrics in Datadog with execution plan operators in Flink?

In my case scenario, Flink is sending the metrics to Datadog. Datadog Host map is as shown below { I have no Idea why is showing me latency here } Flink metrics are sent to localhost. The issue here is that when flink-conf.yaml file configuration…
Amarjit Dhillon
  • 2,718
  • 3
  • 23
  • 62
1 2 3
…
24 25