Stack Overflow
Stack Overflow

Questions tagged [event-stream-processing]

43 questions
1
vote
1 answer

How to code an async stream generator in javascript without a buffer

We want to consume stream data events from a generator into a transformation/transduction pipeline. One form of a generators is the following var generator = async function* () { var value while(true){ value = await new…
Phil Tomlinson
  • 11
  • 2
1
vote
1 answer

"No Extension exists for streamingml" error in WSO2 SP

When I try using #streamingml extension in WSO2 SP, I get the error that extension is not available. However I have made sure that siddhi-execution-streamingml-1.0.15 is downloaded and placed within the /lib directory. I am running this on Ubuntu…
amith hegde
  • 13
  • 2
1
vote
1 answer

How do I process an incoming event and based on a field in the event, write into different streams using wso2?

I am trying to use a single stream, process the incoming json format and write into different streams based on an attribute in the event. For example if the input stream consists of something like this: { "event_type" : "temperature", "json" : { …
SCK
  • 41
  • 4
1
vote
3 answers

No extension exist for esbAnalytics:decompress

Iam trying to decompress incoming events with siddhi app on WSO2 Stream Processor. On earlier version (e.g. DAS) there was siddhi extension esbAnalytics:decompress, but if I use this extension in SP, there is error "No extension exist for…
Dominik Zatloukal
  • 193
  • 1
  • 3
  • 12
1
vote
1 answer

Apache Flink Error Handing and Conditional Processing

I am new to Flink and have gone through site(s)/examples/blogs to get started. I am struggling with the correct use of operators. Basically I have 2 questions Question 1: Does Flink support declarative exception handling, I need to handle…
Aurvoir
  • 267
  • 3
  • 12
1
vote
1 answer

Approaches on ingesting IoT data from cloud gateway

I would like to hear your insights about an IoT data ingesting case. In AWS IoT hub, thing shadows are virtual representation of physical ones. What i understood from the figure below is whenever a thing sends a data to platform via a message…
ccobanoglu
  • 192
  • 3
  • 13
0
votes
0 answers

Flink SQL: Emitting an event after a certain time has passed after original event was recorded

I am trying to setup a data streaming pipeline that starts with a task to emit an event after a certain time (say 5 days) has passed after a certain other event has been registered in Kafka topic. Basically, the use case is a chain of marketing…
srdkr13
  • 1
0
votes
0 answers

Kafka Streams window operation

Let say we have a object Model where we have a few fileds and one of them is "price". I want to calculate AVG of that pirce on every event in 20s window and then filter those events that has "price - calculatedAvg > 5". Input data are like data…
Kacu
  • 438
  • 4
  • 23
0
votes
0 answers

Server Logs aggregation/reduction with KsqlDB

I'm starting my journey of learning KSQLDB with an exciting exercise! I have a Kafka topic that continuously receives log records from my machine. My ultimate goal is to deduplicate the events in the topic and provide an aggregated/windowed…
Abdelwahhab
  • 83
  • 5
0
votes
0 answers

How to fan in multiple streams for projections while maintaining a decent global order?

My initial idea is to buffer and re-order events based on their timestamp (as a fallback) within a tumbling window. I'm still shaping my thinking and figuring out some complexities, e.g., dealing with stream-lagging-related issues. I'm curious to…
reda la
  • 801
  • 1
  • 6
  • 10
0
votes
1 answer

Kappa architecture - conceptual question about historical data processing

This is a question about building a pipeline for data-analytics in a kappa architecture. The question is conceptional. Assume you have a system that emits events, for simplicity let's assume you just have two events CREATED and DELETED which tell…
Sjoerd222888
  • 3,228
  • 3
  • 31
  • 64
0
votes
1 answer

system design - How to update cache only after persisted to database?

After watching this awesome talk by Martin Klepmann about how Kafka can be used to stream events so that we can get rid of 2-phase-commits, I have a couple of questions related to updating a cache only when the database is updated properly. Problem…
ng.newbie
  • 2,807
  • 3
  • 23
  • 57
0
votes
1 answer

How to trigger a line of code in window based CEP

I have a stream of events and want to count the number of events in a specific period time to find out event loss. My code is something similar to the following: DataStream dataStream =…
Majid Lotfian Delouee
  • 5
  • 2
0
votes
2 answers

Recommendations to store streaming events

We're evaluating possible approaches to persist streaming events(user click events in a web browser from many different users) so that it allows us to build custom user dashboards to later analyse those click events. We're planning to use Kafka to…
knightcool
  • 324
  • 1
  • 3
  • 11
0
votes
2 answers

Are Hazelcast Jet Reliable Topic Sinks idempotent? (Hazelcast fault-tolerance of a websocket source)

I cannot find this in the Hazelcast Jet 5.0 (or 4.x) documentation, so I hope someone can answer this here - can a reliable topic be used as an idempotent sink, for example to de-duplicate events coming from two identical unreliable sources (like a…
siddhadev
  • 16,501
  • 2
  • 28
  • 35
1
2
3