Stack Overflow
Stack Overflow

Questions tagged [hazelcast-jet]

Hazelcast Jet is a distributed computing platform for low-latency stream and batch processing. It benefits from integration with Hazelcast IMDG providing fast, scalable Big Data processing that’s lightweight and embeddable.

Hazelcast Jet is a distributed computing platform for low-latency stream and batch processing. It benefits from integration with Hazelcast IMDG providing fast, scalable Big Data processing that’s lightweight and embeddable.

188 questions
1
vote
1 answer

Running a streaming operation ONLY on the node which contains the relevant KEY

Let's say I have a large IStreamMap on a large cluster and I only want to do an operation on a few keys. I could just right a filter expression as shown below, but my understanding is that this will run on all nodes. And 99% of the nodes will be…
Jonathan Sylvester
  • 1,275
  • 10
  • 23
1
vote
1 answer

Hazelcast Jet with an IMap source and OBJECT in-memory format

I have items in a Hazelcast IMap in OBJECT format, and I'm using a Jet aggregation operation with that IMap as a pipeline source. I was hoping, because of the OBJECT format, to avoid any serialisation/deserialisation of the items in my IMap during…
Chris Wiggins
  • 31
  • 2
1
vote
1 answer

Hazelcast Jet IMap remove entry

I've read the documentation of Hazelcast Jet. I've seen it is possible to add/update entries in an IMap sink. But I've seen nowhere how it is possible to remove entries from the IMap. Is there a way for this ?
rico
  • 1,843
  • 2
  • 24
  • 41
1
vote
1 answer

Hazelcast Jet. 'onComplete' event at sink?

In a typical pipeline scenario, say I have a bounded stream, where I read from a file. Is there a way in Jet where I can subscribe to an "OnComplete" event, which will be triggered once the stream is written to sink? I don't seem to find such an…
Rajesh
  • 153
  • 6
1
vote
1 answer

Hazlecast Jet Cluster. Work load not distributed

I have one huge csv file. I have a Jet cluster with 3 nodes. When the job is submitted only one node processes the entire file. What I want is the each part of work can be distributed. Meaning, how can I optimally use the resources in each of the…
Rajesh
  • 153
  • 6
1
vote
1 answer

Adding [hazelcast-jet] to existing Application

I have an existing application that uses Hazelcast for tracking cluster membership and for distributed task execution. I'm thinking that Jet could be useful for adding analytics on top of the existing application, and I'm trying to figure out how…
PatTheGamer
  • 471
  • 4
  • 17
1
vote
1 answer

Hazelcast Jet - drain the list to a stream

My Jet job is transforming the Redis stream data - the tranformation is - I lookup a map for every item in the stream - if found it contains one or more items (list). I would like to write the items in draining stream as separate items and not as…
vvra
  • 2,832
  • 5
  • 38
  • 82
1
vote
0 answers

Hazelcast Jet - submit job to remote Kubernetes cluster

I have a Jet (3 instance) cluster setup in a Kubernetes cluster (Openshift). I would like to submit the job to it. I used proxy to connect but the job isnt submitted. I also tried to upload the far jar file to one of the cluster pod and submit from…
vvra
  • 2,832
  • 5
  • 38
  • 82
1
vote
1 answer

HazelcastSerializationException: When Jet job is running over multiple cluster

I am trying to run one of the Hazelcast-jet example over a Distributed System. My objective is to run code over Disributed System, Utilize Multiple machine's processing power. I have two Laptops connected via LAN. When I run this example in One…
Milan
  • 147
  • 1
  • 4
  • 16
1
vote
1 answer

Hazelcast-jet: got error when enriching stream using direct lookup

I am following Doc to try out how to enrich an unbounded stream by directly looking up from a IMap. I have two Maps: Product: Map (ProductId as key) Seller: Map (SellerId as key) Both Product and Seller are very…
Z.SP
  • 319
  • 1
  • 2
  • 12
1
vote
1 answer

Can Hazelcast jet stream directly trigger events?

After a stream is being processed (applying filter and etc), can each of the remaining items in the processed stream trigger events directly instead of being written to a sink? I have a bunch of devices keep reporting status to the hazelcast jet…
Z.SP
  • 319
  • 1
  • 2
  • 12
1
vote
1 answer

Client unable to connect to Hazelcast-jet ec2 clusters

I setup a Hazelcast-jet cluster on aws ec2 following instructions here. I made use of the hazelcast-aws model so that nodes can automatically discover each other. The cluster is up and running: [2019-09-26 22:26:26.288] [INFO ]…
Z.SP
  • 319
  • 1
  • 2
  • 12
1
vote
0 answers

Hazelcast Jet: How to prevent 'event dropped'?

I am getting 'Event dropped, late by 5051 ms.' How should I build my pipeline that all events are processed, regardless of their late arrival. I have tried several approaches. Basically, what I tried was Without windowing where I didn't get late…
Aliman
  • 43
  • 6
1
vote
2 answers

How to fix unexpected hazelcast client shutdown

I'm using hazelcast jet to perform aggregations on stream data. Problem is, that hazelcast cliend shutsdown unexpectedly. I've implemented simple pipeline with remote map source and then the result is simply sinked. // init pipeline Pipeline p =…
Aliman
  • 43
  • 6
1
vote
1 answer

Hazelcast Jet - Group By Use Case

We have a requirement to group by multiple fields in a dynamic way on a huge data set. The data is stored in Hazelcast Jet cluster. Example: if Person class contains 4 fields: age, name, city and country. We first need to group by city and then by…
Bharat
  • 31
  • 4
1 2 3
…
12 13