Stack Overflow
Stack Overflow

Questions tagged [trident]

Abstraction on top of Apache Storm for doing realtime computation.

Trident is a high-level abstraction for doing realtime computing on top of Apache Storm. It allows you to seamlessly intermix high throughput (millions of messages per second), stateful stream processing with low latency distributed querying. If you're familiar with high level batch processing tools like Pig or Cascading, the concepts of Trident will be very familiar – Trident has joins, aggregations, grouping, functions, and filters. In addition to these, Trident adds primitives for doing stateful, incremental processing on top of any database or persistence store. Trident has consistent, exactly-once semantics, so it is easy to reason about Trident topologies. (source)

122 questions
0
votes
0 answers

Could I use Trident library to generate the absorption spectrum of atoms/molecules like carbon

Hi I wanted to find the absorption spectrum of different elements/molecules of the periodic table and I found that Trident can simulate spectrums (I would prefer real spectrums in a .fit format) and I want to ask if there I could do it with that.…
alvarito mendez
  • 1
  • 2
0
votes
1 answer

Why does Trident not call ack() or fail() in this minimal example?

I tried to create a small example in Trident. The goal was to see how tuples are replayed in Case of failures. Below is the topology definition Random rand = new Random(); Config config = new Config(); …
maow
  • 2,712
  • 1
  • 11
  • 25
0
votes
1 answer

Do we need to remove duplicate by ourselves on at least once delivery case?

Apache Storm and Samza guarantee at least once delivery. It means that there may be some duplicates in the computation process. Do we need to move the duplicates by ourselves(including removing duplicate part in our code)? For example, the word…
SherleyZ
  • 1
  • 1
0
votes
1 answer

How to fix error 'Component: [x] subscribes from non-existent component [y]' in Apache Storm Trident topology

I just implemented a trident DRPC function to process incoming messages and I am trying to persist the count of processed tuples at the final stage of the topology as a Trident state. Here is my topology: topology.newDRPCStream("portfolio") …
gox9
  • 11
  • 4
0
votes
1 answer

How can I return a tuple with multiple fields from Combiner/Reducer/Aggregator function?

Here Storm Documentation states: A CombinerAggregator returns a single tuple with a single field as output. What should I do to return a tuple with multiple fields from Combiner function? I am creating a aggregate function and want to aggregate two…
DP63
  • 67
  • 1
  • 11
0
votes
1 answer

Find the word having the maximum count using a Trident topology

How can I find the word which has the maximum count in a word count topology using Trident topology? Here's the link to the Trident word count…
Rahul Singh
  • 23
  • 6
0
votes
1 answer

KafkaTridentSpoutOpaque Repeated consumption the last message

I use storm+kafka+protobuf to build my stream process system. The problem is KafkaTridentSpoutOpaque repeatedly consumes the last message. I want just one consumer for every message in kafka. Followings are some details: Java…
ChenBo
  • 1
  • 2
0
votes
1 answer

Spout Error from Apache Storm Trident and Kafka Integration

I am using OpaqueTridentKafkaSpout to consume messages from Kafka. Below is the code. I have ignored max spout pending configuration as this is causing same kafka message to arrive in multiple batches. TridentKafkaConfig tridentKafkaConfig = new…
phaigeim
  • 729
  • 13
  • 34
0
votes
0 answers

Logging Kafka Partition and offset from Apache Storm Trident

Where are Kafka Partitions and corresponding Offsets stored while consuming messages from Kafka using Apache Storm Trident ? I could find something in Storm Zookeeper under ls /transactional//coordinator/meta. But I am unable to…
phaigeim
  • 729
  • 13
  • 34
0
votes
1 answer

Checking Offset of Kafka topic for a storm consumer

I am using storm-kafka-client 1.2.1 and creating my spout config for KafkaTridentSpoutOpaque as below kafkaSpoutConfig = KafkaSpoutConfig.builder(brokerURL, kafkaTopic) …
phaigeim
  • 729
  • 13
  • 34
0
votes
1 answer

Open website with Trident web engine in android

Is there any way to load a web page with Trident web engine (windows IE engine) in android. as you know in android website decode with webkit engine.
Saman
  • 2,506
  • 4
  • 22
  • 41
0
votes
1 answer

BaseStatefulBolt (Storm Core) vs StateFactory (Storm Trident)

i am confused about using storm. I am going to measure status of data source using its streamed data. Status will be calculated with combine of some fields, and these field can be achieved different time interval. That's why i need to save fields to…
Joker
  • 96
  • 6
0
votes
0 answers

Storm + Redis or Storm Trident or Spark Streaming

i am going to build stream processing systems. Using Kafka for message transmission. Stream processing can be done with storm trident, storm or spark streaming. But couldn't find the best answer. Stream Processing will be simple if conditions at…
Joker
  • 96
  • 6
0
votes
1 answer

When I use storm trident, if I set the parallelism not less than 2, how can I make all the executors run on different servers not just on one server?

i.e., if the parallelism is 2, the bolt run on 2 different servers, and if the parallelism is 3, the bolt run on 3 different servers. It's important for me, for I don't want all the tasks running on just one server, that'll be too slow.
winfield
  • 15
  • 5
0
votes
1 answer

When I use storm scheduler, can one host be scheduled for two times?

For example topology.newStream("process", new RawDataSpout()) .parallelismHint(1) .shuffle() .each(new Fields("frameID"), new RawToRgb(), new Fields("frameIDRawToRgbCompleted")) …
Niubility
  • 577
  • 5
  • 19
1 2 3
…
8 9