Stack Overflow
Stack Overflow

Questions tagged [trident]

Abstraction on top of Apache Storm for doing realtime computation.

Trident is a high-level abstraction for doing realtime computing on top of Apache Storm. It allows you to seamlessly intermix high throughput (millions of messages per second), stateful stream processing with low latency distributed querying. If you're familiar with high level batch processing tools like Pig or Cascading, the concepts of Trident will be very familiar – Trident has joins, aggregations, grouping, functions, and filters. In addition to these, Trident adds primitives for doing stateful, incremental processing on top of any database or persistence store. Trident has consistent, exactly-once semantics, so it is easy to reason about Trident topologies. (source)

122 questions
1
vote
0 answers

Apache Storm: Supervisor kills and restarts worker process

Using Storm 1.2.2 Kafka 1.1.0 After submitting topology, supervisor launches a worker processes. When checking the worker.log file for that launched Worker Process, it was found out that, somewhere between loading of all the executors, worker…
DP63
  • 67
  • 1
  • 11
1
vote
1 answer

Reading data form Elasticsearch into Flink aggregation?

I'm trying to update documents in Elasticsearch using Kafka messages (as a StreamSource). Writing to Elasticsearch in bulks using windows and the Elasticsearch connector as a sink is fine, however, we need to update existing data in the documents…
Peter Neubauer
  • 6,311
  • 1
  • 21
  • 24
1
vote
0 answers

Setting bolts to read from specific streams of other bolts in trident topology

I am trying to write a TridentTopology, which has multiple bolts. Now I want to make one bolt register to other bolts specific stream as shown below. TridentTopologyBuilder tridentTopologyBuilder = new TridentTopologyBuilder(); FixedBatchSpout spout…
Swetha
  • 87
  • 9
1
vote
0 answers

Apache Trident operator not always executed

First of I'm kinda new to Storm/Trident and I'm struggling with a problem for many hours already. What I have is one Kafka topic with one partition. A producer sends tuples to this topic every x milliseconds. A TransactionalTridentKafkaSpout reads…
Luke
  • 11
  • 3
1
vote
0 answers

When I use storm trident，how can I manully fail a bolt and make the spout re-emit the tuple?

I've tried many ways, but I find some exceptions cannot make the spout re-emit. But in my work, I strongly need that, how can I do? I've tried to throw the FailedException, but it seemed useless. This is my code try { process.waitFor(); }…
Niubility
  • 577
  • 5
  • 19
1
vote
0 answers

Storm Trident - Continuous emits from aggregator even when there is no data in kafka

I have a topology which sources tuples from kafka. Topology looks roughly like this. topology.newStream("kafkaSpout", spout) .shuffle() .each(new Fields("str"), new Filter()) .parallelismHint(5) …
Amber Kulkarni
  • 424
  • 8
  • 17
1
vote
2 answers

Total number of non repeated words in each tweet

I'm new to java and Trident , I imported project for getting tweets but i want to get something How this code get more than one tweet as i got form the code that tuple.getValue(0); means first tweet only ?! Problem with me to get all tweets in…
user1
  • 501
  • 2
  • 9
  • 24
1
vote
0 answers

Persisting State from a DRPC Spout in Trident

I'm experimenting with Storm and Trident for this project, and I'm using Clojure and Marceline to do so. I'm trying to expand the wordcount example given on the Marceline page, such that the sentence spout comes from a DRPC call rather than from a…
cjol
  • 1,485
  • 11
  • 26
1
vote
0 answers

Build Storm Trident topology and persist tuples in HBase , but there's no data in hbase

I build a storm trident topology like this: tridentTopology.newStream(spoutId, spout).parallelismHint(spoutParallel) .each(new Fields("tId", "message"), new VerifyFilter()) .each(new Fields("tId",…
penuel
  • 183
  • 1
  • 3
  • 10
1
vote
1 answer

Apache Storm Trident .each() function explanation

I want to use Apache Storm's TridentTopology in a project. I am finding it difficult to understand the .each() function from the storm.trident.Stream class. Below is the example code given in their tutorial for reference: TridentTopology topology =…
XCEPTION
  • 1,671
  • 1
  • 18
  • 38
1
vote
1 answer

Emit to multiple streams in Storm Trident

How can I emit to multiple streams from the same bolt in Storm Trident? I have a bolt which does some calculation and based on the result I want to pass some values to one stream, and some other values to another stream. In Storm (not Trident), we…
dreamer
  • 1,039
  • 2
  • 16
  • 36
1
vote
1 answer

Apache Storm once-only processing

We are currently using Apache Storm 0.9.5 in clustered topology mode to process Amazon Kinesis records (spout) and store them into a Redshift data warehouse (bolt). Our Storm cluster is deployed in AWS and consists of 1 nimbus + UI node, 1 zookeeper…
Lex Luthor
  • 523
  • 6
  • 18
1
vote
1 answer

Empty data while reading data from kafka using Trident Topology

I am new to Trident. I am writing a trident topology which reads data from kafka. Topic name is 'test'. I have local kafka setup. I started zookeeper, kafka in local. And created a topic 'test' in kafka and opened the producer and typed the message…
Kutty
  • 39
  • 3
1
vote
1 answer

How to Re-balance Storm Trident Topology

From this documentation link I could learn that re-balancing a topology is easy with below command storm rebalance mytopology -n 5 -e blue-spout=3 -e yellow-bolt=10 But how do I re-balance a trident topology as in trident topology we can't label…
Aride Chettali
  • 181
  • 1
  • 7
1
vote
2 answers

How to ensure storm not write message twice to local file?

I build a topo to get messages from by kafka, and then grep some keyword, if fit, write to local file. I use OpaqueTridentKafkaSpout of storm-kafka to ensure the tuple will not miss or repeat, but consider one situation: when writing message to…
jinhong_lu
  • 238
  • 1
  • 2
  • 11
1 2
3
…
8 9