1

I build a storm trident topology like this:

tridentTopology.newStream(spoutId, spout).parallelismHint(spoutParallel)
                       .each(new Fields("tId", "message"), new VerifyFilter())
                       .each(new Fields("tId", "message"), new ParseFunction(), new Fields("rowKey", "column", "value"))
                       .groupBy(new Fields("rowKey", "column"))
                       .aggregate(new Fields("value"),new Count(),new Fields("count"))
                       .each(new Fields("rowKey", "column", "count"), new SaveFunction(), new Fields(columns))
                       .shuffle()
                       .partitionPersist(factory, new Fields(rowKeyANdColumns), new HBaseUpdater())
                       .parallelismHint(boltParallel);

util execute each(new Fields("rowKey", "column", "count"), new SaveFunction(), new Fields(columns), I got print log in SaveFunction, but partitionPersist not wirte date into hbase, and no error in all topology;

Who can give some suggestions?

Matthias J. Sax
  • 59,682
  • 7
  • 117
  • 137
penuel
  • 183
  • 1
  • 3
  • 10
  • What is `new Fields(columns)`? Is `columns` are variable? What is its content? – Matthias J. Sax Jan 06 '16 at 10:57
  • columns is constant : private static final String[] columns = new String[]{"d0","d1","d2","d3","d4","d5","d6","d7"}; SaveFunction will generate 8 output field {"d0","d1","d2","d3","d4","d5","d6","d7"}. – penuel Jan 07 '16 at 03:39
  • I assume `rowKeyANdColumns` is `private static final String[]` with `{"rowKey", "d0","d1","d2","d3","d4","d5","d6","d7"}`. `factory` is `HBaseState`? What is the code of `SaveFunction`? – Matthias J. Sax Jan 07 '16 at 08:34
  • yes,and SaveFunction only for **generateValues** as: `@Override public void execute(TridentTuple tuple, TridentCollector collector) {collector.emit(generateValues(tuple.getString(0),Integer.valueOf(tuple.getString(1)), tuple.getLong(2))); }Values generateValues(String rowKey, int diff, long count) { Values values = new Values(); for ( int i = 0; i <= 7; i++ ) { if ( i == diff ) { values.add(i, count); } else { values.add(i, 0); } } values.add(8, rowKey); return values; }` – penuel Jan 13 '16 at 08:47

0 Answers0