0

I have written a demo about trident. In the demo, I want to generate a series batches of data in a transaction spout and then do persistentAggregate operation for them.

The normal processing like this:

step1: txid=1, {"aaa 3", "bbb 2"}==>persist to DB(OK)

step2: txid=2, {"ccc 6", "ddd 7"}==>persist to DB(OK)

In the case that the operation of persist to DB is abnormal in step1, I suppose the process should be

step1: txid=1, {"aaa 3", "bbb 2"}==>persist to DB(NG)

step2: txid=1, {"aaa 3", "bbb 2"}==>persist to DB(OK)

But the test result is as following:

step1: txid=1, {"aaa 3", "bbb 2"}==>persist to DB(NG)

step2: txid=1, {"ccc 6", "ddd 7"}==>persist to DB(OK)

I think this process is not correct.But I do not know why it happen. I have checked the zookeeper, in the path "meta/1, the value is null.

I wrote a transactional spout myself implement the interface ITridentSpout, I think it might be a problem with the spout I have written.

Anyone can give me some advice? Or give me a implemented transactional spout example for me to reference.

Thanks!

Matthias J. Sax
  • 59,682
  • 7
  • 117
  • 137
iceguo
  • 103
  • 1
  • 12
  • How does your spout handle retries? The coordinator will get a non-null currentMetadata which means something failed downstream and you need to resend that data. – Joshua Martell Jun 25 '15 at 13:38
  • I think my spout does not handle retries. Can you tell me how to handle retries in more detail? Such as in what function to handle this, how to get the currentMetadata and how to resend it. I'd appreciate that If you can provide me an example. – iceguo Jun 26 '15 at 02:23
  • You should carefully read the `ITridentSpout` JavaDocs. https://github.com/apache/storm/blob/master/storm-core/src/jvm/storm/trident/spout/ITridentSpout.java – Joshua Martell Jun 26 '15 at 19:15

0 Answers0