1

I have a noticed recently that when I have two instance of a streaming application submitted and they are working in parallel, the input rate somehow changes. This image is from one of the applications, when I killed the other one. The input rate increases. enter image description here

I am subscribing from MQTT message broker to get the data to the application. Does this mean that the load gets divided between the two applications?

More info: After the data is being processed by the application, it gets written to HBase, and the transaction is idempotent and nothing happens if data gets written twice.

Arsinux
  • 173
  • 1
  • 4
  • 13

1 Answers1

0

There are multiple ways to identify it.

If you are maintaining time-stamp based versions of hbase cell data, you can check how many number of time-stamp versions are present for that particular data cell. You can easily check this through hbase shell. Check this for referance

Another way is, you can log this data with streaming application id from both the streams. Check if same data is getting inserted from both the streams or not.

Sachin
  • 39
  • 5