0

I am using eventhub python sdk consumer for real time processing of the data from eventhub. I have currently 2 partitions of my topic. I would like to know about my consumer lag. I know what is the sequence number of the message I am processing, but cannot get the last sequence number enqueued in the event hub on a particular partition. If I can get this information, it will be very easy for me to find the lag for a particular partition.

Can someone please help me how can I do this?

teoML
  • 784
  • 4
  • 13
Nipun
  • 4,119
  • 5
  • 47
  • 83
  • I am looking at the python sdk for eventhub in azure – Nipun Dec 11 '19 at 06:12
  • I looked at the partitionContext class in python and could not find the RuntimeInformation.LastSequenceNumber attribute which is present in the other sdk. Let me know if I am missing anything – Nipun Dec 11 '19 at 06:15
  • which version of event hub sdk are you using? – Ivan Glasenberg Dec 17 '19 at 02:04
  • 1.3.3, https://pypi.org/project/azure-eventhub/ – Nipun Dec 17 '19 at 06:56
  • Hi Nipun, I want to confirm something. For "the last sequence number", do you mean the latest sequence number of a partition? and when you get the sequence number, do you need to read the data or not? – Ivan Glasenberg Dec 17 '19 at 07:49
  • What I read, when data is being read from the consumer, it receives partionContext object which contains the message sequence number and also the latest sequence number of an event from a partition. Last Sequence number is the message entering event hub at the time of reading the message by the consumer group – Nipun Dec 17 '19 at 08:48
  • I'm still not clear about the last sequence number "Last Sequence number is the message entering event hub at the time of reading the message by the consumer group ". Can you explain it a little more? – Ivan Glasenberg Dec 17 '19 at 09:03
  • When the consumer reads an event from the eventhub on a particular partition, it also gets information of the sequence number that is being written by the publisher at that moment on that particular partition. This information is missing in the python sdk. If I get this information, I can easily get the consumer lag – Nipun Dec 17 '19 at 09:35
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/204398/discussion-between-ivan-yang-and-nipun). – Ivan Glasenberg Dec 17 '19 at 09:37
  • Anyone who can help me out with this? – Nipun Jan 28 '20 at 12:19
  • This is all the more important now as I saw a issue in production today and if there was alert set on consumer lag I would have been notified much earlier – Nipun Jan 28 '20 at 12:20

0 Answers0