1

Problem Statement Single consumer is consuming the message from the single topic partition. The consuming topic partition has 8578 messages The observation from Azure Eventhub Metrics, Outgoing message is high and seeing the Throttling Request. This test is performed in Azure Eventhub with Standard Tier with 1 Throughput Unit.

Expected Result Should see only the 8578 message in Outgoing message metrics and no Throttling Request should see.

Confluent Kafka Nuget Package version 2.1.1 is used for this test

Consumer Configuration

AutoOffsetReset =AutoOffsetReset.Earliest
EnableAutoOffsetStore = true
EnableAutoCommit = false
AutoCommitIntervalMs = 10000 //Used only when "EnableAutoCommit " is set to true
EnablePartitionEof = true
ConnectionsMaxIdleMs = 180000
MaxPartitionFetchBytes = 12518400 // 11.9 MB
PartitionAssignmentStrategy = PartitionAssignmentStrategy.CooperativeSticky
IsolationLevel = IsolationLevel.ReadUncommitted
SocketNagleDisable = true
SocketKeepaliveEnable = true
MetadataMaxAgeMs = 180000
SessionTimeoutMs = 30000
MaxPollIntervalMs = 300000
CancellationDelayMaxMs = 200,
Debug = "consumer,fetch"

Consumer Topic Metrics ConsumerMetrics Note: The incoming messages is there is different time frame. Messages are accumulated once performing the test with different ConsumerGroup for each run for test

Producer Topic Metrics ProducerMetrics

Vishnu Kumar K S D
  • 21
  • 4
  • Have you tried to increase the throughput units? – Serkant Karaca Aug 23 '23 at 16:42
  • Incoming message volume shows as zero on the screenshot you attached. Are you testing consumers only? – Serkant Karaca Aug 23 '23 at 16:43
  • @SerkantKaraca, Yes by increasing the TU i could able to solve it. But the scenario is how to solve with lesser TU. Basically when exceeding the limit (AZ EventHub Tier) how to handle it. Yes I accumulated the message, and trying to consume the message only. With the selected time frame No Incoming Messages to the EventHub. – Vishnu Kumar K S D Aug 23 '23 at 18:02
  • Couple of observation now after tuning the below configuration. FetchMaxBytes to 102400 message.max.bytes to 102399 With this I can see the Throttling Request happening in first 2 min when the application starts [Screenshot](https://user-images.githubusercontent.com/9166122/262944432-3858bcce-408e-4279-8687-ee24a55ca6bd.png) – Vishnu Kumar K S D Aug 24 '23 at 10:26
  • It is hard to tell which type of throttling was in place from the screenshot. Can you try to identify if throttling is on the message count or the byte count? – Serkant Karaca Aug 25 '23 at 16:36
  • Throttle Request is based on the number of request. Definition from Microsoft The number of requests that were throttled because the usage was exceeded. https://github.com/MicrosoftDocs/azure-docs/blob/main/articles/event-hubs/monitor-event-hubs-reference.md#request-metrics – Vishnu Kumar K S D Aug 27 '23 at 06:11
  • @Moderator I have updated the post. Could you reopen it? – Vishnu Kumar K S D Aug 31 '23 at 02:09

0 Answers0