0

I am reading events from an Azure EventHub cluster synchronously via the receiveFromPartition method on the EventHubConsumerClient class.

I create the client once like so:

EventHubConsumerClient eventHubConsumerClient = new EventHubClientBuilder()
            .connectionString(eventHubConnectionString)
            .consumerGroup(consumerGroup)
            .buildConsumerClient());

I then just use a ScheduledExecutorService to retrieve events every 1.5s via:

IterableStream<PartitionEvent> receivedEvents = eventHubConsumerClient.receiveFromPartition(
                partitionId, 1, eventPosition);

The equivalent logic in V3 of the SDK worked fine (using PartitionReceivers), but now I am seeing OOMs in my JVM.

Running a profiler against a local version of the logic I see the majority of the heap (90%, mainly in OG) is being taken up by byte[]s, referenced by org.apache.qpid.proton.codex.CompositeReadableBuffer. This pattern is not present when I profile the V3 logic.

What could be causing a leak of the AMQP messages here, do I need to interact with the SDK further, for example close a connection that I'm not aware of after each call?

Any advise would be very appreciated, thanks!

steve
  • 537
  • 1
  • 7
  • 17
  • 1
    Thanks for reporting this. We're actively keeping track of issues in the client library in our GitHub repository. Is it possible to get a dump of your application? I'm curious what is holding onto that memory. :( http://github.com/Azure/azure-sdk-for-java/issues. – Connie Yau Aug 03 '20 at 17:36
  • Created an issue here https://github.com/Azure/azure-sdk-for-java/issues/13775, unfortunately I can't share the heap dump as it will contain sensitive data, happy to answer any questions / help however I can though! – steve Aug 04 '20 at 08:03

1 Answers1

0

Turns out it was a bug, solved here: https://github.com/Azure/azure-sdk-for-java/issues/13775

steve
  • 537
  • 1
  • 7
  • 17