2

I am trying to use Azure Event Hubs as a message bus using the Confluent Kafka sdk's. When I started testing my code, it was working, but after I created 10 topics, I can no longer create any new topics. I am using IAdminClient.CreateTopicsAsync to create the topics. I can use MetaData.Topics.ToList() to see that the first 10 topics that I created are still there, I just can't create any new ones. I used IAdminClient.DeleteTopicsAsyn to delete one of my topics, and after that I was able to add 1 new one, bringing the total back up to 10. I still cannot create an 11th. Is there some kind of limit that would allow me to create 10 topics, but not an 11th? Is this something I can increase with a simple configuration change? Thank you.

Error thrown by CreateTopicsAsync:

CreateTopicsException: An error occurred creating topics: [TopicName11]: [The service was unable to process the request; please retry the operation. For more information on exception types and proper exception handling, please refer to http://go.microsoft.com/fwlink/?LinkId=761101].

CHEEKATLAPRADEEP
  • 12,191
  • 1
  • 19
  • 42
Eric
  • 2,120
  • 1
  • 17
  • 34
  • If you wouldn't mind sharing how you got `CreateTopicAsync` to work, that would be very much appreciated, thanks. https://stackoverflow.com/questions/71212255/how-can-i-create-a-topic-on-event-hub-kafka-using-the-adminclient – Asik Feb 21 '22 at 19:55

1 Answers1

3

Yes; a topic in Kafka is synonymous with an Event Hub in the Event Hubs ecosystem. (see: Kafka and Event Hub conceptual mapping).

Each Event Hubs namespace has a limit on the number of Event Hub instances that it can contain, dictated by the tier. For a non-dedicated tier, an Event Hubs namespace may have up to 10 Event Hub instances, each containing 1-32 partitions. The Event Hubs quotas and limits documentation has the full details.

Jesse Squire
  • 6,107
  • 1
  • 27
  • 30
  • does anyone know why only 10? tool small. what to do if we need more? use another EventHubs namespace is not an option. Consolidate topics - hard to process different payloads – Roman Marusyk May 13 '23 at 01:03
  • That, I do not know. You'd need to ask the Event Hubs service team to understand the rationale. The best way to engage them would be to open an Azure support request: https://docs.microsoft.com/azure/azure-portal/supportability/how-to-create-azure-support-request – Jesse Squire May 15 '23 at 14:37