Questions tagged [azure-eventhub]

Microsoft Azure EventHubs is a highly-scalable high-throughput low-latency durable event ingestion system in Azure cloud. EventHubs serves as an event buffer for applications to process events with variable load profiles. Microsoft Azure Event Hubs is comparable to Apache Kafka or AWS Kinesis. This is a fully-managed service-offering built & run by the teams at Microsoft.

Microsoft Azure EventHubs is a highly scalable high throughput low latency durable event ingestion system in Azure cloud. EventHubs serves as an event buffer for applications to process events with variable load profiles.

Stream millions of events per second into multiple applications

Event Hubs is a highly scalable publish-subscribe ingestor that can intake millions of events per second so that you can process and analyze the massive amounts of data produced by your connected devices and applications. Once collected into Event Hubs you can transform and store data using any real-time analytics provider or with batching/storage adapters.

Enable applications to process events with variable load profiles

Today’s connected world is defined by big data. Big data originates from many sources such as connected cars and thermostats that produce telemetry data every few minutes, application performance counters that generate events every second or mobile apps that capture telemetry for every user’s individual action. In this connected world, you need a managed ingestor service capable of elastic scale in order to accommodate variable load profiles as well as the spikes caused by intermittent connectivity.

Connect millions of devices across platforms

The rapid proliferation of connected devices raises challenges due to the variety of platforms and protocols involved. Connecting these disparate data sources while handling the scale of the aggregate stream is a significant challenge. Event Hubs provide easy provisioning of capacity to ingest events from millions of devices while preserving event order on a per device basis. Support for AMQP and HTTP allow many platforms to work with Event Hubs. Native client libraries also exist for popular platforms.

1941 questions
3
votes
1 answer

Azure Event Hub timeouts

I occasionally see timeout errors in event hub client when trying to send messages to Azure event hub. It looks like resource limit is reached for client but I'm not sure... Here the code: MessagingFactory messagingFactory = null; …
user9194426
3
votes
1 answer

Can I change the Azure Function Event Hub trigger starting checkpoint?

When I stand up a new Azure Function listening to our event hub, with a new consumer group, it gets all the messages that have ever been sent to the event hub, even from months ago somehow. I thought setting the message retention on the hub to one…
Josh
  • 6,944
  • 8
  • 41
  • 64
3
votes
1 answer

How can I create consumer group programmatically for event hub using JAVA?

Scenario: I am trying to create consumer group for Azure Eventhub testing data in event hub. I want to create a consumer group every time I start my tests and destroy it after the tests are completed. I am trying to program it in Java. Are there…
user8828251
  • 61
  • 1
  • 1
  • 4
3
votes
2 answers

Programmatically query Event Hub Consumer Group (and create if required)

Does anyone know how to query and create consumer groups in Azure Event Hubs using the .NET SDK. I've googled loads and can only find a way through the REST API (which I can do, but it would nicer if I can do it through the SDK). Thanks in advance
Ross Ellerington
  • 149
  • 1
  • 12
3
votes
1 answer

Receive events from Azure Eventhub using AMQP

GOAL: Receive messages from EventHub with python. Setup: Python 3.5 AMQP 1.0 EventHub created in new portal. Ubuntu 16.04 and Mac OS X. proton==0.8.8 and python-qpid-proton==0.17.0 I have been trying to get a connection with AMQP to Azure…
JimYong
  • 43
  • 4
3
votes
1 answer

Retry logic for Azure EventHub PartitionedSender

I am sending / recieving data from an Azure Event Hub. As I have a requirement messages must be processed in sequence by the consumer, I have decided to go with a PartitionSender and always send events to the same partition. Under "Consistency" in…
David Andersson
  • 55
  • 1
  • 1
  • 6
3
votes
1 answer

Azure Event Hub ServiceBusException causing skipped messages

We are using the Azure Java event hub library to read messages out of an event hub. Most of the time it works perfectly, but periodically we see exceptions of type "com.microsoft.azure.servicebus.ServiceBusException" occur that correspond to times…
Matt
  • 43
  • 6
3
votes
2 answers

How are different events handled in Azure Event Hubs?

Different event types are discriminated by some combination of topics/queues and message headers. In Kafka, record streams are still separated out as topics (https://kafka.apache.org/intro). In going through the Azure Event Hubs documentation, I do…
Michael
  • 1,306
  • 1
  • 12
  • 30
3
votes
1 answer

Is it a good idea to use QueueBackgroundWorkItem for event hub send data operations?

I have a wep api that getting more than 100 request in a second and it needs to answer back fast. There are two main job in every request: Get the answer from cache and return it to user Send the result to event hub (for some hourly…
can
  • 63
  • 1
  • 8
3
votes
1 answer

Azure Event Hub to Stream Analytics with Partitions

Azure documentation states that: Partitions are a data organization mechanism and are more related to the degree of downstream parallelism required in consuming applications than to Event Hubs throughput. Assuming that the only consumer of the…
3
votes
1 answer

Get the item count for an Azure EventHub

So I am using EventHubs for data pipeline queuing, but in order to get the benefit of buffered ingestion, I need to know how many items are in a partition at any one time. Depending on the consumer group and partition id this can give massive…
tommed
  • 1,521
  • 2
  • 19
  • 33
3
votes
1 answer

Azure EventHubs: recommended values for PartitionManagerOptions

Are there any guidelines on setting values for the parameters in PartitionManagerOptions?, specifically: AcquireInterval: 10 secs (def value) RenewInterval: 10 secs (def value) LeaseInterval: 30 secs (def value) I am concerned that if the…
Jose Parra
  • 877
  • 9
  • 23
3
votes
4 answers

Azure ioT and Event Hub?

I am trying to read the events that my device is sending. I am using azure npm lib to read the what i think is right. Ok so first, under my Azure ioT Hub account for that service there is a tab call Messaging. There is something called "Event…
3
votes
1 answer

Event Hub Lease Management

Does anyone know where I can find details about how event hub lease management works? Specifically I'm trying to find how do I know where in the event hub the EventProcess picks up processing (after a reboot, shutdown, lease lost)? What is the best…
user1116536
  • 373
  • 1
  • 5
  • 11
3
votes
1 answer

Azure Event Hub : What is a 'Field Gateway'?

Scott Guthrie's announcement of Azure Event Hubs shows the below diagram which includes something called a 'field gateway'. I've not been able to find a definition of what this concept means. I'm hoping it's a device-side SDK for sending events to…
urig
  • 16,016
  • 26
  • 115
  • 184