Questions tagged [azure-eventhub]

Microsoft Azure EventHubs is a highly-scalable high-throughput low-latency durable event ingestion system in Azure cloud. EventHubs serves as an event buffer for applications to process events with variable load profiles. Microsoft Azure Event Hubs is comparable to Apache Kafka or AWS Kinesis. This is a fully-managed service-offering built & run by the teams at Microsoft.

Microsoft Azure EventHubs is a highly scalable high throughput low latency durable event ingestion system in Azure cloud. EventHubs serves as an event buffer for applications to process events with variable load profiles.

Stream millions of events per second into multiple applications

Event Hubs is a highly scalable publish-subscribe ingestor that can intake millions of events per second so that you can process and analyze the massive amounts of data produced by your connected devices and applications. Once collected into Event Hubs you can transform and store data using any real-time analytics provider or with batching/storage adapters.

Enable applications to process events with variable load profiles

Today’s connected world is defined by big data. Big data originates from many sources such as connected cars and thermostats that produce telemetry data every few minutes, application performance counters that generate events every second or mobile apps that capture telemetry for every user’s individual action. In this connected world, you need a managed ingestor service capable of elastic scale in order to accommodate variable load profiles as well as the spikes caused by intermittent connectivity.

Connect millions of devices across platforms

The rapid proliferation of connected devices raises challenges due to the variety of platforms and protocols involved. Connecting these disparate data sources while handling the scale of the aggregate stream is a significant challenge. Event Hubs provide easy provisioning of capacity to ingest events from millions of devices while preserving event order on a per device basis. Support for AMQP and HTTP allow many platforms to work with Event Hubs. Native client libraries also exist for popular platforms.

1941 questions
4
votes
2 answers

Azure Eventhubs (Python): checkpointing with blob storage - keyerror issue in EventProcessor when checkpointing is enabled

I'm having an issue with blob storage checkpointing in eventhubs. My application runs fine if I don't have checkpoint_store set when getting the consumer client. Whenever I try to set the checkpoint_store variable and run my code it throws the…
4
votes
1 answer

Connect to Azure EventHub(Kafka like) with Spring Boot using connection string

I need to connect to event hub with enabled kafka with Spring Boot, and I have connection string and name space where should I connect. I'm using such dependencies com.microsoft.azure
Bohdan Myslyvchuk
  • 1,657
  • 3
  • 24
  • 39
4
votes
2 answers

Azure EventHub: checkpoint best practices in C# EventProcessorClient

According to documentation EventProcessorClient is recommended way to consume eventhub streams: EventProcessorClient should be used for reading events from all partitions in a production scenario, as it offers a much more robust experience with…
mtkachenko
  • 5,389
  • 9
  • 38
  • 68
4
votes
1 answer

Getting Azure.WebJobs.Host.Listeners.FunctionListenerException for eventhub triggers

I am using azure webjobs v3 for eventhub trigger. Till sometime everything was working fine. Suddenly I am getting below error. Can someone help me in resolving this? public async Task Trigger00([EventHubTrigger("%eventhub-00%", Connection =…
4
votes
0 answers

How to store eventhub checkpoint data locally in Java

I'm trying to find an example of Custom Checkpoint Manager in JAVA, that can store checkpoint data in a local folder. Basically, I'm building a java application that reads data from azure event hub with multiple consumer groups. Previously I was…
4
votes
3 answers

Data from event hub not getting populated in ADX database

I created a sample application to send events to the event hub, which subsequently sends data to the azure data explorer database. I can see the events appearing in the event hub, but the same is not getting ingested in the Azure Data Explorer…
4
votes
1 answer

SchemaException: Error reading field 'topics' in Kafka and Event Hubs

I am working with Azure Event Hubs and Kafka. I have a producer that would send messages to a topic with 4 partitions in Event Hubs. Then, 4 consumers would consume from that topic. Most of the time, it works properly. But then, I would sometimes…
kpalos
  • 41
  • 2
4
votes
3 answers

How do you monitor Azure Event Hub consumer lag?

We have been trying monitor consumer lag on Event Hub partitions. I have looked in the azure portal and the Event Hub Api's but so far found nothing. Was wondering if anyone has tried this and if so could you point me in the right direction.…
Chimmy
  • 157
  • 2
  • 10
4
votes
1 answer

Invoke ADF pipeline using Azure EventHubs

Would like to know whether we can invoke ADF pipeline using Azure EventHub? Thanks.
Abhi
  • 341
  • 1
  • 6
  • 23
4
votes
1 answer

Is it possible to configure Azure Event Hub to retain message if Azure Function fails processing it?

I have an Azure Function that listens for messages in an Event Hub. The function takes messages from the Event Hub, processes them, and passes them to another Hub. At this point the messages are removed from the Event Hub. If the Function fails…
DaveDev
  • 41,155
  • 72
  • 223
  • 385
4
votes
2 answers

How to get Azure EventHub Depth

My EventHub has millions of messages ingestion every day. I'm processing those messages from Azure Function and printing offset and squence number value in logs. public static async Task Run([EventHubTrigger("%EventHub%", Connection =…
Pankaj Rawat
  • 4,037
  • 6
  • 41
  • 73
4
votes
1 answer

Message not sending to Azure Events Hub: Put token failed. status-code: 404, status-description: The messaging entity... ...could not be found

I am learning Azure Events Hub. A simple application i have downloaded from this link https://learn.microsoft.com/en-us/azure/event-hubs/event-hubs-dotnet-standard-getstarted-send . But when i try to send message, its giving me this error:…
sebu
  • 2,824
  • 1
  • 30
  • 45
4
votes
0 answers

Sensor data to Azure IoT Hub and then to a web client using SignalR on ASP.net Core 2.1

Despite my efforts, I did not find any tutorials on how to get live sensor data via Azure IoT Hub to a web client using SignalR, where the web backend is running on ASP.net Core. I somehow get it running, so here's what I did, and a couple…
4
votes
2 answers

How to trigger to multiple eventhubs in azure webjob or function

I have a multiple Azure Event Hubs located in different Azure Regions. I want to write a webjob/function that can receive messages from all of these Event Hubs without me having to hard code which EHs to listen to. So, normally you would have…
viblo
  • 4,159
  • 4
  • 20
  • 28
4
votes
2 answers

How to implement filters in Eventhub?

In my application, various events published to a Eventhub. But my consumer group needs only specific set of events. How to filter this in Eventhub?
K.P.
  • 105
  • 1
  • 9