Questions tagged [azure-eventhub]

Microsoft Azure EventHubs is a highly-scalable high-throughput low-latency durable event ingestion system in Azure cloud. EventHubs serves as an event buffer for applications to process events with variable load profiles. Microsoft Azure Event Hubs is comparable to Apache Kafka or AWS Kinesis. This is a fully-managed service-offering built & run by the teams at Microsoft.

Microsoft Azure EventHubs is a highly scalable high throughput low latency durable event ingestion system in Azure cloud. EventHubs serves as an event buffer for applications to process events with variable load profiles.

Stream millions of events per second into multiple applications

Event Hubs is a highly scalable publish-subscribe ingestor that can intake millions of events per second so that you can process and analyze the massive amounts of data produced by your connected devices and applications. Once collected into Event Hubs you can transform and store data using any real-time analytics provider or with batching/storage adapters.

Enable applications to process events with variable load profiles

Today’s connected world is defined by big data. Big data originates from many sources such as connected cars and thermostats that produce telemetry data every few minutes, application performance counters that generate events every second or mobile apps that capture telemetry for every user’s individual action. In this connected world, you need a managed ingestor service capable of elastic scale in order to accommodate variable load profiles as well as the spikes caused by intermittent connectivity.

Connect millions of devices across platforms

The rapid proliferation of connected devices raises challenges due to the variety of platforms and protocols involved. Connecting these disparate data sources while handling the scale of the aggregate stream is a significant challenge. Event Hubs provide easy provisioning of capacity to ingest events from millions of devices while preserving event order on a per device basis. Support for AMQP and HTTP allow many platforms to work with Event Hubs. Native client libraries also exist for popular platforms.

1941 questions
3
votes
1 answer

How one instance Azure Function handle multiple partition event hub trigger

I have an Azure Function hosting on App service plan with Event Hub trigger. The Event Hub has 4 partitions. My question is, if the function has only one instance, how the function process the events from 4 partitions? Does this one instance…
Youxu
  • 1,050
  • 1
  • 9
  • 34
3
votes
1 answer

Event Hub API POST: 40103: Invalid authorization token signature

Before this gets marked as a duplicate: I have read every question/answer on SO with the above error message, and none of them solved my problem. I must be missing something simple, as what should be simple is not working. I have created an Event…
Billy Brown
  • 2,272
  • 23
  • 25
3
votes
1 answer

Is Eventhub message's enqueue time unique?

I routed incoming iot-hub message to Blob Storage. Then, messages will be parsed and stored in to SQL database. I plan to use: EnqueueTime as primary key, so is Eventhub message's enqueue time unique? Here is an example of enqueue time:…
duy
  • 579
  • 5
  • 16
3
votes
1 answer

POST data on Azure Event hub is failing

This is regarding Azure Event hub and I am trying to send data using a POST api call from POSTMAN to my Event hub. Steps I followed: Created Event Hub, Generated SAS send token, Created Consumer group Now in postman I am struggling to format the…
Ankit Kumar
  • 476
  • 1
  • 11
  • 38
3
votes
2 answers

Is it possible to reuse Connections on Azure Functions when sending Device-to-Cloud messages to IoTHub?

I have an Azure IoTHub with thousands of devices registered. These devices communicate through a Telco provider who sends messages through an Azure Storage Queue. This Storage Queue triggers an Azure Function which needs to parse the messages and…
Has AlTaiar
  • 4,052
  • 2
  • 36
  • 37
3
votes
0 answers

Can Confluent Kafka .NET API connect to Event Hub using Managed Service Identity

We would like to connect our producers and consumers to our Kafka-enabled Event Hub using the Confluent Kafka .NET API. This works fine if we keep and pass the SAS keys and secrets, but it would be much better if we could use the Managed Service…
3
votes
3 answers

Data flow: Azure Event Hubs to Cosmos db VS Cosmos db to Azure Event Hub - Better option?

I want to monitor some events coming from my application. One option is to send data to Azure Event Hub and use stream analytics to do some post-processing and enter the data into cosmos db. Another option is to store to cosmos db from application…
3
votes
2 answers

EventGrid vs EventHub

I am working on a service fabric application and want to publish few events from this application and subscribe or process those publish events in another application. I have tried EventGrid concept and observed that there is a delay while…
3
votes
1 answer

How to retrieve properties of IoT hub data and not only the actual payload

I have a iot-hub that receives both JSON and non-json (hex) messages. These all go to my Java function app to decode. Based on the device-id I'm calling a different decoder. I'm trying to get the actual iothub-connection-device-id of the message I'm…
54m
  • 719
  • 2
  • 7
  • 18
3
votes
1 answer

How to write end to end tests against AzureEventProcessor Host

We have a console app that uses dotnet core bodyBuilder to run an instance of AzureEventProcessor host. I would like to be able to run end to end tests against the console app but have not been able to replace the AzureEventProcessorHost with an…
Farhad-Taran
  • 6,282
  • 15
  • 67
  • 121
3
votes
1 answer

How to increase message limit size of azure event hub from 1 MB to send 4 MB?

I have a requirement of sending 1,00,000 batch of records per second. What i got after experimenting is azure event hub has limit of 10,00,000 events in bytes. My each record is of 145 bytes and total records i have to send are 1,00,000 as i already…
Tanmay Sharma
  • 71
  • 1
  • 5
3
votes
2 answers

Send Azure Blob Storage event notifications to Event Hubs on another account

In Azure, I'm trying to send event notifications from a Storage Account in one Active Directory to an Event Hub in another Active Directory. I'm having trouble figuring out how to share/link the resource. In AWS, I was able to accomplish this by…
3
votes
1 answer

Does multiple units means multiple connection strings in Azure IoT hub and Event hub?

Are connection strings independent of the number of units we create in IoT hub and throughput units in Event hub? Will there be only one connection string for single IoT hub and multiple units are create only when scaling is necessary?
Yatin Gaikwad
  • 1,140
  • 1
  • 13
  • 23
3
votes
1 answer

Is there any way to validate events payload schema while using Azure Event Hub API?

I must send events to Azure Event Hub directly using http protocol. How can I validate payload json schema of the event in Azure (same as validating in a web api) before the event ingestion?
raduS
  • 33
  • 5
3
votes
1 answer

How to expose metrics stored in azure tables or Event Hub to prometheus?

After following the following guide here I have collected all my vm logs and metrics to a single storage account in a set of tables. I can also set this up to expose the metrics to Azure Event Hub. I would now like to expose these metrics to…
OneTwo
  • 2,291
  • 6
  • 33
  • 55