Questions tagged [azure-eventhub]

Microsoft Azure EventHubs is a highly-scalable high-throughput low-latency durable event ingestion system in Azure cloud. EventHubs serves as an event buffer for applications to process events with variable load profiles. Microsoft Azure Event Hubs is comparable to Apache Kafka or AWS Kinesis. This is a fully-managed service-offering built & run by the teams at Microsoft.

Microsoft Azure EventHubs is a highly scalable high throughput low latency durable event ingestion system in Azure cloud. EventHubs serves as an event buffer for applications to process events with variable load profiles.

Stream millions of events per second into multiple applications

Event Hubs is a highly scalable publish-subscribe ingestor that can intake millions of events per second so that you can process and analyze the massive amounts of data produced by your connected devices and applications. Once collected into Event Hubs you can transform and store data using any real-time analytics provider or with batching/storage adapters.

Enable applications to process events with variable load profiles

Today’s connected world is defined by big data. Big data originates from many sources such as connected cars and thermostats that produce telemetry data every few minutes, application performance counters that generate events every second or mobile apps that capture telemetry for every user’s individual action. In this connected world, you need a managed ingestor service capable of elastic scale in order to accommodate variable load profiles as well as the spikes caused by intermittent connectivity.

Connect millions of devices across platforms

The rapid proliferation of connected devices raises challenges due to the variety of platforms and protocols involved. Connecting these disparate data sources while handling the scale of the aggregate stream is a significant challenge. Event Hubs provide easy provisioning of capacity to ingest events from millions of devices while preserving event order on a per device basis. Support for AMQP and HTTP allow many platforms to work with Event Hubs. Native client libraries also exist for popular platforms.

1941 questions
0
votes
1 answer

Possible to connect to an azure event hub through a browser and websockets?

I'm trying to implement a web based chat program, my plan is to update an azure cosmos database with chat messages, then push an event to an azure event hub. The missing element is for connected web browsers to receive these events. I tried using…
meds
  • 21,699
  • 37
  • 163
  • 314
0
votes
3 answers

MaxBatchSize in Host.json for Azure Function not working

I have below confiugration for my Azure Function in host.json & I am sending 30000 msgs in multiple batches to EvenHub but I see that my azure function doesn't pick more than 64 which is a default value. Any particular reason that it doesn't pick…
Punit
  • 1,347
  • 3
  • 20
  • 39
0
votes
1 answer

What is different between EventHub Client with EventHub Processor?

I find there are two way to receive EventHub message data: Using EventHub Processor, it seems will use checkpoint to save. It will make sure when the process running EventProcessor on a specific partition dies/crashes. public class…
v11
  • 2,124
  • 7
  • 26
  • 54
0
votes
1 answer

Azure Event Grid and Azure Event Hub

I am using Azure Services with node js. I am pushing and getting the messages from service bus. I want to know what is the role that of event grid , event hub. When i create them they want end point what is that. And how should i proceed
Amaan Imtiyaz
  • 254
  • 6
  • 16
0
votes
1 answer

SPARK java.lang.OutOfMemoryError: Not enough memory to build and broadcast the table to all worker nodes

I am using spark to join a static dataset I fetch from azure storage and streaming dataset I get from eventhub. I have not used broadcast join anywhere. I tried df.explain() after joining, it shows sortmerge join is happening. I am not sure why I am…
0
votes
2 answers

Azure eventhub to NoSQL database

I am currently using azure eventhub to send/receive data. I want to connect my IoT device to NoSQL database from Azure in a way that JSON message produced from IoT data be stored in NoSQL database(maybe cosmosDB). I was trying to using function app…
Brian Lee
  • 173
  • 3
  • 14
0
votes
2 answers

Error code: BadArgument Error message while sampling data from Azure Stream Analytics Job

I tried to take sample data from a running Azure Stream Analytics Job. The job is running successfully and the output is writing into cosmos DB successfully. But when I tried to take sample data from the streaming input, Sampling is failed and I am…
Antony
  • 970
  • 3
  • 20
  • 46
0
votes
1 answer

Passing events through Event Hub

I am passing high number of events(more than 1000)/second from multiple sensors to a single event hub. While passing data from sensor to an event hub i don't have access to sensor id, so i can only use 1 partition as event ordering is essential.…
0
votes
1 answer

How does EventHubClient.CreateFromConnectionString / SendAsync when event hub is down or connection string is wrong

I have a .net application writing to event hub via https://learn.microsoft.com/en-us/dotnet/api/microsoft.servicebus.messaging.eventhubclient?view=azure-dotnet : var eventHubClient = EventHubClient.CreateFromConnectionString(connectionString,…
Greg Bala
  • 3,563
  • 7
  • 33
  • 43
0
votes
0 answers

How to get the consumer lag in eventhub

I am using eventhub python sdk consumer for real time processing of the data from eventhub. I have currently 2 partitions of my topic. I would like to know about my consumer lag. I know what is the sequence number of the message I am processing, but…
Nipun
  • 4,119
  • 5
  • 47
  • 83
0
votes
1 answer

Do azure eventHub binding(Java) supports partitionKey for @EventHubOutput?

I could find a document or example of eventhub output binding supporting a partitionKey. The below link says it is an option for trigger metadata however there is no java…
T D
  • 1,080
  • 2
  • 13
  • 20
0
votes
1 answer

How to provide the complete path of the container in AzureStorageCheckpointLeaseManager class

I am using EPH ( EventProcessorHost) class of Azure python SDK to receive events from the eventhub. It actually uses AzureStorageCheckpointLeaseManager for checkpointing and partitioning in the storage account. But I cannot see where I can write…
Nipun
  • 4,119
  • 5
  • 47
  • 83
0
votes
1 answer

How can an Azure EventProcessor access a SignalR HubContext?

I have a .net core 3.0 web application. In Startup.cs, I register an EventProcessor (from Microsoft.Azure.EventHubs.Processor) that listens to an Azure EventHub for events. I do it like this: await…
Jon Th
  • 347
  • 2
  • 13
0
votes
2 answers

Azure eventhub library for python

I am using eventhub for ingesting a lot of events. I have multiple consumers which are running behing a scaling group reading these events from the eventhub which has multiple partitions. I was going through the Azure SDK in python and was confused…
Nipun
  • 4,119
  • 5
  • 47
  • 83
0
votes
1 answer

How to stream data from Azure eventhub using Pyspark?

I have created an event hub in azure and published some messages on the same using a python script. I'm able to fetch the messages from event hub using another python script but I'm unable to stream the messages using Pyspark. Below is the Pyspark…
1 2 3
99
100