Questions tagged [azure-eventhub]

Microsoft Azure EventHubs is a highly-scalable high-throughput low-latency durable event ingestion system in Azure cloud. EventHubs serves as an event buffer for applications to process events with variable load profiles. Microsoft Azure Event Hubs is comparable to Apache Kafka or AWS Kinesis. This is a fully-managed service-offering built & run by the teams at Microsoft.

Microsoft Azure EventHubs is a highly scalable high throughput low latency durable event ingestion system in Azure cloud. EventHubs serves as an event buffer for applications to process events with variable load profiles.

Stream millions of events per second into multiple applications

Event Hubs is a highly scalable publish-subscribe ingestor that can intake millions of events per second so that you can process and analyze the massive amounts of data produced by your connected devices and applications. Once collected into Event Hubs you can transform and store data using any real-time analytics provider or with batching/storage adapters.

Enable applications to process events with variable load profiles

Today’s connected world is defined by big data. Big data originates from many sources such as connected cars and thermostats that produce telemetry data every few minutes, application performance counters that generate events every second or mobile apps that capture telemetry for every user’s individual action. In this connected world, you need a managed ingestor service capable of elastic scale in order to accommodate variable load profiles as well as the spikes caused by intermittent connectivity.

Connect millions of devices across platforms

The rapid proliferation of connected devices raises challenges due to the variety of platforms and protocols involved. Connecting these disparate data sources while handling the scale of the aggregate stream is a significant challenge. Event Hubs provide easy provisioning of capacity to ingest events from millions of devices while preserving event order on a per device basis. Support for AMQP and HTTP allow many platforms to work with Event Hubs. Native client libraries also exist for popular platforms.

1941 questions
10
votes
2 answers

Azure Event Hubs limits and its comparison to pure Kafka cluster

Recently Azure released a feature called Azure Event Hubs for Kafka that allows to use Event Hubs like if it were a Kafka cluster, using the same Kafka libraries. That would allow us to migrate from our current IaaS Kafka solution to a PaaS…
joanlofe
  • 3,360
  • 2
  • 26
  • 44
10
votes
0 answers

Azure Functions Eventhub Input Trigger scaling

We have an EventHub with a retention of 1 day, containing millions of messages. To consume this, we have an Azure Function which reads from this event hub via the Event Hub Binding. The function basically reads the raw bytes, deserializes it into a…
Sam Neirinck
  • 268
  • 5
  • 11
10
votes
4 answers

Java Connecting to azure event hub: SunCertPathBuilderException

I'm not very used to java world, so I'm not sure if my problem is on my Azure set up, or java set up. I am getting the following exception after attempting the tutorial…
Seth
  • 954
  • 1
  • 15
  • 42
10
votes
4 answers

Is it correct to create EventHubClient for every message I send?

I'm creating an EventHub client application that sends message to an Event Hub on Azure. Now when I have a message to send (every 5-6 seconds) I create a new instance of the class EventHubClient, then I use it to send data: public async static…
Andrea Cattaneo
  • 568
  • 1
  • 6
  • 18
10
votes
2 answers

How to get rid of old messages from Azure EventHub?

When we subscribe to an EventHub for a particular type. I see my worker keeps processing the old messages as well. Is there any way to get rid of older messages in the event hub or change the partition's offset value at a desired message…
user3904654
  • 133
  • 1
  • 1
  • 5
9
votes
2 answers

Azure Function: 500 internal internal server error in Run/Test Mode

I want to Test my azure function using the Azure Apps feature to Run/Test mode but it is throwing the '500 internal server error'. I am able to debug the same code in my local environment but when to trigger the same code on the azure portal then it…
Stark
  • 604
  • 3
  • 11
  • 30
9
votes
2 answers

Can the EventHubProducerClient be reused across multiple request

I'm leveraging services of the Azure SDK to log telemetry data to Azure Event Hubs. My client application calls an API endpoint which instantiates the EventHubProducerClient to log data to Event Hubs. I am managing many api calls and was wondering…
Mike Check
  • 89
  • 3
9
votes
1 answer

New receiver with higher epoch Error in EventProcessorHost

I am using EventProcessorHost for reading Eventhub data and i got following error. New receiver with higher epoch of '798393595' is created hence current receiver with epoch '17' is getting disconnected. If you are recreating the receiver,…
asthanarht
  • 139
  • 2
  • 8
9
votes
4 answers

Simplest way to log all messages from an Azure Event Hub

I'm using a service which outputs to an Event Hub. We want to store that output, to be read once per day by a batch job running on Apache Spark. Basically we figured, just get all messages dumped to blobs. What's the easiest way to capture messages…
Iain
  • 1,797
  • 1
  • 20
  • 38
9
votes
1 answer

Why not always configure for max number of event hub partitions?

The Azure Event Hubs overview article states the following: The number of partitions is specified at the Event Hub creation time and must be between 8 and 32. Partitions are a data organization mechanism and are more related to the degree of…
kspearrin
  • 10,238
  • 9
  • 53
  • 82
9
votes
1 answer

SignalR scaling out with Azure EventHub

I am looking high frequency scaling solution for SignalR. I am wondering if I can do it with Azure EventHub. If I use EventHub as my backplane for SignalR messages, will it become a bottleneck for me? I've checked this page but there is nothing…
Andrei
  • 42,814
  • 35
  • 154
  • 218
8
votes
3 answers

Azure IoT Hub, EventHub and Functions

I have an IoTHub with a route that points to an EventHub which triggers a Functions. I'm having problem getting the DeviceId and other IoT Hub properties from the event object without adding those explicitly to the payload. If I set the input type…
Kimmen
  • 717
  • 6
  • 18
8
votes
1 answer

Route events to eventhub EventProcessor

I have events of different types. For example, some data is telemetry data, some is error information etc. I thought it would be a good idea to create several IEventProcessor implementations, one for each event type. So each implementation will…
Peter Bons
  • 26,826
  • 4
  • 50
  • 74
8
votes
2 answers

How to pass parameters to an implementation of IEventProcessor

I am busy implementing a EventProcessorHost client for an azure EventBus client. I have a class that implements IEventProcessor as follows: public class MyEventProcessor : IEventProcessor { Stopwatch checkpointStopWatch; …
Zapnologica
  • 22,170
  • 44
  • 158
  • 253
8
votes
1 answer

Should I put my events inside a queue after getting them from Azure Event Hub?

I'm currently developing an application hosted on Azure that uses Azure Event Hub. Basically I'm sending messages (or should I say, events) to the Event Hub from a Web API, and I have two listeners: a Stream Analytics task for real-time analysis a…
ken2k
  • 48,145
  • 10
  • 116
  • 176