Questions tagged [azure-eventhub]

Microsoft Azure EventHubs is a highly-scalable high-throughput low-latency durable event ingestion system in Azure cloud. EventHubs serves as an event buffer for applications to process events with variable load profiles. Microsoft Azure Event Hubs is comparable to Apache Kafka or AWS Kinesis. This is a fully-managed service-offering built & run by the teams at Microsoft.

Microsoft Azure EventHubs is a highly scalable high throughput low latency durable event ingestion system in Azure cloud. EventHubs serves as an event buffer for applications to process events with variable load profiles.

Stream millions of events per second into multiple applications

Event Hubs is a highly scalable publish-subscribe ingestor that can intake millions of events per second so that you can process and analyze the massive amounts of data produced by your connected devices and applications. Once collected into Event Hubs you can transform and store data using any real-time analytics provider or with batching/storage adapters.

Enable applications to process events with variable load profiles

Today’s connected world is defined by big data. Big data originates from many sources such as connected cars and thermostats that produce telemetry data every few minutes, application performance counters that generate events every second or mobile apps that capture telemetry for every user’s individual action. In this connected world, you need a managed ingestor service capable of elastic scale in order to accommodate variable load profiles as well as the spikes caused by intermittent connectivity.

Connect millions of devices across platforms

The rapid proliferation of connected devices raises challenges due to the variety of platforms and protocols involved. Connecting these disparate data sources while handling the scale of the aggregate stream is a significant challenge. Event Hubs provide easy provisioning of capacity to ingest events from millions of devices while preserving event order on a per device basis. Support for AMQP and HTTP allow many platforms to work with Event Hubs. Native client libraries also exist for popular platforms.

1941 questions
8
votes
1 answer

Getting 'An existing connection was forcibly closed by the remote host.' from Azure

While trying to follow the 'Get Started with Events Hub' example for Microsoft Azure (http://azure.microsoft.com/en-us/documentation/articles/service-bus-event-hubs-csharp-ephcs-getstarted/#introduction), when I try to run either receiver or sender…
Omer Meshar
  • 81
  • 1
  • 2
8
votes
1 answer

Should event hubs be split on message type?

I am considering using Azure event Hub for a project I am currently working on. We are using Service Bus Queues for commands today and here we are using one queue per messagetype. Would it make sense to have several Event Hubs or is it better to use…
7
votes
0 answers

Azure Event hubs for Kafka : Attempt to join group failed due to unexpected error

I'm facing an issue related to Azure Event Hubs for Kafka. I have a Kafka consumer processing a message topic name "ABC" that has the below config: ConsumerConfig values: allow.auto.create.topics = true auto.commit.interval.ms =…
Tapioca
  • 71
  • 2
7
votes
2 answers

How do you maintain idempotency with Azure EventGrid webhooks?

I have configured an EventGrid subscription to initiate a web hook call for events in a resource group when a resource is created. The web hook call is successfully handled, and I return a 200 OK. To maintain idempotency, I store all events that…
7
votes
3 answers

PySpark: Deserializing an Avro serialized message contained in an eventhub capture avro file

Initial situation AVRO serialized events are sent to an azure event hub. These events are stored persistently using azure event hubs capture feature. Captured data, along with event hub metadata, is written in Apache Avro format. The original events…
devel0p3r
  • 71
  • 1
  • 3
7
votes
0 answers

Azure eventhub capture replay

Scenario I create an eventhub on Jan 1st 2018. My eventhub retention period is set to be 1 day. I enable the "capture", leaving the default capture parameters of every 5 mins or 300 mb, feature and store in a container named "customerevents" I send,…
CheGuevarasBeret
  • 1,364
  • 2
  • 14
  • 33
7
votes
3 answers

Is Event Hub's intended to be used for Event Sourcing / append-only log architectures

Event Hubs don't let you store messages longer than 7 (maybe up to 30) days. What is Azure's suggested architecture for PaaS Event Sourcing with these limitations? If it's Event Hub + snapshotting, what happens if we somehow need to rebuild that…
randomsolutions
  • 2,075
  • 1
  • 19
  • 22
7
votes
2 answers

Azure Function Event Hub Trigger reliability

I'm a bit confused regarding the EventHubTrigger for Azure functions. I've got an IoT Hub, and am using its eventhub-compatible endpoint to trigger an Azure function that is going to process and store the received data. However, if my function fails…
Schweder
  • 1,464
  • 2
  • 13
  • 19
7
votes
1 answer

Azure Event Hub - multiple event types in same Event Hub

I have set up Azure Event Hub. There are 2 publishers: Publisher 1 (with Send Policy) Publisher 2 (with Send Policy) Publisher 1 will send Event 1 and Publisher 2 will send Event 2. Both Event 1 & Event 2 are different format. Question 1: This…
khar
  • 201
  • 4
  • 12
7
votes
1 answer

How do I enable the create consumer group button for event hubs in Azure?

I cannot create Consumer Groups for any event hub I create. The "+" button is always disabled.
Richard Kemp
  • 313
  • 1
  • 2
  • 7
7
votes
2 answers

EventHub Exception :Cannot allocate more handles to the current session or connection

I have C# console application through which I am sending data to event hub frequently.This console application basically read some data from SQL storage and start pushing data to event hub. This entire program run in endless loop/fashion like in…
JARVIS
  • 765
  • 1
  • 8
  • 28
7
votes
2 answers

How can I remove event hub partitions from Azure blob storage through code?

I am using Azure Event Hubs in a C# Winforms project. I create EventProcessorHost and EventReciever objects to carry out the work of retrieving messages from the event hub and displaying them. Part of my message retrieval process involves creating a…
Mike T
  • 482
  • 1
  • 6
  • 16
7
votes
2 answers

Where is Azure Event Hub messages stored?

I generated a SAS signature using this RedDog tool and successfully sent a message to Event Hub using the Events Hub API refs. I know it was successful because I got a 201 Created response from the endpoint. This tiny success brought about a…
mr i.o
  • 952
  • 2
  • 10
  • 20
6
votes
1 answer

Understanding Azure Event Hubs partitioned consumer pattern

Azure Event Hub uses the partitioned consumer pattern described in the docs. I have some problems understanding the consumer side of this model when it comes to a real world scenario. So lets say I have 1000 messages send to the event hub with 4…
Tobias von Falkenhayn
  • 1,355
  • 5
  • 26
  • 59
6
votes
1 answer

How to send a message to azure service bus & event to event hub bus from single azure functions in NodeJS?

I have an azure function which makes a promise based http post request and gets a response; now I want to send this response to a service bus and to a different event hub (the azure function is being triggered by a different event hub). function…