Questions tagged [azure-eventhub]

Microsoft Azure EventHubs is a highly-scalable high-throughput low-latency durable event ingestion system in Azure cloud. EventHubs serves as an event buffer for applications to process events with variable load profiles. Microsoft Azure Event Hubs is comparable to Apache Kafka or AWS Kinesis. This is a fully-managed service-offering built & run by the teams at Microsoft.

Microsoft Azure EventHubs is a highly scalable high throughput low latency durable event ingestion system in Azure cloud. EventHubs serves as an event buffer for applications to process events with variable load profiles.

Stream millions of events per second into multiple applications

Event Hubs is a highly scalable publish-subscribe ingestor that can intake millions of events per second so that you can process and analyze the massive amounts of data produced by your connected devices and applications. Once collected into Event Hubs you can transform and store data using any real-time analytics provider or with batching/storage adapters.

Enable applications to process events with variable load profiles

Today’s connected world is defined by big data. Big data originates from many sources such as connected cars and thermostats that produce telemetry data every few minutes, application performance counters that generate events every second or mobile apps that capture telemetry for every user’s individual action. In this connected world, you need a managed ingestor service capable of elastic scale in order to accommodate variable load profiles as well as the spikes caused by intermittent connectivity.

Connect millions of devices across platforms

The rapid proliferation of connected devices raises challenges due to the variety of platforms and protocols involved. Connecting these disparate data sources while handling the scale of the aggregate stream is a significant challenge. Event Hubs provide easy provisioning of capacity to ingest events from millions of devices while preserving event order on a per device basis. Support for AMQP and HTTP allow many platforms to work with Event Hubs. Native client libraries also exist for popular platforms.

1941 questions
5
votes
4 answers

Not able connect to EventHub via KAFKA api

am getting below exception while connecting Event Hub via kafka libraries. Caused by: org.apache.kafka.common.protocol.types.SchemaException: Error reading field 'sasl_auth_bytes': Bytes size -1 cannot be negative ERROR [pool-6-thread-1] STREAM -…
5
votes
3 answers

How do I debug Azure EventHubs?

I am trying to send data to an azure event hub later to be consumed by stream analytics and PowerBI. My C# application is sending a simple test string (JSON formatted) while I am testing the Event Hub, but when I want to process the data in the…
Matt Douhan
  • 2,053
  • 1
  • 21
  • 40
5
votes
1 answer

Unable to connect to Event HUB from Service bus explorer

I have downloaded the Service Bus Explorer from GIT. I am able to connect to the Service Bus Queue & Topics. But when i am trying to connect to the Event Hub, i am getting some issue. Error--> Exception: System.UnauthorizedAccessException: The…
lokanath das
  • 736
  • 1
  • 10
  • 35
5
votes
2 answers

EventHubTrigger function app, pass event hub name and consumer group name dynamically to event hub attribute

We know the Connection string for event hub can be used from local.setting.json file. So for the same function app in different environments, I can add the event hub connection string setting in Application settings in the azure portal. As the…
Vivek Ranjan
  • 1,432
  • 2
  • 15
  • 37
5
votes
4 answers

Is it possible to send MQTT message to Event Hub? Or is there another way?

I am new with Azure, MQTT, and IoT. I have been given a task to create a POC on how to send MQTT message to Azure Event Hub. And from the Event Hub processing, save the message to cloud sql server. Is this possible? Since i read from here, that…
MRu
  • 1,225
  • 7
  • 20
  • 44
5
votes
0 answers

(Go) Azure Event Processor Host to read events from now?

I am using azure-event-hubs-go client to read events from an Azure IoTHub. I am creating an Event Processor Host and providing an Handler to process events (as show in the sample in https://github.com/Azure/azure-event-hubs-go). How do I set the…
Tany
  • 1,252
  • 14
  • 30
5
votes
1 answer

Deserializing Event Hub messages in Azure Databricks

I have an Azure Databricks script in Python that reads JSON messages from Event Hub using Structured Streaming, processes the messages and saves the results in Data Lake Store. The messages are sent to the Event Hub from an Azure Logic App that…
5
votes
1 answer

HTTP POST in Arduino MKGSM to Eventhub

I'm using the Arduino library MKGSM in order to make an HTTP POST to my Azure Eventhub. I am confident with the parameters provided as I have tried them by hand with curl and the HTTP POST did work, so the problem is for sure in my Android syntax.…
Alvaro Gomez
  • 350
  • 2
  • 7
  • 22
5
votes
1 answer

Best way to Fetch connectionState from 1000's of devices - Azure IoTHub

Best way to Fetch connectionState from 1000's of devices. Currently there is not that much devices, but need an efficient solution. Based on my understanding, currently I can fetch connectionState using IotHub Queries(select * from devices) or…
5
votes
2 answers

Consume events from EventHub In Azure Databricks using pySpark

I could see spark connectors & guidelines for consuming events from Event Hub using Scala in Azure Databricks. But, How can we consume events in event Hub from azure databricks using pySpark? any suggestions/documentation details would help.…
Shankar
  • 571
  • 14
  • 26
5
votes
2 answers

Should we use the Microsoft.Azure.EventHubs.Processor library with .NET framework

Context We're currently using Microsoft.Azure.ServiceBus.EventProcessorHost for data ingestion from an Azure Event-Hub. We run on .NET framework, not .NET Core. This Microsoft announcement (published Feb 2, 2017) suggests the newer…
Francis MC
  • 53
  • 3
5
votes
1 answer

Is it possible to map a custom domain (Via C Name or A Record) to an azure event hub

Would it be possible to map a url to point to an instance of the azure event hub? Could I add a CNAME to a domain and point to sb://.servicebus.windows.net I think that A Records map to a static IP address, and as there doesn't seem to be an option…
Mark McGookin
  • 932
  • 1
  • 16
  • 39
5
votes
1 answer

How can I use a GUID as ID in CosmosDB when I insert data from Azure Streaming Job

Hi I have a Azure Streaming Job that takes data from an Event Hub and inserts them in a CosmosDB Database. I would like to use a GUID as id when Data are inserted instead of a value inside the document. How can I achieve that in the query?
Georgia Kalyva
  • 737
  • 2
  • 9
  • 23
5
votes
1 answer

Best Practices: to partition eventhub data & achieve high-scale, low-latency and high-throughput via azure eventhubs to external store (azure blobs)

As part of a security product I have high scale cloud service (azure worker role) that reads events from event hub, batches them to ~2000 and stores in blob storage. Each event has a MachineId (the machine that sent it). Events are coming from the…
Zorik
  • 207
  • 1
  • 3
  • 9
5
votes
1 answer

How to optimize throughput when sending to EventHubs with .Net core app

How can I create multiple EventHubClients that use multiple underlying TCP connects to allow fast writes to the EventHubs from a .net core web app? In the programming guide for EventHubs…
viblo
  • 4,159
  • 4
  • 20
  • 28