5

I am trying to send data to an azure event hub later to be consumed by stream analytics and PowerBI.

My C# application is sending a simple test string (JSON formatted) while I am testing the Event Hub, but when I want to process the data in the event hub it tells me no data has been sent.

I am unsure how to debug the hub to see why the message does not show up, I tried google for a debug document but found none when searching for debugging azure event hubs

How can I find any error messages about this?

My C# code is simplified like this for testing purposes

private async Task SendMessageToEventHub(string messageToSend)
{
    var connectionStringBuilder = new EventHubsConnectionStringBuilder(EventHubConnectionString)
    {
        EntityPath = EventHubName
    };

    eventHubClient = EventHubClient.CreateFromConnectionString(connectionStringBuilder.ToString());

    try
    {
        await eventHubClient.SendAsync(new EventData(Encoding.UTF8.GetBytes(messageToSend)));
    } catch (Exception exception) {
        _logger.LogInformation($"-PimLog- -ProductInfoController- {DateTime.Now}, > Exception: {exception.Message} ");
    }

    _logger.LogInformation($"-PimLog- -ProductInfoController- {DateTime.Now}, EventHub Message Sent Successfully ");
}

I do see the log message that Message Sent Successfully so no obvious error is there.

When I try and run the query in the azure portal I get the following message

There is no data from input 'pimhub'. Please make sure the input source has data and then try again.

How can I debug further?

Matt Douhan
  • 2,053
  • 1
  • 21
  • 40

3 Answers3

3

In my experience, i've always used Azure portal metrics to debug the Event Hub and stream Analytics chain.

First of all, you should go to the event hub overview and check if the output requests has the same amount as input requests, as shown in the picture below

enter image description here

Morehover, check the User Errors and Server Errors metrics to see if events are sent with no errors. If all metrics are normal, you should check the Stream Analytics. Use metrics to check if there are one of the following errors

  1. Data Conversion Erorrs --> Errors when Stream Analytics Try to save to te output
  2. Input Deserialization Errors --> Most of the time are Query Errors
  3. Runtime Errors --> Have to investigate

I also noticed that many times the portal can't capture events from the EH in order to test the ASA query (as you mentioned). Honestly speacking i don't know why, but a trick may be to copy the json object you would send in a file, then test the query through the portal using the option "Upload Sample Data From File" (ASA --> Query --> Inputs).

Hope it helps

ElettroAle
  • 119
  • 1
  • 8
  • I have the same problem as OP. I end up writing a second program in a different language to read my messages back and it works fine. It looks like that the built-in deserializer used by Azure Portal UI could only deal with JSON, CSV, AVRO. Once I changed the test application which generate message to send JSON formatted message instead of plain text. Things start to look much more positive. – airmanx86 Oct 23 '19 at 00:08
0

To debug if your messages are actually in Event Hub - and in the format you expect, the Service Bus Explorer tool is very useful: https://github.com/paolosalvatori/ServiceBusExplorer (unlike the name suggests, it fully supports Event Hubs as well).

If messages are showing up there but you still don't get them in ASA, double-check your connection settings for your input in ASA, especially things like the consumer group (you should have a dedicated consumer group for your ASA job).

silent
  • 14,494
  • 4
  • 46
  • 86
  • But I cannot even preview and I do have the preview consumer group that’s setup by default – Matt Douhan Aug 15 '19 at 19:03
  • what do you mean by "cannot even preview"? You should just create a new consumer group when connecting ASA. Rule of thumb: NEVER use the $Default CS - unless maybe for something like debugging with the service bus explorer – silent Aug 15 '19 at 19:05
  • I haven’t connected ASA yet I am just trying to inject messages into the hub and then click process data and then click explore but when I try run the query in the comment above it says no data in pimhub, that’s what I mean – Matt Douhan Aug 15 '19 at 19:08
  • I'm not sure I follow you. Where exactly do you click on "explorer"? Which portal? Which service? – silent Aug 15 '19 at 19:09
  • In azure portal, event hub name space, event hub – Matt Douhan Aug 15 '19 at 19:10
  • well, that is actually connecting ASA... Instead of doing this, please use the Service Bus Explorer tool that I linked to see if messages are actually there. – silent Aug 15 '19 at 19:12
0

In order to view data using an Azure Stream Analytics query you must first Deploy the query.

Azure Stream Analytics Query

Once you've done this you need to open the query and set a time range for the sample data you want to query by clicking: "Select time range".

Azure Stream Analytics - Select Time Range

You should now be able to query your data using the query tool.

Azure Stream Analytics - Query Restults

timdougall
  • 71
  • 2