0

How do I receive device messages in Event Hub using Python? I am seeing summary messages from Event Hub, but detailed device messages are missing.

Using the Azure IOT Remote Monitoring Example (https://github.com/Azure/azure-iot-remote-monitoring, https://www.azureiotsolutions.com/Accelerators) and IOT Hub Explorer https://github.com/Azure/iothub-explorer provides messages like these:

==== From: 'chiller-01.0' at '2018-06-27T17:35:13.835Z' ==== { "temperature": 74.8813305672404, "temperature_unit": "F", "humidity": 67.345695720448, "humidity_unit": "%", "pressure": 155.648674178239, "pressure_unit": "psig" } ---- application properties ---- { "iothub-message-schema": "chiller-sensors;v1", "iothub-creation-time-utc": "2018-06-27T17:35:13.5066336Z", "$$CreationTimeUtc": "2018-06-27T17:35:13+00:00", "$$MessageSchema": "chiller-sensors;v1", "$$ContentType": "JSON" }

To get those messages in python, I am using the azure-event-hubs-python library and examples at https://github.com/Azure/azure-event-hubs-python to read IOT messages from an Event Hub using the Event Hub connection string specified for the IOT Hub on the Operations tab. The IOT Hub Explorer written in Node JS uses the IOT Hub Connection String, but writing the corresponding code using python fails and EventHubClient.from_connection_string fails because 'EndPoint' is not in the connection string. Changing 'HostName' to 'EndPoint' in the connection string gives an error in connection.pyx:

File "./src/connection.pyx", line 20, in uamqp.c_uamqp.create_connection TypeError: expected bytes, NoneType found

Pre-pending the protocol sb:// to the endpoint complains that the EventHub is not specified:

ValueError: No EventHub specified

Adding ;EntityPath= gets further, but fails with a CBS Token authentication error:

azure.eventhub.EventHubError: Receive failed: CBS Token authentication failed. Status code: 401 Description: b'The specified SAS token has an invalid signature. It does not match either the primary or secondary key.'

Using the Event Hub Connection string for the IOT Hub connects and receives messages fixes these errors, but receives summary data only without any device telemetry, for example:

{
  "count": 0,
  "total": 0,
  "minimum": 0,
  "maximum": 0,
  "average": 0,
  "resourceId": "/SUBSCRIPTIONS/{SUBSCRIPTION}/RESOURCEGROUPS/{RESOURCEGROUP}/PROVIDERS/MICROSOFT.DEVICES/IOTHUBS/{IOTHUB}", 
  "time": "2018-06-27T16:59:00.0000000Z", 
  "metricName": "jobs.failed", 
  "timeGrain": "PT1M"
}

Since the device data is missing, I am not sure if I am passing the correct parameters to the Python EventHubClient, or if the client is just not returning them. This stack exchange from a year ago How can I receive from Azure IoT Hub messages sent by devices? indicated that reading the telemetry using Python was not available, but since the Python EventHub SDK is out there I'm hoping this has been fixed.

vahdet
  • 6,357
  • 9
  • 51
  • 106
Art Huston
  • 1
  • 1

1 Answers1

0

Yes, reading the telemetry using python was available with the latest EventHubs SDK. I don't know if you have referred to the issue(#28) posted on GitHub. I think the response commented by annatisch on Apr 28 is helpful for this issue.

Michael Xu
  • 4,382
  • 1
  • 8
  • 16
  • Thanks, Michael that's really helpful. I'm going through the instructions of Issue #28 now. – Art Huston Jun 28 '18 at 16:24
  • If there is any problem, please feel free let me know. – Michael Xu Jun 29 '18 at 01:24
  • Michael, I am still having issues. As indicated in the earlier problem statement, I *am* able to connect to the Event Hub using the event hub namespace address, shared access key and shared access key value, but I only see summary data for the events and no detail data. ADDRESS = 'amqps://{eventhub-namespace-name}.servicebus.windows.net/{event-hub-name}' SHARED_ACCESS_KEY_NAME = 'RootManageSharedAccessKey' SHARED_ACCESS_KEY_VALUE = '{shared access key value}' client = EventHubClient(ADDRESS, SHARED_ACCESS_KEY_NAME, SHARED_ACCESS_KEY_VALUE) – Art Huston Jun 29 '18 at 11:21
  • Using the Issue #28 example, I used the CLI to get the endpoint and constructed the EventHubClient like this: `code` ADDRESS = 'amqps://iothub-ns-.servicebus.windows.net/' USERNAME = "iothubowner" PASSWORD = "" client = EventHubClient(ADDRESS, username=USERNAME, password=PASSWORD) receiver = client.add_receiver(consumer_group="$default", partition="0") batch = receiver.receive(timeout=5)`code` – Art Huston Jun 29 '18 at 11:24
  • This gives me the following error: uamqp.errors.TokenAuthFailure: CBS Token authentication failed. Status code: 404 Description: b'@H+\x05\x01' – Art Huston Jun 29 '18 at 11:25
  • Connecting to the Event Hub using the Event Hub connection string from the Event Hub dashboad works, but I am unable to see any device messages. client = EventHubClient.from_connection_string(ADDRESS) receiver = client.add_receiver(consumer_group="$default", partition="0") while True: batch = receiver.receive(timeout=5) for message in batch: payload = str(next(message.body), 'utf8') print(payload) annotations = message._annotations print(annotations)` – Art Huston Jun 29 '18 at 12:13
  • Our requirement is to receive all the monitoring information that is described here: https://learn.microsoft.com/en-us/azure/iot-hub/iot-hub-operations-monitoring, e.g. device identity, device to cloud communications, cloud to device communications, file uploads and message routing. Perhaps this is something in the setup of the IOT Hub? The Operations Monitoring option for the IOT Hub in the Dashboard says this is deprecated and recommends moving to use the Diagnostic settings and use the Azure Monitor. I have tried both. Does Azure Monitor have another API that I should use? – Art Huston Jun 29 '18 at 15:21
  • @ArtHuston, you can find the Event Hub compatible endpoint in Operation monitoring tab in Azure Portal, and then get the address and key from the endpoint. It can connect successfully. Did you tried with azure-mgmt-eventhub(https://learn.microsoft.com/en-us/python/api/azure.mgmt.eventhub?view=azure-python)? – Michael Xu Jul 02 '18 at 07:47
  • (I've been away for a few days for the holiday). As noted above, I able to connect to the Event Hub, but when I do I see only summary information about the events and not the detailed telemetry events that I expect to see. We need all the monitoring information that is here: learn.microsoft.com/en-us/azure/iot-hub/…, e.g. device identity, device to cloud communications, cloud to device communications, file uploads and message routing. Is Event Hub the correct way to get that? – Art Huston Jul 08 '18 at 21:10