0

I am following this tutorial and I already have my code publishing messages to /devices/sm1/events topic, in which sm1 is my device id.

I would like to know how to subscribe to this topic since the tutorial says to use /devices/sm1/config but I am getting empty messages. I already tried use the same "path" used in publishing (/devices/sm1/events), but it also did not work.

It is strange that the name I gave to the topic was sm1 and the topic associated to my device is on GoogleIoT console is exhibited as projects/myprojectname/topics/sm1. So, besides to discover how to subscribe to mentioned topic, I appreciate also any explanation related to the correct way of using pub/sub topics in GoogleIoT (the documentation is not so clear).

This is my subscribe.py:

mqtt_url = "mqtt.googleapis.com"
mqtt_port = 8883
topic = "/devices/sm1/config"

def on_connect(client, userdata, flags, response_code):
    print("Connected with status: {0}".format(response_code))
    client.subscribe(topic, 1)

def on_message(client, userdata, msg):
    print("Topic: {0}  --  Payload: {1}".format(msg.topic, msg.payload))

if __name__ == "__main__":    
    client = mqtt.Client("projects/{}/locations/{}/registries/{}/devices/{}".format(
                         project_id,
                         cloud_region,
                         registry_id,
                         device_id))

    client.username_pw_set(username='unused',
                           password=jwt_maker.create_jwt(project_id,
                                               private_key,
                                               algorithm="RS256"))

    client.tls_set(root_ca,
                   certfile = public_crt,
                   keyfile = private_key,
                   cert_reqs = ssl.CERT_REQUIRED,
                   tls_version = ssl.PROTOCOL_TLSv1_2,
                   ciphers = None)


    client.on_connect = on_connect
    client.on_message = on_message

    print "Connecting to Google IoT Broker..."
    client.connect(mqtt_url, mqtt_port, keepalive=60)
    client.loop_forever()

My output:

Connected with status: 0
Topic: /devices/sm1/config -- Payload:
Topic: /devices/sm1/config -- Payload:

Dalton Cézane
  • 3,672
  • 2
  • 35
  • 60

3 Answers3

2

After reading the discussion in the comments section in the answer by @GabeWeiss, I think there is a bit of confusion of what you want to achieve and how (or what for) are you trying to use Pub/Sub.

Given that I think the issue here is more conceptual, let me first refer you to a generic documentation page about Cloud IoT Core key concepts, where you will actually find some information regarding the relationship between Cloud IoT Core and Pub/Sub. In summary, device telemetry data is published into a Cloud IoT Core topic, which later, through the Data Broker, is published into a Cloud Pub/Sub topic, which you can use externally for other purposes: triggering Cloud Functions, analyzing the stream of data with Dataflow, etc.

Now, if you follow the Quickstart guide of Cloud IoT Core, you will see how, at a given point, you create a device registry which is bound to a Pub/Sub topic where device telemetry events are published. If instead of writing to the default Pub/Sub topic you wish to write to multiple topics, you can follow the explanations under this other section on the documentation.

Finally, getting to the issue of subscribing to topics (Cloud IoT Core topics, and not Pub/Sub topics, as only the former are relevant for the devices), you can subscribe to MQTT topics with the following command, where (as an example), the device is subscribing to the config topic, where configuration updates are published:

# This is the topic that the device will receive configuration updates on.
mqtt_config_topic = '/devices/{}/config'.format(device_id)

# Subscribe to the config topic.
client.subscribe(mqtt_config_topic, qos=1)

Then, with the on_message() function you can can process messages published on the topics where you are actually subscribed. Note that the payload has to be parsed as string (str(message.payload)).

def on_message(unused_client, unused_userdata, message):
    payload = str(message.payload)
    print('Received message \'{}\' on topic \'{}\' with Qos {}'.format(
            payload, message.topic, str(message.qos)))

Then, in your question you stated that you first subscribed to /devices/{device-id}/config, but this might not be what you want, as this is the topic were configuration updates are published (i.e. not where telemetry events are published). Then, I understand that you should subscribe to /devices/{device-id}/events which is the actual MQTT topic where telemetry metrics are published. If that does not work, there might be another issue related, so make sure that you are parsing the message variable correctly, and maybe try to use Pub/Sub with the default topic created with the registry in order to check whether telemetry metrics are being properly published or not.

dsesto
  • 7,864
  • 2
  • 33
  • 50
  • Hello, dsesto. What I want is clear in the question: subscribing to telemetry topics in order to receive notifications from devices, without using the google pub/sub library. All documentation you mentioned I already had read before and, as I stated in my question and in the comments, the documentation is not clear. Also, all the suggestions you said, I had tried before. So, unfortunately, you answer did help in anything. Thanks, anyway. – Dalton Cézane Apr 11 '18 at 16:29
1

Edit: Clarifying based on comment below...

There's two GCP components in play here. There's the MQTT topic (which is the /events topic), which is used by the device to talk to IoT Core. Then there's the projects/myprojectname/topics/sm1 which isn't in IoT Core, it's in Pub/Sub. When you send messages to the /events MQTT topic, IoT Core brokers the payloads from your device that was sent to the /events MQTT topic through to the Pub/Sub topic that was created and attached to the IoT Core registry where your device was registered.

To see those messages, you have to create a subscription in Pub/Sub on the topic projects/myprojectname/topics/sm1. If you go to the console, and Pub/Sub->topics. Click the three dots next to the topic and select "New subscription". Once you have the subscription, you can send some data from your device, then on commandline you can run (assuming you have the gcloud tools installed):

gcloud beta pubsub subscriptions pull --max-messages=3 <subscription_id>

To do anything with the messages, you can script subscribing to the Pub/Sub topic (check out the Pub/Sub APIs) to trigger on messages being added to the topic.

Original message:

Are you sending a config message to the device? The confusion might be that the MQTT topics are one-directional.

So: 1) the /events topic is for device->IoT Core. 2) the /config topic is for IoT Core Admin SDK->device

In another script somewhere, or from the IoT Core UI interface you need to send a configuration message to see the on_message fire properly.

In the IoT Core UI (on console.cloud.google.com) you can drill down to an individual device you have registered, and at the top of the screen, click on "Update Config". A popup window will come up that lets you send a text or a base64 encoded message to that device and it will come in on the /config topic.

Gabe Weiss
  • 3,134
  • 1
  • 12
  • 15
  • It is still not clear. The documentation does not treat nothing about that. As explained in the question, I have the following topic created at IoT Core `projects/myprojectname/topics/sm1`. But my client has to publish in a topic named `events` and I still do not know in which topic I should subscribe to receive the interested messages of the desired topic `sm1`... – Dalton Cézane Apr 04 '18 at 17:48
  • Ah, okay I think I understand, will edit my answer...sec – Gabe Weiss Apr 04 '18 at 18:17
  • I changed my topic to `projects/myprojectname/topics/sm1` instead of `/devices/sm1/config`, but I am still not able to see the published values: "Connecting to Google IoT Broker... Connected with status: 0". Should it be related to some kind of permissions? Is there any other things missing? – Dalton Cézane Apr 04 '18 at 19:02
  • You're still trying to connect to IoT Core. Pub/Sub isn't MQTT. There's a separate API to subscribe to Pub/Sub topics. Start here: https://cloud.google.com/pubsub/docs/ – Gabe Weiss Apr 04 '18 at 20:04
  • So, I think it should be described at the tutorials related to IoT Core... even more that in these is described how to create topics... if we learn how to create, we also have to learn how to subscribe and this information is not there. I will read. Thank you. – Dalton Cézane Apr 04 '18 at 20:32
  • Yup, I've got a note in to the docs folks about it as well so we at least have a breadcrumb to the Pub/Sub docs to follow. – Gabe Weiss Apr 04 '18 at 21:37
  • Although in looking, in the Quickstart guide, under the section for "Run a Node.js sample to connect a virtual device and view telemetry", step 5 describes creating the subscription from commandline, and step 7 is the command to read Pub/Sub subscription messages. – Gabe Weiss Apr 04 '18 at 21:44
  • It is still not clear. I just want to create my own application, independent of language. I do not want to use cli, since the topic is already created in IoT Core; I do not want to use the google samples (like [this client](https://github.com/GoogleCloudPlatform/google-cloud-python/tree/master/pubsub))... I just want to read any documentation that tells what are the urls used to subscribe and how is the format of the messages. I am reading the [subscriber documentation](https://cloud.google.com/pubsub/docs/subscriber) and it also does not mention what I need... – Dalton Cézane Apr 05 '18 at 19:52
  • You mean like this: https://cloud.google.com/pubsub/docs/push ? There's two types of subscriptions, that describes the push method, so when you setup the subscription, you setup an endpoint for the messages to be sent. The process is described there. It also has an example of what the response will look like. – Gabe Weiss Apr 05 '18 at 21:19
  • I have a subscription with this address `/projects/myproject/subscriptions/sm1`, but I do not have success with the connection. Here is [my short code](https://pastebin.com/s7ik6E4i). I do not know if the MQTT protocol is valid for this kind of operation, although it should be, like other platforms. – Dalton Cézane Apr 05 '18 at 21:34
  • I don't think that will work, but you can try it, you're missing: `client.connect('mqtt.googleapis.com', 8883)` `client.loop_start()` To initiate the MQTT connection though. In fact, I'm 99% sure it won't, because the MQTT endpoint is only associated with IoT Core, not Pub/Sub. You need to use the Python Pub/Sub APIs in order to read the subscription. – Gabe Weiss Apr 05 '18 at 23:23
  • Do you understand now my situation? There is no such information related to this. I already tried also to see the code of pub/sub library, but it does not use a library to communicate through mqtt. So, I dont know if this protocol is supported for pub/sub by gcp like in others platforms. There are still some gaps in my understanding due to the provided documentation . – Dalton Cézane Apr 06 '18 at 06:35
  • it seems the IoT Core is still very limited, restricted, right? I think so because, apparently, it just works with "events", "state" and "config" topics... It seems they serve only for a purpose: devices configuration/administration. If I want to monitor some sensors, with specific topics, I have to use google pub/sub. In this case, IoT Core can stay unused. Am I right? Thank you again. – Dalton Cézane Apr 19 '18 at 19:24
  • Sorry Dalton, I was traveling. :) With IoT Core you can have up to 10 Pub/Sub topics in each registry. So you could have: `/events/temp/` `/events/pressure/` `/events/humidity/` etc. One reason you wouldn't want to go straight to Pub/Sub (amongst others) is the device management piece. While it's true, you could go straight to Pub/Sub for config, for example, if your device isn't connected when the config goes through, then your device won't get the update unless you write retry, etc. Also, security. You'd have to manage secure communication yourself without IoT Core. – Gabe Weiss May 01 '18 at 16:26
1

I had to surrender to Google Pub/Sub library in order to receive notifications related to my specified topic.

My publish code (only the important parts):

mqtt_url = "mqtt.googleapis.com"
mqtt_port = 8883
mqtt_topic = "/devices/sm1/events"

client = mqtt.Client("projects/{}/locations/{}/registries/{}/devices/{}".format(
                     project_id,
                     cloud_region,
                     registry_id,
                     device_id), protocol=mqtt.MQTTv311)

client.username_pw_set(username='unused',
                       password=jwt_maker.create_jwt(project_id,
                                           private_key,
                                           algorithm="RS256"))

client.tls_set(root_ca, 
               certfile = public_crt, 
               keyfile = private_key, 
               cert_reqs = ssl.CERT_REQUIRED, 
               tls_version = ssl.PROTOCOL_TLSv1_2, 
               ciphers = None)

client.connect(mqtt_url, mqtt_port, keepalive=60)
res = client.publish(mqtt_topic, some_data, qos=1)

In the Google Cloud Platform portal, I had to create a subscription, in the Pub/Sub section, assigning it to my created topic, which was already my default topic (probably linked to /events topic). The created subscription has the following format: 

projects/project_name/subscriptions/subscription_name

My subscribe code, using the Google Pub/Sub library, since it is not possible to use the MQTT protocol:

from google.cloud import pubsub

def callback(message):
    print(message.data)
    message.ack()

project_id = "project_name"
subscription_name = "sm1"

subscriber = pubsub.SubscriberClient()
subscription_name = 'projects/{}/subscriptions/{}'.format(project_id, subscription_name)

subscription = subscriber.subscribe(subscription_name)
future = subscription.open(callback)

try:
    future.result()
except Exception as ex:
    subscription.close()
    raise

I hope this can help anyone. More details can be found here(github).

Dalton Cézane
  • 3,672
  • 2
  • 35
  • 60