Google Cloud IoT - Invalid MQTT publish topic

Question

I am using a Python client with paho-mqtt to publish in this specific topic of Google Cloud IoT: projects/my_project/topics/sm1. My code is below, based on examples of the Google IoT documentation:

import paho.mqtt.client as mqtt
import ssl, random, jwt_maker
from time import sleep

root_ca = './../roots.pem'
public_crt = './../my_cert.pem'
private_key = './../my_pr.pem'

mqtt_url = "mqtt.googleapis.com"
mqtt_port = 8883
mqtt_topic = "/projects/my_project/topics/sm1"
project_id   = "my_project"
cloud_region = "us-central1"
registry_id  = "sm1"
device_id    = "sm1"

connflag = False

def error_str(rc):
    """Convert a Paho error to a human readable string."""
    return "Some error occurred. {}: {}".format(rc, mqtt.error_string(rc))

def on_disconnect(unused_client, unused_userdata, rc):
    """Paho callback for when a device disconnects."""
    print("on_disconnect", error_str(rc))

def on_connect(client, userdata, flags, response_code):
    global connflag
    connflag = True
    print("Connected with status: {0}".format(response_code))

def on_publish(client, userdata, mid):
    print("User data: {0} -- mid: {1}".format(userdata, mid))
    #client.disconnect()

if __name__ == "__main__":

    client = mqtt.Client("projects/{}/locations/{}/registries/{}/devices/{}".format(
                         project_id,
                         cloud_region,
                         registry_id,
                         device_id))

    client.username_pw_set(username='unused',
                           password=jwt_maker.create_jwt(project_id,
                                               private_key,
                                               algorithm="RS256"))

    client.tls_set(root_ca,
                   certfile = public_crt,
                   keyfile = private_key,
                   cert_reqs = ssl.CERT_REQUIRED,
                   tls_version = ssl.PROTOCOL_TLSv1_2,
                   ciphers = None)

    client.on_connect = on_connect
    client.on_publish = on_publish
    client.on_disconnect = on_disconnect

    print("Connecting to Google IoT Broker...")
    client.connect(mqtt_url, mqtt_port, keepalive=60)
    client.loop_start() 

    while True:
        sleep(0.5)
        print connflag
        if connflag == True:
            print("Publishing...")
            ap_measurement = random.uniform(25.0, 150.0)
            #payload = "sm1/sm1-payload-{}".format(ap_measurement)
            res = client.publish(mqtt_topic, ap_measurement, qos=1)
            if not res.is_published():
               print("Data not published!!")
            else:
               print("ActivePower published: %.2f" % ap_measurement)
        else:
            print("Waiting for connection...")

When I run, the client connect but does not publish. At Google IoT Console, I can see the following error message:

Invalid MQTT publish topic: projects/my_project/topics/sm1

And here is the output:

Connecting to Google IoT Broker...
Connected with status: 0 -- msg: Connection Accepted.
True
Publishing...
Data not published!!
('on_disconnect', 'Some error occurred. 1: Out of memory.')

It is really strange, because the topic is there, created, and have a subscription associated to it!

Any help will be appreciated. I already read the following documentation and code:

score 6 · Accepted Answer · edited Jul 14 '21 at 19:23

6

You have the incorrect topic name.

It's confusing (I just hit this too).

The client ID (as you have) must be:

"projects/{}/locations/{}/registries/{}/devices/{}".format(
  project_id,
  cloud_region,
  registry_id,
  device_id
)

The topics must then be of the form:

/devices/{}/config
/devices/{}/state
/devices/{}/events
/devices/{}/events/some/other/topic

As stated in the comment of @ptone:

In cloud iot core, MQTT topics are many-to-one with Cloud PubSub topics. For security - devices may only publish to MQTT topics prefixed with their device namespace. You can then match sub-topics to other Cloud PubSub topics as noted here, but only up to 10. This is not designed to allow a 1:1 mapping device to PubSub Topic.

edited Jul 14 '21 at 19:23

halfer

19,824
17
99
186

answered Apr 20 '18 at 00:03

DazWilkin

32,823
5
47
88

Hello, Daz. But what happen with the topics we create at Google Core IoT console? I would like to publish and subscribe to my own topics, different from those "standadards" (events, config or state). If just these three are possible, I think the Google IoT is too restricted/limited. Agree? By the way, do you know how to correctly subscribe to one of these topics? I have no success with this task. – Dalton Cézane Apr 20 '18 at 03:12
Yes, you may publish to multiple Pub/Sub topics using subfolders, explained here: https://cloud.google.com/iot/docs/how-tos/mqtt-bridge#publishing_telemetry_events_to_multiple_pubsub_topics – DazWilkin Apr 20 '18 at 03:49
It is not what I want. As I said, I want use my own topics not related to "events". This link you informed just tell how to work with "subfolders", like "events/sub1", "events/sub2", etc. I want to use the topic created at console, named sm1, whose address is /projects/my_project/topics/sm1. If you can also help with the subscribe problem, [here I have another question](https://stackoverflow.com/questions/49638010/google-iot-right-mode-to-receive-notifications-subscribe-working) – Dalton Cézane Apr 20 '18 at 03:55
In Python, you define a callback that's called when your device receives a message on any of its subscriptions. See Google's example. The callback function is defined here: https://github.com/GoogleCloudPlatform/python-docs-samples/blob/19f7f65c7badc37e23ad9f0663da8bd78823a1d7/iot/api-client/mqtt_example/cloudiot_mqtt_example.py#L114. It's added to the client in the same file at line #150 and the subscription (for the default topic /config) is created in the same file at line #159. Hope that helps! – DazWilkin Apr 20 '18 at 03:55
Also, as referenced in my question, I already saw this mqtt example from google (I think almost 10 times) and still it is not what I want. Unfortunately. – Dalton Cézane Apr 20 '18 at 03:59
1

You cannot (currently) use arbitrary topic names. You must follow the /events/... subfolder mechanism. I'll check with the Engineering team tomorrow and see whether there's a feature request filed for your need. Why are you unable to map /events/... ? – DazWilkin Apr 20 '18 at 03:59
In the code above, I don't see an (a) on_message handler defined, (b) the client being configured with it; (c) nor the subscription itself. All 3 need to be provided for a subscription to be effected. – DazWilkin Apr 20 '18 at 04:06
Now I have a good answer: "it is not possible, yet". In this case, I do not know why we create those pub/sub topics in the IoT Core, if they are not accessible by devices through MQTT/HTPP. I think this is good feature to have, since the MQTT protocol works this way. For example, AWS IoT, for example, allow developers create general topics in a way devices can publish and subscribe to those specific ones. – Dalton Cézane Apr 20 '18 at 04:06
It is because I have two scripts, one for subscribe and another for publish. The problem with the subscribe one is here: https://stackoverflow.com/questions/49638010/google-iot-right-mode-to-receive-notifications-subscribe-working (as mentioned some comments before). – Dalton Cézane Apr 20 '18 at 04:09
Could you please complete your answer here, informing about not being possible do what I wanted with the topics, for now? In this case, I will mark it as the answer. – Dalton Cézane Apr 20 '18 at 04:11
1

I'll update it once I've confirm my answer with Engineering. – DazWilkin Apr 20 '18 at 04:19
1

In cloud iot core, MQTT topics are many-to-one with Cloud PubSub topics. For security - devices may only publish to MQTT topics prefixed with their device namespace. You can then match sub-topics to other Cloud PubSub topics as noted here: https://cloud.google.com/iot/docs/how-tos/devices#creating_a_device_registry_with_multiple_pubsub_topics , but only up to 10. This is not designed to allow a 1:1 mapping device to PubSub Topic. – ptone Apr 20 '18 at 15:23
Thank you by your comment, @ptone . It is clearer now (together with the comments of DazWilkin). I think the IoT Core documentation could be improved with some explanation related to this. – Dalton Cézane Apr 23 '18 at 15:07

score 0 · Answer 2 · answered Jun 03 '18 at 04:35

My python code works when I change the mqtt_topic from events to state. In this example, I am trying to upload the current temperature and humidity from my device to Google IoT Core.

  ...
  data =  {'temp': temperature, 'humid': humidity}
  device_id = 'freedgePrototype'
  topic = 'state'

  mqtt_topic = '/devices/{}/{}'.format(device_id, topic)
  payload = json.dumps(data)
  print('Publishing message  {}'.format(payload))

  client.publish(mqtt_topic, payload, qos=1)

And the result would look like this.

Can you see your `events` topic messages aswell ? – MadJlzz Oct 27 '18 at 12:41 — MadJlzz, Oct 27 '18 at 12:41

Google Cloud IoT - Invalid MQTT publish topic

2 Answers2

Linked