1

I have a device publishing through a gateway on the events topic (/devices/<dev_id>/events/motion) to PubSub. It's landing in PubSub correctly but subFolder is just an empty string.

On the gateway I'm publishing using the code below. f"mb.{device_id}" is the device_id (not the gateway ID and attribute could be anything - motion, temperature, etc

def report(self, device_id, attribute, value):
    topic = f"/devices/mb.{device_id}/events/{attribute}"
    timestamp = datetime.utcnow().timestamp()
    client.publish(topic, json.dumps({"v": value, "ts": timestamp}))

And this is the cloud function listening on the PubSub queue.

def iot_to_bigtable(event, context):
    payload = json.loads(base64.b64decode(event["data"]).decode("utf-8"))
    timestamp = payload.get("ts")
    value = payload.get("v")
    if not timestamp or value is None:
        raise BadDataException()
    attributes = event.get("attributes", {})
    device_id = attributes.get("deviceId")
    registry_id = attributes.get("deviceRegistryId")
    attribute = attributes.get("subFolder")
    if not device_id or not registry_id or not attribute:
        raise BadDataException()

A sample of the event in Pub/Sub:

{
    @type: 'type.googleapis.com/google.pubsub.v1.PubsubMessage', 
    attributes: {
        deviceId: 'mb.26727bab-0f37-4453-82a4-75d93cb3f374', 
        deviceNumId: '2859313639674234', 
        deviceRegistryId: 'mb-staging', 
        deviceRegistryLocation: 'europe-west1', 
        gatewayId: 'mb.42e29cd5-08ad-40cf-9c1e-a1974144d39a', 
        projectId: 'mb-staging', 
        subFolder: ''
    }, 
    data: 'eyJ2IjogImxvdyIsICJ0cyI6IDE1OTA3NjgzNjcuMTMyNDQ4fQ=='
}

Why is subFolder empty? Based on the docs I was expecting it to be the attribute (i.e. motion or temperature)

Matthew Brown
  • 1,028
  • 3
  • 11
  • 24
  • 1
    Okay, Occam's Razor... print out {attribute} and ensure that you're sending a subfolder and it's not just landing in the general Pub/Sub topic? Also, is the subfolder configured for that event in IoT Core itself? Just sending something to a subfolder doesn't do anything unless it's configured in IoT Core to shunt that subfolder to a separate topic than the base folder topic. – Gabe Weiss May 30 '20 at 06:49
  • Thanks for replying so quickly. I added a print statement into the `report` function and it prints out ``` sending {"v": 1, "ts": 1590818418.307897} to /devices/mb.26727bab-0f37-4453-82a4-75d93cb3f374/events/motion sending {"v": 26, "ts": 1590818418.30808} to /devices/mb.26727bab-0f37-4453-82a4-75d93cb3f374/events/temperature sending {"v": "high", "ts": 1590818418.308173} to /devices/mb.26727bab-0f37-4453-82a4-75d93cb3f374/events/level ``` so definitely sending it to a subfolder - I think? – Matthew Brown May 30 '20 at 07:03
  • @GabeWeiss And re your second question - I found this in the docs: "Messages published to a subfolder are forwarded to the Cloud Pub/Sub topic with the same name. The corresponding registry must be configured with the Cloud Pub/Sub topic; otherwise, messages are forwarded to the default Cloud Pub/Sub topic." I took that the mean that any subfolder I publish should land in the default pub/sub queue if a topic with the same name hasn't been configured? – Matthew Brown May 30 '20 at 07:04
  • Right, but I THINK the way it works, is the subfolder doesn't get set in the pub/sub metadata unless there's a topic configured for that subfolder. Just as a test, if you can, try creating another topic for the registry, and configure the subfolder to point to that topic, and see if the subfolder gets set on that pub/sub message. – Gabe Weiss May 30 '20 at 07:27

1 Answers1

1

This issue has nothing to do with Cloud IoT Core. It is instead caused by how Pub/Sub handles failed messages. It was retrying messages from ~12 hours ago that had failed (and didn't have an attribute).

You fix this by purging the Subscription in Pub/Sub.

Matthew Brown
  • 1,028
  • 3
  • 11
  • 24