4

I'm triggering a cloud function every minute with cloud scheduler [* * * * *].

The Stackdriver logs indicate the function appears to have been triggered and run twice in the same minute. Is this possible?

PubSub promises at least once delivery but I assumed that GCP would automatically handle duplicate triggers for scheduler -> function workflows.

What is a good pattern for preventing this function from running more than once per minute?

ProGirlXOXO
  • 2,170
  • 6
  • 25
  • 47

1 Answers1

8

Your function needs to be made "idempotent" in order to ensure that a message gets processed only once. In other words, you'll have to maintain state somewhere (maybe a database) that a message was processed successfully, and check that state to make sure a message doesn't get processed twice.

All non-HTTP type Cloud Functions provide a unique event ID in the context parameter provided to the function invocation. If you see a repeat event ID, that means your function is being invoked again for the same message, for whatever reason.

This need for idempotence is not unique to pubsub or cloud scheduler. It's a concern for all non-HTTP type background functions.

A full discussion on writing idempotent functions is a bit too much a Stack Overflow answer, but there is a post in the Google Cloud blog that covers the issue pretty well.

See also: Cloud functions and Firebase Firestore with Idempotency

Doug Stevenson
  • 297,357
  • 32
  • 422
  • 441
  • Thanks Doug! Are you saying that if I use cloud scheduler to trigger a function with HTTP then I *can* guarantee exactly once execution? – ProGirlXOXO Nov 21 '19 at 23:13
  • 2
    No, any background function might be invoked more than once per event (though it should be rare). That's an unfortunate reality for systems decoupled through an asynchronous interface, while allowing that errors can occur on either side of that interface that could cause the loss of data. In computer science, it's call the Two Generals Problem. https://en.wikipedia.org/wiki/Two_Generals%27_Problem – Doug Stevenson Nov 21 '19 at 23:16