4

The documentation for Event Grid states that it has a delivery and retry mechanism built in, and gives an example of what would classify as a successful or failed attempt. The documentation is very clear about what happens with a single event handler.

My question is, what happens if there are multiple event handlers, and only one handler fails to receive the event? Is the event retried only for that handler, or will all handlers see the retry?

Andrew Williamson
  • 8,299
  • 3
  • 34
  • 62
  • I can only assume that if the event cannot be delivered to **any** of the subscribers, it would be dead-lettered. Saying that, it's better to raise this issue with the documentation. Which I did [here](https://github.com/MicrosoftDocs/azure-docs/issues/23047). – Sean Feldman Jan 22 '19 at 22:28
  • Thanks Sean. My question was actually about the re-delivery, and whether a message is re-delivered to *all* subscribers or only the one that failed, but the dead-lettering is also a good question. I'll add this to the GitHub issue. In the meantime, I'm not sure whether to close this question or leave it open – Andrew Williamson Jan 23 '19 at 01:15
  • Leave as open. Either EG team will answer or yourself. Either way people will find this with an answer. – Sean Feldman Jan 23 '19 at 01:22
  • The AEG eventing model using a Fan-out pattern for delivering an event message to the subscribed handlers. This logical connectivity for each Fan-output is described by subscription and they are a full transparently to each other and loosely decouple. The behavior of the each event delivery in this Fan-out pattern is based on their subscriptions, such as retrying, deadlettering, filtering, etc. Note, that based on the subscriber endpoint type, we can easy change a default delivery mode such as PushWithAck to the Pull mode (async), for instance: the storage queue and HybridConnection – Roman Kiss Jan 23 '19 at 05:17
  • 1
    for the test purposes (such as your question) you can use a tester tool https://www.codeproject.com/Articles/1254463/Azure-Event-Grid-Tester where can be simulated your case, for instance: a custom topic + storage queue subscriber + HybridConnection subscriber with an error (503) delivery simulation. – Roman Kiss Jan 23 '19 at 05:26

3 Answers3

8

Basically, the Azure Event Grid eventing Pub/Sub model can handle two messaging/mediation patterns such as Fan-In pattern and Fan-Out (broadcasting) pattern. The following screen snippets show their differences:

enter image description here

enter image description here

The logical connectivity between the Event Source and Event Sink is described by Subscription which it is basically a metadata artifact of the Pub/Sub model. Each logical connectivity (represented by Subscription) is independent and loosely decouple to others. In other words, each subscriber can handle in this Pub/Sub model only one logical connectivity such as only one event source.

Your question is related to the Fan-Out (broadcasting) pattern, where the event interest is broadcasting to the multiple subscribers using a PushWithAck delivery mode. Each subscription within this Fan-Out pattern has own "a message state delivery machine" declared by subscriber such as retrying option, deadlettering, filtering, etc.

In other words, the event delivery to the subscribers is processing in parallel way based on their subscription in the transparent manner without any dependences each other. Note, that the subscriber doesn't have any information about who, where, how, etc. are event delivering to other once, so each subscriber can see only own delivery state, for instance, the value of the Aeg-Delivery-Count shows a retry counter of the state machine.

So, in the case of the failed event delivery to the one of the multiple subscribers, the enabled retrying process is performing only for that subscriber.

Roman Kiss
  • 7,925
  • 1
  • 8
  • 21
  • Thanks Roman. I didn't realize the dead-lettering is an option per-subscriber, I thought it was per-topic, so that explains how this can all work as expected – Andrew Williamson Jan 23 '19 at 19:53
4

As Roman explained, each endpoint is handled independently. If one event handler fails, it will be retried without affecting the other event handlers, and of course, if that particular endpoint continues to fail, it will eventually be deadlettered (assuming deadlettering has been configured on the event subscription), or dropped.

Bahram Banisadr
  • 312
  • 1
  • 7
2

When coming to event publishing in event grids, the events from custom event grid topics, or system event grid topics(say Service Bus Namespaces) are forwarded to the event grid subscriptions configured with them. The events are then sent to the endpoints configured with the event grid subscription.

Whenever the event delivery to an endpoint fails, it is retried based on the retry policy configured.If the number of retries exceed the retry policy configured, the events are stored in the storage account blob if configured as the dead-letter destination, else the events will be lost.

By default, Event Grid expires all events that aren't delivered within 24 hours. You can customize the retry policy when creating an event subscription. You provide the maximum number of delivery attempts (default is 30) and the event time-to-live (default is 1440 minutes).

When there are multiple subscribers(event grid subscriptions) to a same event grid topic, retry occurs only with the event grid subscription whose event delivery has failed.

refer Event Grid message delivery and retry for more info on retry policy.

Ranjith Eswaran
  • 333
  • 2
  • 12