Fine-grained control over what kind of failures trip a dapr circuit breaker

Question

Question

Is it possible to further specifiy what kind of failures trip a dapr circuit breaker targeting a component (AWS SQS)?

Use case

I am sending emails via AWS SES. If you reach your sending limits, the AWS sdk throws an error (code 454). In this case, I want a circuit breaker to stop the processing of the queue and retry sending the emails later.

However, when you have another error, e.g. invalid email address, I don't want this to trip the circuit breaker as it is not transient. I would like to send the message to the DLQ though, as to manually examine these messages later (-> that's why I am still throwing here and not failing siltently).

Simplified setup

I have a circuit breaker defined that trips when my snssqs-pubsub component has more than 3 consecutive failures:

circuitBreakers:
  pubsubCB:
    # Available variables: requests, totalSuccesses, totalFailures, consecutiveSuccesses, consecutiveFailures
    trip: consecutiveFailures > 3

# ...

targets:
  components:
    snssqs-pubsub-emails:
      inbound:
        circuitBreaker: pubsubCB

In my application, I want to retry sending emails that failed because the AWS SES sending limit was hit:

try {
  await this.sendMail(options);
} catch (error) {
  if (error.responseCode === '454') {
    // This error should trip the cicuit breaker
    throw new Error({ status: 429, 'Rate limited. Should be retried.' })
  } else {
    // This error should not trip the circuit breaker.
    // Because status is 404, dapr directly puts the message into DLQ and skips retry
    throw new NotFoundError({ status: 404 })
  }
}

Answer 1

You may not have a problem to worry about if you have business case that does not violate AWS Terms of Service. You can put a support ticket and get SES Service Limit's raised.

It does not appear that dapr retry policies don't support the customization you need but .NET does.

If you don't want to process the message, then don't delete it. You can then set visibility timeout of the message in the SQS so they stay hidden to avoid processing again too quickly. Any exception thrown regardless will end up in the DLQ.