Can we configure retry attempts on AWS Go SDK service calls

Question

AWS by default provides retries support on its service calls, which is usually set to a max of 3 attempts.

Can we configure the retry object to set retry attempts to 5?

Please do not post multiple questions about the same issue. You already have a question related to this here: [How can we avoid frequent 5XX errors and i/o timeout errors with AWS SQS service calls](https://stackoverflow.com/questions/69343275/how-can-we-avoid-frequent-5xx-errors-and-i-o-timeout-errors-with-aws-sqs-service) — arco444, Sep 27 '21 at 08:29
I faced the problems at the same time but they were two independent questions that needed different solutions, hence posted it separately. However, will keep this in mind next time — Riya John, Sep 27 '21 at 17:32

score 1 · Answer 1 · answered Sep 27 '21 at 08:31

you can define a custom retry strategy for the SDK to use:

func main() {
    sess := session.Must(
        session.NewSession(&aws.Config{
            // Use a custom retryer to provide custom retry rules.
            Retryer: CustomRetryer{
                DefaultRetryer: client.DefaultRetryer{
                    NumMaxRetries: client.DefaultRetryerMaxNumRetries,
                }},

            // Use the SDK's SharedCredentialsProvider directly instead of the
            // SDK's default credential chain. This ensures that the
            // application can call Config.Credentials.Expire. This  is counter
            // to the SDK's default credentials chain, which  will never reread
            // the shared credentials file.
            Credentials: credentials.NewCredentials(&credentials.SharedCredentialsProvider{
                Filename: defaults.SharedCredentialsFilename(),
                Profile:  "default",
            }),
            Region: aws.String(endpoints.UsWest2RegionID),
        }),
    )
    // Add a request handler to the AfterRetry handler stack that is used by the
    // SDK to be executed after the SDK has determined if it will retry.
    // This handler forces the SDK's Credentials to be expired, and next call to
    // Credentials.Get will attempt to refresh the credentials.
    sess.Handlers.AfterRetry.PushBack(func(req *request.Request) {
        if aerr, ok := req.Error.(awserr.RequestFailure); ok && aerr != nil {
            if aerr.Code() == "InvalidClaimException" {
                // Force the credentials to expire based on error code.  Next
                // call to Credentials.Get will attempt to refresh credentials.
                req.Config.Credentials.Expire()
            }
        }
    })

See the sample code here

score 1 · Accepted Answer · answered Sep 27 '21 at 08:38

Yes, AWS provides support to configure their retry and timeouts features. Here are two ways to increase the max number of retries to 5 in AWS Golang SDK v2:

Configure the retry logic on the AWS Config object cfg and it can be used with various AWS service clients using NewFromConfig function

cfg, err := config.LoadDefaultConfig(context.TODO(), config.WithRetryer(func() aws.Retryer {
    return retry.AddWithMaxAttempts(retry.NewStandard(), 5)
}))

client := s3.NewFromConfig(cfg)

Configure the retry logic only for a specific AWS service client

customRetry := retry.NewStandard(func(o *retry.StandardOptions) {
        o.MaxAttempts = 5
    })

sqsClient := sqs.NewFromConfig(creds,
    func(o *sqs.Options) {
        o.Retryer = customRetry
    },
)

More info can be found at https://aws.github.io/aws-sdk-go-v2/docs/configuring-sdk/retries-timeouts/ and https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/aws/retry#hdr-Standard

Can we configure retry attempts on AWS Go SDK service calls

2 Answers2