8

Due to AWS deprecating Signature Version 3 in Oct 2020 for SES, I want to handle this issue with AWS boto (Python). But I didn't see any doc related to boto supporting signature version 4 for SES. Is anyone having similar issue and have solutions?

Jian Cui
  • 93
  • 1
  • 3
  • Do you mean boto, or boto3? – jarmod Jul 06 '20 at 14:23
  • @jarmod, currently i am using boto, but if boto3 support ses signature version 4, that would be ok too. If possible, can you show me how you use ses signature version 4 in boto or boto3? Really appreciate it – Jian Cui Jul 06 '20 at 14:34
  • All current AWS SDKs support signature v4. It's actually the default signing method for all situations other than creation of S3 pre-signed URLs (for which you have to explicitly configure v4) as far as I know. I can't quickly confirm this for SES but hopefully you can. It should be quick to create a small SES client with boto3 for a test. – jarmod Jul 06 '20 at 14:45
  • @jarmod, Thank you so much. I will do a quick test on that. Appreciate your help – Jian Cui Jul 06 '20 at 15:01
  • @jarmod, Hi so boto is not supporting signature version 4 for SES right? – Jian Cui Jul 06 '20 at 16:01
  • I don't know for sure, but I think not. The boto library is essentially deprecated and any new application should use boto3. From a quick glance at the (older) boto docs, it does support signature v4 for S3 access, [EC2](https://github.com/boto/boto/commit/4d780bd), and for S3 pre-signed URLs so it *might* also support it for SES, but I see no evidence that it does. – jarmod Jul 06 '20 at 16:16
  • @jarmod thank you for your detailed explanation. – Jian Cui Jul 06 '20 at 17:41
  • 4
    I found where SignatureV4 support was [added](https://github.com/boto/boto/commit/c9342baad5ed2da7be76e8498dcae54a5906e486) to boto (in 2012). However, cross-referencing [this](https://github.com/boto/boto/blob/91ba037e54ef521c379263b0ac769c66182527d7/boto/ses/connection.py#L60) and [this](https://github.com/boto/boto/blob/91ba037e54ef521c379263b0ac769c66182527d7/boto/auth.py#L236), it appears that the boto.ses code still uses HmacAuthV3Handler. From this, I conclude that users of the older 'boto.ses' library will not be able to continue utilizing AWS SES past 2020-10-01. – j0nam1el Jul 19 '20 at 10:55
  • 1
    AWS just started to increase their nag emails on the Signature Version 3 now to once a week, so I'm pretty sure this question will get more visitors now ;) There are a lot of legacy projects on AWS that are on boto (not boto3). I have one myself and must make some sort of mashup hack solution now if I can't upgrade boto. Perhaps boto3 can be inserted in parallell for SES only, in projects that are too large to update everything to boto3.. – BjornW Mar 16 '21 at 10:15

2 Answers2

7

Promoting j0nam1el's comment to an answer, as it directly answers the question:

I found where SignatureV4 support was added to boto (in 2012). However, cross-referencing this and this, it appears that the boto.ses code still uses HmacAuthV3Handler. From this, I conclude that users of the older 'boto.ses' library will not be able to continue utilizing AWS SES past 2020-10-01.

As noted in this github issue, the deadline has been extended:

We are extending support for Signature Version 3 until February 28, 2021 (the previously announced deprecation date was October 1, 2020). Beginning March 1 2021, support for Signature Version 3 will be turned off in Amazon SES, and only Signature Version 4 will be supported going forward. Amazon SES customers who are currently using Signature Version 3 must migrate to Signature Version 4 by February 28, 2021. After that, requests using Signature Version 3 will be progressively throttled in Amazon SES.

I'm not sure exactly what "progressively throttled" means - would old-style requests fail randomly, or just take longer? Please edit this answer if you know.

legoscia
  • 39,593
  • 22
  • 116
  • 167
  • 1
    For those playing along at home: this means that requests are randomly declined with an error, with the declination rate increasing as time goes on. – Kye Aug 12 '21 at 08:26
3

My recommendation is that you migrate from boto, which is essentially deprecated, to boto3 because boto3 supports signature v4 by default (with the exception of S3 pre-signed URLs which has to be explicitly configured).

jarmod
  • 71,565
  • 16
  • 115
  • 122
  • Hi jarmod, Thanks. I am actively switching to boto3 for SES. And I would like to build unit tests to make sure signature version 4 works for boto3 SES. Do you have any idea how to work on those unit test or do you have any instructions? Really appreciate it – Jian Cui Jul 14 '20 at 13:14
  • You can see the general signing process and examples [here](https://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html#sigv4_signing-steps-overview). I'm not sure exactly want you want to do here but unit testing a boto3 client is probably going to require that you mock parts of boto3 or botocore. Maybe take a look at [moto](https://github.com/spulec/moto) or read [this blog post](https://adamj.eu/tech/2019/04/22/testing-boto3-with-pytest-fixtures/). – jarmod Jul 14 '20 at 14:12
  • Thank you so much. I will definitely take a look – Jian Cui Jul 14 '20 at 14:21
  • 2
    This doesn't answer the question (does boto support signature version 4). I tried to find this info and got this question, so I hoped to find the answer. There are legacy codebases on boto and for example right now AWS SES SMTP is starting to spam emails that sig version 3 is going away, but they don't explain which AWS SDKs use sig version 3 :/ – BjornW Aug 17 '20 at 14:42
  • 1
    @BjornW earlier comments attempt to address "does boto support sigv4?" (answer: yes, in certain scenarios such as S3 but not, as far as we know, SES). – jarmod Aug 17 '20 at 15:22
  • Ran into this, for details see https://boto3.amazonaws.com/v1/documentation/api/1.9.42/guide/s3.html#generating-presigned-urls where they show how to explicitly configure the v4 sig for pre-signed urls. – synthesizerpatel Jun 29 '21 at 17:07
  • What is the normal period that aws annouces next sig version and then deprecates previosu ? How frequently these keep changing ? – Simplecode Jul 12 '21 at 09:42
  • @Simplecode there is no 'normal period' and we don't know how frequently new signature versions will come along. For reference sigv2 started in 2006 and sigv4 started in 2012 afaik. I'm not aware there was ever a sigv1 or sigv3. So, two versions in 15 years. – jarmod Jul 12 '21 at 13:39