How to switch between multiple AWS accounts with Zappa

Question

I am experimenting with how to deploy lambdas into different AWS accounts in continuous delivery environment. At the moment I am stuck with that. Can you please give me a clue about this? As an example with AWS CLI we could define which profile we need to use.

Ex: aws s3 ls --profile account2

In the AWS config file, we define the profile as follows.

[default]

aws_access_key_id = XXXXXXXXXXXXXXXXXX

aws_secret_access_key = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[account2]

aws_access_key_id = XXXXXXXXXXXXXXXXXX

aws_secret_access_key = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Can we use the same approach with zappa deployments?

Highly appreciate any clue to solve this issue.

score 5 · Accepted Answer · answered Jan 09 '18 at 04:05

5

There is an options to nominate the profile name, did you try it?

        "profile_name": "your-profile-name", // AWS profile credentials to use. Default 'default'. Removing this setting will use the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables instead.

https://github.com/Miserlou/Zappa/blob/b12bc67aac00b1302a7f9b18444a51f21deac46a/README.md

answered Jan 09 '18 at 04:05

BMW

42,880
12
99
116

Thank you very much, BMW. I didn't see this before. But after using this, it said the given profile name couldn't be found. The reason was that the ~/.aws/config and ~/.aws/credentials were not accessible by the continuous delivery environment. After giving proper access rights it worked like a charm. More importantly, I had to set two environment variables AWS_CONFIG_FILE and AWS_SHARED_CREDENTIALS_FILE with respective paths for above files. – Hareendra Chamara Philips Jan 09 '18 at 14:12
@HareendraChamaraPhilips huh? You took the suggestions given in my answer, and all those suggestions combined got your CI working, but still accepted BMW's answer in the end? BMW was first, but only with a partial answer :-/ What you are trying to say to the SO community is that they should look at BMW's answer and not mine even though it doesn't solve your original question completely? – mislavcimpersak Jan 09 '18 at 14:38
@mislavcimpersak No hard feelings bro. I saw BMW has edited my question in the morning and then saw his answer a bit later and worked on therefrom. When simply it didn't work out I looked into other solutions. Then I came through this https://github.com/Miserlou/Zappa/issues/909, which helped me to resolve it fully. That's what I have commented. But I have commented in the old browser. I saw your answer when I have refreshed. I am using GOCD instead of travis. Thank you very much for helping bro. Keep the good work up. I really appreciate you. – Hareendra Chamara Philips Jan 09 '18 at 15:49
@mislavcimpersak I think I had to be much clear that my problem is not with CI. My question was what's identical to --profile argument in (aws commands). That was answered sufficiently by BMW. That's why I started commenting, after ticking it as the answer. So everybody, if you are comming here to solve a problem in CI please look at mislavcimpersak answer. – Hareendra Chamara Philips Jan 09 '18 at 15:58

mislavcimpersak · Answer 2 · 2018-01-09T12:15:51.140

You can define which profile to use on your own using Zappa's setting:

"profile_name": "your-profile-name", // AWS profile credentials to use. Default 'default'. Removing this setting will use the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables instead.

But in your CI you first have to create your AWS config file and populate it with your profile from environment variables that are set in your CI's web interface.

In CircleCI (same would be done for TravisCI) I'm doing it like this for my mislavcimpersak profile:

mkdir -p ~/.aws
echo -e "[mislavcimpersak]" >> ~/.aws/credentials
echo -e "aws_access_key_id = "$AWS_ACCESS_KEY_ID >> ~/.aws/credentials
echo -e "aws_secret_access_key = "$AWS_SECRET_ACCESS_KEY >> ~/.aws/credentials

Complete working CircleCI config file is available in my repo:

https://github.com/mislavcimpersak/xkcd-excuse-generator/blob/master/.circleci/config.yml#L58-L60

And also complete working TravisCI config file:

https://github.com/mislavcimpersak/xkcd-excuse-generator/blob/feature/travis-ci/.travis.yml#L25-L29

Also, as it says in Zappa's docs:

Removing this setting will use the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables instead

So you can remove "profile_name": "default" from your zappa_settings.json and set AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in your CI's web interface. Zappa should be able to use those.

How to switch between multiple AWS accounts with Zappa

2 Answers2