24

I am trying to get travis-ci to run a custom deploy script that uses awscli to push a deployment up to my staging server.

In my .travis.yml file I have this:

before_deploy:
  - 'curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"'
  - 'unzip awscli-bundle.zip'
  - './awscli-bundle/install -b ~/bin/aws'
  - 'export PATH=~/bin:$PATH'
  - 'aws configure'

And I have set up the following environment variables:

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION

with their correct values in the travis-ci web interface.

However when the aws configure runs, it stops and waits for user input. How can I tell it to use the environment variables I have defined?

Dave Sag
  • 13,266
  • 14
  • 86
  • 134
  • 4
    Found this question while trying to install the awscli on Travis -- ultimately what I ended up with was to use `pip install --user awscli` instead of the above before_deploy stanzas. – ankon Aug 04 '16 at 07:53
  • If you run `aws configure set help` you will see that you can supply settings individually on the command line and they will be written to the relevant credentials or config file. – franchb Mar 08 '19 at 09:54

3 Answers3

30

Darbio's solution works fine but it's not taking into consideration that you may end up pushing your AWS credentials in your repository.

That is a bad thing especially if docker is trying to pull a private image from one of your ECR repositories. It would mean that you probably had to store your AWS production credentials in the .travis.yml file and that is far from ideal.

Fortunately Travis gives you the possibility to encrypt environment variables, notification settings, and deploy api keys.

gem install travis

Do a travis login first of all, it will ask you for your github credentials. Once you're logged in get in your project root folder (where your .travis.yml file is) and encrypt your access key id and secret access key.

travis encrypt AWS_ACCESS_KEY_ID="HERE_PUT_YOUR_ACCESS_KEY_ID" --add
travis encrypt AWS_SECRET_ACCESS_KEY="HERE_PUT_YOUR_SECRET_ACCESS_KEY" --add

Thanks to the --add option you'll end up with two new (encrypted) environment variables in your configuration file. Now just open your .travis.yml file and you should see something like this:

env:
    global:
        - secure: encrypted_stuff
        - secure: encrypted_stuff

Now you can make travis run a shell script that creates the ~/.aws/credentials file for you.

ecr_credentials.sh

#!/usr/bin/env bash

mkdir -p ~/.aws

cat > ~/.aws/credentials << EOL
[default]
aws_access_key_id = ${AWS_ACCESS_KEY_ID}
aws_secret_access_key = ${AWS_SECRET_ACCESS_KEY}
EOL

Then you just need to run the ecr_credentials.sh script from your .travis.yml file:

before_install:
    - ./ecr_credentials.sh

Done! :-D

Source: Encription keys on Travis CI

j0k
  • 22,600
  • 28
  • 79
  • 90
Francesco Casula
  • 26,184
  • 15
  • 132
  • 131
  • 2
    This is outstanding and was a huge help to me, thank you. I should note, though, that creating the `~/.aws/credentials` file isn't actually necessary. Just having the environment variables `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are enough to get the `aws` CLI to work. – Nick Williams Jan 27 '19 at 18:15
  • 1
    If your repo is public, you are still giving everyone your keys, as any one could take values for your .travis.yml file and use them to deploy stuff to you AWS account. Encrypted or not, I would create the variables in the travis UI, and then still use the script above to create the credentials file. – Mark Jones Mar 22 '19 at 12:13
  • 1
    Quoting the Travis docs: `These encrypted values can be added by anyone, but are only readable by Travis CI. The repository owner does not keep any secret key material. Please note that encrypted environment variables are not available for pull requests from forks.`. If you're doing this on a public repo just pay attention to what PRs you're merging and don't give to anyone permissions to spawn debug builds and you should be fine (as long as you don't `echo` the variable content in your build). See first paragraph https://docs.travis-ci.com/user/encryption-keys/ – Francesco Casula Mar 22 '19 at 13:10
  • Awesome! Thanks, Francesco - this was exactly what I needed. I was aware of the `travis encrypt` command to create the encrypted secrets, but couldn't get them to be read as part of the build. You're a life saver! – mdmjsh Jun 19 '20 at 17:07
14

You can set these in a couple of ways.

Firstly, by creating a file at ~/.aws/config (or ~/.aws/credentials).

For example:

[default]
aws_access_key_id=foo
aws_secret_access_key=bar
region=us-west-2

Secondly, you can add environment variables for each of your settings.

For example, create the following environment variables:

AWS_DEFAULT_REGION
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY

Thirdly, you can pass region in as a command line argument. For example:

aws eb deploy --region us-west-2

You won't need to run aws configure in these cases as the cli is configured.

There is further AWS documentation on this page.

Darbio
  • 11,286
  • 12
  • 60
  • 100
  • Not sure I have the ability to create a `~/.aws/credentials` file on Travis. The alternative is to just leave the environment vars set and drop the `aws configure` line. – Dave Sag May 17 '16 at 05:52
0

Following the advice from @Darbio, I came up with this solution:

- stage: deploy
  name: "Deploy to AWS EKS"
  language: minimal
  before_install:
    # Install kubectl
    - curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl
    - chmod +x ./kubectl
    - sudo mv ./kubectl /usr/local/bin/kubectl
    # Install AWS CLI
    - if ! [ -x "$(command -v aws)" ]; then curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" ; unzip awscliv2.zip ; sudo ./aws/install ; fi
    # export environment variables for AWS CLI (using Travis environment variables)
    - export AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
    - export AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
    - export AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION}
    # Setup kubectl config to use the desired AWS EKS cluster
    - aws eks update-kubeconfig --region ${AWS_DEFAULT_REGION} --name ${AWS_EKS_CLUSTER_NAME}

  deploy:
    - provider: script
      # bash script containing the kubectl commands to setup the cluster
      script: bash k8s-config/deployment.sh
      on:
        branch: master

It is also possible to avoid installing AWS CLI altogether. Then you need to configure kubectl:

kubectl config set-cluster --server= --certificate-authority=
kubectl config set-credentials --client-certificate= --client-key=
kubectl config set-context myContext --cluster= --namespace= --user=
kubectl config use-context myContext

You can find most of the needed values in your users home directory in /.kube/config, after you performed the aws eks update-kubeconfig command on your local machine. Except for the client certificate and key. I couldn't figure out where to get them from and therefore needed to install AWS CLI in the pipeline as well.

user11964644
  • 64
  • 1
  • 4