Getting AWS role credentials in Gradle

Question

I have an S3 repository that I want to access in my build process. It contains some of my project's dependencies. My project is deployed to an EC2 instance with a designated role - Repo_dependent. The role has an Access_Repo policy attached to it:

{
"Version": "2012-10-17",
"Statement": [
    {
        "Sid": "Stmt1484560548000",
        "Effect": "Allow",
        "Action": [
            "s3:GetObject",
            "s3:ListBucket",
            "s3:GetBucketLocation"
        ],
        "Resource": [
            "arn:aws:s3:::my_bucket",
            "arn:aws:s3:::my_bucket/*"
        ]
    }
  ]
}

When I deploy the new server I get a The AWS Access Key Id you provided does not exist in our records. (Service: Amazon S3; Status Code: 403; Error Code: InvalidAccessKeyId; Request ID: 02169BFDCF7AFE10) exception.

My build script is this (abbreviated for simplicity)

buildscript {
  repositories {
    jcenter()
  }
  dependencies {
    classpath 'com.amazonaws:aws-java-sdk:1.11.83'
  }
}

import com.amazonaws.auth.*

repositories {
  jcenter()
  maven {
    url "s3://my_bucket.s3.amazonaws.com"
    credentials(AwsCredentials) {
        def providercreds = new InstanceProfileCredentialsProvider().getCredentials()
        accessKey providercreds.getAWSAccessKeyId()
        secretKey providercreds.getAWSSecretKey()
    }
  }
}

My assumption is that I'm missing something in either how EC2 instances access their roles or something in how roles are defined. When trying to run the same script locally, with a user that has the Access_Repo policy attached to it and instead of using InstanceProfileCredentialsProvider use DefaultAWSCredentialsProviderChain, the build runs fine. However using DefaultAWSCredentialsProviderChain and deploying the instance again resulted in the same exception.

Any help would be very much appreciated.

Update:

Testing done using the AWS CLI and using the STSAssumeRoleSessionCredentialsProvider shows the the build script is using the right role with the DefaultAWSCredentialsProviderChain provider. Adding AmazonS3FullAccess policy to the role did not change the result

I'm using Jenkins to deploy the code so my next lead is that maybe something is wrong there

Update2:

I tried capturing the network traffic to see which credentials are sent to AWS and it seems like different credentials are sent by Gradle and by the AWS CLI so I'm back to my original assumption that Gradle doesn't pull the right role

By looking into URL, I think you was created bucket in Northern Virginia region. You can comment the credentials definition in build script part. From my point, it searches access key and secret key within EC2 Instance(local drive) rather than IAM role. — Kumaresh Babu N S, Jan 23 '17 at 14:40
Not sure I understand.. If I remove the credentials part everything would work? — Guy Grin, Jan 23 '17 at 14:45
It should work. Code shows it searches within local for aws credentials.. — Kumaresh Babu N S, Jan 23 '17 at 15:00
But typically we should not have AWS credentials in the project. For AWS Credentials we have to use IAM role with specific permission. — Kumaresh Babu N S, Jan 24 '17 at 03:02
They are not in the project. I'm trying to load the credentials of the role. The gradle code requires some credentials — Guy Grin, Jan 24 '17 at 07:53
@GuyGrin thanks, I had come up with the same. But turns out more is needed in a codebuild scenario. — Kevin Bayes, Jul 06 '17 at 07:55

score 11 · Accepted Answer · answered Jul 04 '17 at 09:56

11

If any one encounters this issue. I fixed it after migrating from Gradle 3.0 to 3.3. This allowed me to use the following code:

buildscript {
  repositories {
    jcenter()
  }
  dependencies {
  }
}


repositories {
  jcenter()
  maven {
    url "s3://my_bucket.s3.amazonaws.com"
    authentication {
        awsIm(AwsImAuthentication)
    }
  }
}

task wrapper(type: Wrapper) {
  gradleVersion = '3.3'
}

dependencies {
    compile 'com.amazonaws:aws-java-sdk-iam:1.11.78'
    compile 'com.amazonaws:aws-java-sdk-ec2:1.11.78'
}

answered Jul 04 '17 at 09:56

Guy Grin

1,968
2
17
38

Nice find. Here's an example of the [`AwsImAuthentication` option in the gradle docs](https://docs.gradle.org/current/userguide/dependency_management.html#mavenIvyS3RepositoriesAuthWithIam). – endeavor Dec 24 '17 at 12:32
you should mark your answer, I've been struggling to find this. thanks btw – Gujarat Santana Feb 12 '18 at 10:49
Nice Solution. Thanks. Its fixed my issue – Aman Goyal Jun 05 '19 at 06:42

score 4 · Answer 2 · answered Jan 14 '19 at 10:35

When you are using aws profiles, you'll have to set the AWS_PROFILE environment variable as described in the docs to make this working, cause gradle (or aws-sdk) is either using the environment variable AWS_ACCESS_KEY_ID and AWS_SECREST_ACCESS_KEY or the default profile located at ~/.aws/ if set

export AWS_PROFILE=project-x
./gradlew build

or inline

AWS_PROFILE=project-x ./gradlew build

or as an alias in ~/.bashrc or similar, where you can also add some logic based on a config file

alias gw='AWS_PROFILE=project-x ./gradlew'

Getting AWS role credentials in Gradle

Update:

Update2:

2 Answers2