Accessing AWS S3 with temporary credentials

Question

Hi I have an application written in Scala (distributed system - Spark) and I need to have read access to my S3 bucket. I have access to this bucket through AWS console using an IAM user and I want to gain a temporary credentials to that bucket using the IAM user. can anyone explain how or if it is possible..?

I've seen this tutorial : http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/prog-services-sts.html but the whole point of using temporary credentials is that I don't want to add the AWS credentials to my code, and it looks like AWSSecurityTokenServiceClient requires it.

Answer 1

If you don't want to use your AWS credentials in your code you can use the InstanceProfileCredentialsProvider. If you are running your code from inside an ec2 instance you can get temporary credentials that have the same permissions as those associated with the IAM role associated with the EC2 instance.

val instanceProfileCredentialsProvider = new com.amazonaws.auth.InstanceProfileCredentialsProvider()
val credentials: AWSCredentials = instanceProfileCredentialsProvider.getCredentials

hadoopConf.set("fs.s3a.awsAccessKeyId", credentials.getAWSAccessKeyId)
hadoopConf.set("fs.s3a.awsSecretAccessKey", credentials.getAWSSecretKey)

or

def s3 = new AmazonS3Client(instanceProfileCredentialsProvider)

Click here for more info.