40

I'm trying to set up Amazon AWS EC2 instance to talk to s3. The basic command is 

aws configure

then follow the prompt to enter

AWS Access Key ID [None]: my-20-digit-id
AWS Secret Access Key [None]: my-40-digit-secret-key
Default region name [None]: us-east-1
Default output format [None]: text

However, what I really want is to have the command 

aws configure

automatically without interaction, i.e., no prompt and wait for input

I know there are files at

~.aws/credentials
~.aws/config

where I put those 4 key=value pairs. And the "credentials" file looks like

[default]
aws_secret_access_key = my-40-digit-secret-key
aws_access_key_id = my-20-digit-id

while the "config" file looks like

[default]
region = us-east-1
output = text

However, with those file at ~/.aws/, I get into ~/.aws/, and at the command line, I type and enter command 

aws configure

I still got the prompt to ask me

AWS Access Key ID [None]:
AWS Secret Access Key [None]:
Default region name [None]:
Default output format [None]:

If I don't enter valid values at prompt, I won't be able to connect to s3, for example via command

aws s3 ls s3://mybucket

I turned help to amazon aws documentation pages. At this page, it mentions this option

"Command line options – region, output format and profile can be specified as command options to override default settings."

as the first option for aws configure

https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html

However, it didn't mention how to use the command line options. I tried something like this

aws configure --region us-east-1

but I still got

AWS Access Key ID [None]:
AWS Secret Access Key [None]:
Default region name [None]:
Default output format [None]:

exactly like I have no option of "--region us-east-1"

If I try to 

aws configure --aws_access_key_id my-20-digit-id --aws_secret_access_key my-40-digit-secret-key --region us-east-1

I get this

usage: aws [options] <command> <subcommand> [parameters]
aws: error: argument subcommand: Invalid choice, valid choices are:

How I can run the command 

aws configure

automatically, no prompt, no interaction.

Please help! TIA

Edit and response to helloV, as the format in main post is much clearer than comment. I tried the command helloV mentioned, but I got error

aws configure set aws_access_key_id my-20-digit-id
usage: aws [options] <command> <subcommand> [parameters]
aws: error: argument subcommand: Invalid choice, valid choices are:

Thanks though. Continue on "aws configure set" On another EC2 instance where I've already set connection to s3, I enter

aws configure set region us-east-1

runs and returns to command prompt ">"

aws configure set aws_access_key_id my-20-digit-id

runs and returns to command prompt ">"

aws configure set aws_secret_access_key my-40-digit-secret-key

runs and returns to command prompt ">"

aws configure

runs but comes with prompts and waits for interaction

AWS Access Key ID [****************ABCD]:
AWS Secret Access Key [****************1234]:
Default region name [us-east-1]:
Default output format [text]:

helloV: here is my screen looks like

ubuntu@ip-11111:~/.aws$ more config
[default]
region = us-east-1
output = text
ubuntu@ip-11111:~/.aws$ more credentials
[default]
aws_secret_access_key = my-40-digit-secret-key
aws_access_key_id = my-20-digit-id
ubuntu@ip-11111:~/.aws$ aws s3 ls s3://

I got this

Unable to locate credentials. You can configure credentials by running "aws configure".

After this, I run

aws configure list
  Name                    Value             Type    Location
  ----                    -----             ----    --------
profile                <not set>             None    None
access_key                <not set>             None    None
secret_key                <not set>             None    None
region                us-east-1      config_file    ~/.aws/config

Looks like it does not check ~/.aws/credentials file, but ~/.aws/config file is in the list.

helloV
  • 50,176
  • 7
  • 137
  • 145
Tony Xu
  • 3,031
  • 4
  • 32
  • 43

6 Answers6

28

These commands worked for me. If this doesn't works for you. Try do the first time using the interaction mode aws configure

aws --profile default configure set aws_access_key_id "my-20-digit-id"

aws --profile default configure set aws_secret_access_key "my-40-digit-secret-key"

palhares
  • 1,663
  • 17
  • 13
  • 2
    I got this to work by omitting the `--profile default` part: `aws configure set region "us-east-1"`. – Travis Jun 25 '21 at 01:37
24

I figured out, finally. Use export such as

export AWS_ACCESS_KEY_ID=my-20-digit-id
export AWS_SECRET_ACCESS_KEY=my-40-digit-secret-key
export AWS_DEFAULT_REGION=us-east-1

then run

aws s3 ls s3://

would work. Don't run "aws configure" as others mentioned.

Thank you all.

Tony Xu
  • 3,031
  • 4
  • 32
  • 43
6

You describe the file very well. Why not just create a file and put it in the right place? I just tried... it's exactly the same as running aws configure

UPDATE: You mention that you want to access S3 from EC2 instance. In this case you shouldn't be using credentials at all. You should user Roles instead

Felix
  • 9,248
  • 10
  • 57
  • 89
  • What do you mean "right place"? I have those 2 files at ~/.aws/credentials and ~/.aws/config with the content of key-value pairs mentioned above. But when I run "aws configure" I still get prompts. – Tony Xu Feb 08 '18 at 23:04
  • 3
    Once you've created the files, don't run "aws configure". The whole point of "aws configure" is to create those files. Just create the files and run your S3 commands. – Peter Dolberg Feb 08 '18 at 23:17
  • with those files, I tried "aws s3 ls s3://mybucket" tells me "Unable to locate credentials. You can configure credentials by running "aws configure"" – Tony Xu Feb 08 '18 at 23:22
  • @Felix, I think I'm using IAM role credentials. However I'm not so sure ---- aws confuses me a lot by all kinds of different concepts. How do I see if I'm using Roles? Thanks – Tony Xu Feb 08 '18 at 23:48
  • IAM role does *not* have credentials. You can see whether you are using role in EC2 screen. select the instance in question and there is a line on the left side *IAM role* (I am writing from memory). If you are using role, and the role has the right policy (that is, it can read from S3 or whatever you need to do), then you don't need to run `aws config` at all! `aws s3 ls` should show you your buckets. That is the recommended way to go! – Felix Feb 08 '18 at 23:52
  • I saw "IAM role -" which I think it's blank or no-value, indicating I'm not using IAM role, I guess. How to use IAM role? – Tony Xu Feb 08 '18 at 23:56
  • If IAM role is definitely necessary, why I can access s3 by setting aws configure (with interaction at prompts though). – Tony Xu Feb 08 '18 at 23:58
  • 1
    C'mon! You need to start with AWS documentation which is vast. First you need to create a role and assign an appropriate policy (there are two preconfigured policies that may work for you - AwsS3 Read and AwsS3 ReadWrite; but if you need more granular policy, you will need to design your own). Once you have the role simple right-click on the instance, choose modify, and assign the role. – Felix Feb 09 '18 at 00:00
  • IAM role is not *necessary*; it is recommended to use on EC2 instances instead of credentials. You can *still* use credentials if you insist! – Felix Feb 09 '18 at 00:01
  • IAM role maybe is the direction, however I don't want to change too much to current settings ---- I knew credentials work, so at this time I'll stick to it and just have to figure out how to do it automatically. Thanks! – Tony Xu Feb 09 '18 at 00:09
  • 2
    @TonyXu note that with an IAM role on your instance, all of this *just works*. – Michael - sqlbot Feb 09 '18 at 00:31
2

Follow this command

$aws configure set aws_access_key_id default_access_key
$ aws configure set aws_secret_access_key default_secret_key
$ aws configure set default.region us-west-2

or

aws configure set aws_access_key_id <key_id> && aws configure set aws_secret_access_key <key> && aws configure set default.region us-east-1

For more details use this link https://awscli.amazonaws.com/v2/documentation/api/latest/reference/configure/set.html

Shihab Uddin Shakil
  • 85
  • 3
1

I use something like this:

aws configure --profile my-profile-name <<-EOF > /dev/null 2>&1
${AWS_ACCESS_KEY_ID}
${AWS_SECRET_ACCESS_KEY}
${AWS_REGION}
text
EOF

also to cleanup after automated process, and not remove `~/.aws/ directory (since some other credentials might be stored there) I run:

aws configure --profile my-profile-name <<-EOF > /dev/null 2>&1
null
null
null
text
EOF
Mateusz
  • 1,149
  • 1
  • 16
  • 33
0

The solution is that you actually don't have to run aws configure! After you run it for the 1st time and established the credentials (~/.aws/credentials) and config (~/.aws/config), going forward you simply have to run the required aws command. I tested this with a cron job and did a "aws s3 ls" command and it worked without having to provide a configure command before it.

Senthil
  • 171
  • 4