13

I'm making an application in Python and using Amazon Web Services in some modules.

I'm now hard coding my AWS access id and secret key in *.py file. Or might move them out to an configuration file in future.

But there's a problem, how can I protect AWS information form other people? As I know python is a language that easy to de-compile.

Is there a way to do this?

Well what I'm making is an app to help user upload/download stuff from cloud. I'm using Amazon S3 as cloud storage. As I know Dropbox also using S3 so I'm wondering how they protects the key.

After a day's research I found something. I'm now using boto (an AWS library for python). I can use a function of 'generate_url(X)' to get a url for the app to accessing the object in S3. The url will be expired in X seconds. So I can build a web service for my apps to provide them the urls. The AWS keys will not be set into the app but into the web service.

It sounds great, but so far I only can download objects with this function, upload doesn't work. Any body knows how to use it for uploading?

Does anyone here know how to use key.generate_url() of boto to get a temporary url for uploading stuff to S3?

michael.luk
  • 551
  • 2
  • 6
  • 11
  • What kind of application? Will it be running on your server or distributed to clients? – pcalcao Mar 29 '12 at 13:59
  • 1
    There's no foolproof way to protect a secret key in distributed code in *any* language. And I'm going to have to remember that term "mudules", I think I've worked on a few of those. – Mark Ransom Mar 29 '12 at 14:09
  • To pcalcao: It's an app that help user to upload, download stuff from cloud. So I use Amazon S3 as the cloud storage. – michael.luk Mar 29 '12 at 15:54
  • @MarkRansom: I just made a typo mistake, it should be "modules" :) – michael.luk Mar 29 '12 at 15:56
  • I knew it was a typo, or perhaps a [Freudian slip](http://en.wikipedia.org/wiki/Freudian_slip). Every once in a while I see a new word created accidentally, I love it when that happens. – Mark Ransom Mar 29 '12 at 16:05
  • Well I think someone can make an app on this kind of fun. Make a new word with the app and then it will be publish in facebook or twitter :) – michael.luk Mar 29 '12 at 16:10

4 Answers4

9

There's no way to protect your keys if you're going to distribute your code. They're going to be accessible to anyone who has access to your server or source code.

There are two things you can do to protect yourself against malicious use of your keys.

  1. Use the amazon IAM service to create a set of keys that only has permission to perform the tasks that you require for your script. http://aws.amazon.com/iam/

  2. If you have a mobile app or some other app that will require user accounts you can create a service to create temporary tokens for each user. The user must have a valid token and your keys to perform any actions. If you want to stop a user from using your keys you can stop generating new tokens for them. http://awsdocs.s3.amazonaws.com/STS/latest/sts-api.pdf

Specifically to S3 if you're creating an application to allow people to upload content. The only way to protect your account and the information of the other users is to make them register an account with you.

  1. The first step of the application would be to authenticate with your server.
  2. Once your server authenticates you make a request to amazons token server and return a token
  3. Your application then makes a request using the keys built into the exe and the token.
  4. Based on the permissions applied to this user he can upload only to the bucket that is assigned to him.

If this seems pretty difficult then you're probably not ready to design an application that will help users upload data to S3. You're going to have significant security problems if you only distribute 1 key even if you can hide that key from the user they would be able to edit any data added by any user.

The only way around this is to have each user create their own AWS account and your application will help them upload files to their S3 account. If this is the case then you don't need to worry about protecting the keys because the user will be responsible for adding their own keys after installing your application.

bwight
  • 3,300
  • 17
  • 21
  • What I'm now doing is writing python scripts (put the key in the .py file directly) and pack it up with pyinstaller to a single .exe file which can running in Windows. – michael.luk Mar 29 '12 at 16:04
  • 1
    Since your strings are in plain text anyone can steal them even if you package them in an exe file. Not to mention if you're not using SSL with amazon then they could also see any of the network traffic when talking to amazon. Don't assume their private just because they're in the exe. In your case you can probably just limit the permissions of the keys. If your service requires a subscription then you can authenticate them first and return a temporary token. The token is required for the keys to work so if they dont have a valid subscription they cant use your keys even if they know them. – bwight Mar 29 '12 at 16:15
  • 1
    I added some information to my answer specific to your use case. – bwight Mar 29 '12 at 16:22
  • Does the way means that AWS have to store my secret key in plain text instead of slat hash, in order to generates its own HMAC/sign key? – petertc Aug 31 '16 at 12:08
1

You're right, you can't upload using pre-signed URLs.

There is a different, more complex capability that you can use called GetFederationToken. This will return you some temporary credentials, to which you can apply any policy (permissions) that you like.

So for example, you could write a web service POST /upload that creates a new folder in S3, then creates temporary credentials with permissions to PutObject to only this folder, and returns the folder path and credentials to the caller. Presumably, some authorization check would be performed by this method as well.

You can't embed cloud credentials, or any other credentials, in your application code. Which isn't to say that nobody ever accidentally does this, even security professionals.

To safely distribute credentials to your infrastructure, you need tool support. If you use an AWS facility like CloudFormation, you can (somewhat more) safely give it your credentials. CloudFormation can also create new credentials on the fly. If you use a PaaS like Heroku, you can load your credentials into it, and Heroku will presumably treat them carefully. Another option for AWS is IAM Role. You can create an IAM Role with permission to do what you need, then "pass" the role to your EC2 instance. It will be able to perform the actions permitted by the role.

A final option is a dedicated secrets management service, such as Conjur. (Disclaimer: I'm a founder of the company). You load your credentials and other secrets into a dedicated virtual appliance, and you define access permissions that govern the modification and distribution of the credentials. These permissions can be granted to people or to "robots" like your EC2 box. Credentials can be retrieved via REST or client APIs, and every interaction with credentials is recorded to a permanent record.

kgilpin
  • 2,201
  • 18
  • 18
1

I've been trying to answer the same question... the generate_url(x) looks quite promising.

This link had a suggestion about creating a cloudfront origin access identity, which I'm guessing taps into the IAM authentication... meaning you could create a key for each application without giving away your main account details. With IAM, you can set permissions based on keys as to what they can do, so they can have limited access.

Note: I don't know if this really works, I haven't tried it yet, but it might be another avenue to explore.

2 - Create a Cloudfront "Origin Access Identity"

This identity can be reused for many different distributions and keypairs. It is only used to allow cloudfront to access your private S3 objects without allowing everyone. As of now, this step can only be performed using the API. Boto code is here:

# Create a new Origin Access Identity
oai = cf.create_origin_access_identity(comment='New identity for secure videos')

print("Origin Access Identity ID: %s" % oai.id)
print("Origin Access Identity S3CanonicalUserId: %s" % oai.s3_user_id)
Adam Morris
  • 8,265
  • 12
  • 45
  • 68
  • Now my problem is not in downloading stuff from S3 without given my AWS keys. Key.generate_url(expires_in=60, method='GET') works perfectly. I just don't know how to upload in this way. – michael.luk Apr 06 '12 at 14:28
0

Don't put it in applications you plan to distribute. It'll be visible and they can launch instances that are directly billable to you or worst..they can take down instances if you use it in production.

I would look at your programs design and seriously question why I need to include that information in the app. If you post more details on the design I'm sure we can help you figure out a way in which you don't need to bundle this information.

Lostsoul
  • 25,013
  • 48
  • 144
  • 239
  • ahh, I understand now. They def do not put the keys in the client. I'm just guessing but I suspect the app connects to some server which in turn either redirects them to upload the S3 bucket or more likely the server takes the uploads it to S3 bucket on their behalf without the user ever directly connecting to it. – Lostsoul Mar 29 '12 at 16:14
  • @michael.luk Think of the risks if that key is ever compromised. A single person would have access to all data people store in dropbox. I don't think dropbox would ever encode that into the app, even with the strongest encryption, all it would take is one smart and determined cracker to compromise your entire business. – Lostsoul Mar 29 '12 at 16:16
  • My suggestion is to look at open source projects and see if you can see how they are designing their system to protect the keys. Check http://code.google.com/p/s3fs/wiki/FuseOverAmazon but there's a ton more. – Lostsoul Mar 29 '12 at 16:17
  • Its very easy to protect yourself from a public key being able to launch instances etc... If i only want a key i distribute to be able to read information in S3 or SimpleDB i can give it read-only access etc. – bwight Mar 29 '12 at 16:18