AWS Secret can't be converted into key names and value pairs

Question

I have a python script that creates some access keys in AWS and stores them in secrets manager.

However, when I store the keys I get an error that says:

The secret value can't be converted to key name and value pairs

The secret is stored like this in secrets manager (obfuscated the secret key):

[{'Access Key': 'AKIA5AODVC64THTZNML7'}, {'Secret Key': 'SecretSecretsecretSecretSecretSecretSecr'}]

My script is using these lines to create the secret:

secret_name = 'my_secret'
secret_description = 'describing the secret'
kms_key_id = create_kms_key()
key_info = str([{"Access Key":access_key},{"Secret Key":secret_key}])
aws_secret = `secrets_client.create_secret(Name=secret_name,Description=secret_description,KmsKeyId=kms_key_id,SecretString=key_info,Tags=[{'Key': 'Name','Value': user_name}])`

How can I get the access / secret key into a format that secrets manager understands enough to convert to key / value pairs?

A couple of clarification questions: 1. What Python library are you for the `create_kms_key ` and the `create_secret ` call. 2. Is the line "[{'Access Key': 'AKIA5AODVC64THTZNML7'}, {'Secret Key': 'SecretSecretsecretSecretSecretSecretSecr'}]" the desired format that you want or current and likely erroneous format that you currently have? — entpnerd, Oct 08 '19 at 02:57
I'm using boto3 for the `create_secret` and `create_kms_key` functions. The line is the desired format, but I am not including the real secret key. — bluethundr, Oct 11 '19 at 15:37

MacSanhe · Accepted Answer · 2019-10-14T18:35:47.733

First, you can store AWS Access Key and Secret Key in AWS Secret Manager, but I strongly not recommend that. But I can still offer a solution to easily do that:

manually put your secret value in json or create one with pysecret.

from pysecret import AWSSecret

aws_profile = "my_aws_profile"
aws = AWSSecret(profile_name=aws_profile)

secret_id = "my-example-secret"
secret_data = {
    "iam_user_1": {
        "access_key": "AAAAAAAAAAAAAAAAAAAAA",
        "secret_key": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
    },
    "iam_user_2": {
        "access_key": "BBBBBBBBBBBBBBBBBBBBB",
        "secret_key": "YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY"
    }
}
aws.deploy_secret(name=secret_id, secret_data=secret_data) # or you can pass kms_key_id if you created a custom kms key

read your secret value in lambda function or in any of your python code.

aws = AWSSecret(profile_name=aws_profile) # in lambda code, don't need ``profile_name=aws_profile``

access_key = aws.get_secret_value(secret_id="my-example-secret", key="iam_user_1.access_key") # AAAAAAAAAAAAAAAAAAAAA
secret_key = aws.get_secret_value(secret_id="my-example-secret", key="iam_user_1.secret_key") # XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

access_key = aws.get_secret_value(secret_id="my-example-secret", key="iam_user_1.access_key") # BBBBBBBBBBBBBBBBBBBBB
secret_key = aws.get_secret_value(secret_id="my-example-secret", key="iam_user_1.secret_key") # YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY

I created an open source library called pysecret, here's the documentation of AWS Secret Manager integration: https://github.com/MacHu-GWU/pysecret-project#aws-key-management-service-and-secret-manager-integration

Second

When ever you want to create an AWS Access Key pair, think of who is going to use it, Human or Machine. If it is Machine, please use IAM Role instead of IAM User. If it is Human, the Human should be responsible to securely store it, NOT AWS Secret Manager.

Hope this answers your question.

Thanks! I will try that. I agree with what you're saying about making users responsible for storing their keys and only using secrets manager for automated key retrieval in scripts. However, reason we use secrets manager for storing keys for people is that email is disallowed as a way of sending credentials. Secrets manager is approved by security to distribute key information in my organization. — bluethundr, Oct 11 '19 at 20:30
@bluethundr be aware that secret manager can only manage secret value up to 10KB. If your organization is very big, this is not a scalable way. Based on your description, I assume that you are trying to give everyone the same and single IAM User, then ask them to use it to retrieve Another access key, and then perform some operation. If this is what you are trying to do, DON't DO THAT, it is a FAMOUS ANTI PATTERN. I recommend you to use Assumed Role method to manage mass IAM User's access to different Account and different services. — MacSanhe, Oct 14 '19 at 18:40
Hi, thanks, yes we are NOT doing what you describe. Each developer will have their own set of access keys. And they will retrieve them from secrets manager. They access the different accounts through role switching. I tried your solution and unfortunately the formatting is still broken. I only want to present the key data in a clean format that the users won't complain about! Sure it works as is, but you know how people like to complain. This is the code that I used and the result: https://pastebin.com/t6kMWYQf — bluethundr, Oct 24 '19 at 16:20
@bluethundr because ``str(secret_data)`` is a bad idea. use ``json.dumps(secret_data)`` instead. — MacSanhe, Nov 06 '19 at 00:12

score 2 · Answer 2 · answered Sep 24 '19 at 22:47

First off, you should reconsider if you really need to store AWS access keys in Secrets Manager. How are you planning to get the credentials that will be used to make the call to Secrets Manager?

Look at using temporary role creds, for example - Don't use the creds you have to get another cred pair from SecretsManager. Use the creds you have to get temp creds instead.

On to your actual question, it looks like you are trying view the secret value in the console (since that is when the error "The secret value can't be converted to key name and value pairs" is shown). The python code snippet that you used to create the secret worked correctly. The error is shown when the console cannot parse the SecretString to key value pairs. This is expected behaviour, since your top level obj in the SecretString is an array.

If you click on the "Plaintext" tab right above the red error display box, you'll see the stored SecretString without any attempt at parsing into key value pairs.

Obviously I am not doing this on my user account. I am using this script for the other people in my company who have to get their access keys from secrets manager for their apps. — bluethundr, Oct 11 '19 at 15:17

score 0 · Answer 3 · answered Nov 11 '21 at 20:14

This Worked for me by adding json.dumps to send a json within the text values:

import json
secret_name = 'my_secret'
secret_description = 'describing the secret'
kms_key_id = create_kms_key()
key_info = str(json.dumps([{"Access Key":access_key},{"Secret Key":secret_key}]))
aws_secret = `secrets_client.create_secret(Name=secret_name,Description=secret_description,KmsKeyId=kms_key_id,SecretString=key_info,Tags=[{'Key': 'Name','Value': user_name}])`

AWS Secret can't be converted into key names and value pairs

3 Answers3