Manage RDS access with AWS Secrets Manager

Question

I am currently working with Eclipse and the AWS Toolkit for Eclipse. My project already works and it is doing its job, which is to connect to an RDS instance and return JSON objects to API Gateway calls.

I just got a new requirement, we are to use the service SecretsManager to authomatically rotate RDS configuration such as Users, passwords and so on.

The problem is when I try to import classes such as GetSecretValueResponse, I get a The import com.amazonaws.services.secretsmanager cannot be resolved. When I explore the documentation and the SDK, there exists a GetSecretValueRequest but not a GetSecretValueResponse, so I am not being able to make sense on what should I do, nor I have found anything similar to an example I can study.

The following code is what I am trying to implement and is given by Amazon itself (in the Secrets Manager page there is a button you can click to see how it would go with Java, in this case), and it is presented without any modification yet because as I said I do not know how to import several classes:

// Use this code snippet in your app.
public static void getSecret() {
String secretName = "secretName";
String endpoint = "secretEndpoint";
String region = "region";

AwsClientBuilder.EndpointConfiguration config = new AwsClientBuilder.EndpointConfiguration(endpoint, region);
AWSSecretsManagerClientBuilder clientBuilder = AWSSecretsManagerClientBuilder.standard();
clientBuilder.setEndpointConfiguration(config);
AWSSecretsManager client = clientBuilder.build();

String secret;
ByteBuffer binarySecretData;
GetSecretValueRequest getSecretValueRequest = GetSecretValueRequest.builder()
        .withSecretId(secretName)
        .build();
GetSecretValueResponse getSecretValueResponse = null;
try {
    getSecretValueResponse = client.getSecretValue(getSecretValueRequest);

} catch(ResourceNotFoundException e) {
    System.out.println("The requested secret " + secretName + " was not found");
} catch (InvalidRequestException e) {
    System.out.println("The request was invalid due to: " + e.getMessage());
} catch (InvalidParameterException e) {
    System.out.println("The request had invalid params: " + e.getMessage());
}

if(getSecretValueResponse == null) {
    return;
}

// Decrypted secret using the associated KMS CMK
// Depending on whether the secret was a string or binary, one of these fields will be populated
if(getSecretValueResponse.getSecretString() != null) {
    secret = getSecretValueResponse.getSecretString();
}
else {
    binarySecretData = getSecretValueResponse.getSecretBinary();
}

// Your code goes here. 
}

score 7 · Accepted Answer · answered Jun 13 '18 at 17:11

I had the same problem, the code that is present on AWS page doesn't work out of the box. The class you are looking for is GetSecretValueResult Here are the latest java docs

https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/secretsmanager/model/GetSecretValueResult.html

Here is a piece that shall work:

public void printRdsSecret() throws IOException {
    String secretName = "mySecretName";

    System.out.println("Requesting secret...");
    AWSSecretsManager client = AWSSecretsManagerClientBuilder.standard().build();

    GetSecretValueRequest getSecretValueRequest = new GetSecretValueRequest().withSecretId(secretName);

    GetSecretValueResult getSecretValueResult = client.getSecretValue(getSecretValueRequest);

    System.out.println("secret retrieved ");
    final String secretBinaryString = getSecretValueResult.getSecretString();
    final ObjectMapper objectMapper = new ObjectMapper();

    final HashMap<String, String> secretMap = objectMapper.readValue(secretBinaryString, HashMap.class);

    String url = String.format("jdbc:postgresql://%s:%s/dbName", secretMap.get("host"), secretMap.get("port"));
    System.out.println("Secret url = "+url);
    System.out.println("Secret username = "+secretMap.get("username"));
    System.out.println("Secret password = "+secretMap.get("password"));
 }

This was tested with aws-java-sdk-secretsmanager of version 1.11.337

Thank you very much for your help @Nune Isabekyan, I just found just that yesterday, I dont know why I tripped so much over that but I could not get past this problem! Your solution is exactly the same, I am glad I am not alone and that you shared! — monkey intern, Jun 14 '18 at 10:22
I find super interesting your use of ObjectMapper, did not know about the class. I use JSONObject to parse it, but yours seems cleaner... — monkey intern, Jun 14 '18 at 10:25
glad it helped! Ideally, you would have a custom type - a Java object, holding your configuration and then you would read your configs/secrets into the object instead of a "generic" hashmap... SecretConfig user = mapper.readValue(jsonInString, SecretConfig.class); — Nune Isabekyan, Jun 15 '18 at 10:38
they updated the console to show GetSecretValueResult as well in SampleCode — committedandroider, Jul 02 '18 at 18:06
@committedandroider - but it still gives the result as complete json. You still have to parse json to get securestring value. There is no method which gives the secretvalue directly to you. — sumit sachdeva, Mar 15 '19 at 12:59

Eugene Maysyuk · Answer 2 · 2019-07-06T21:05:29.107

I think the main problem was in lack of dependencies to AWS SDK v2.

Adding a code snippet here that utilizes AWS SDK v2. Just in case anybody is looking for this.

package com.may.util;

import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.secretsmanager.SecretsManagerClient;
import software.amazon.awssdk.services.secretsmanager.model.DecryptionFailureException;
import software.amazon.awssdk.services.secretsmanager.model.GetSecretValueRequest;
import software.amazon.awssdk.services.secretsmanager.model.GetSecretValueResponse;
import software.amazon.awssdk.services.secretsmanager.model.InternalServiceErrorException;
import software.amazon.awssdk.services.secretsmanager.model.InvalidParameterException;
import software.amazon.awssdk.services.secretsmanager.model.InvalidRequestException;
import software.amazon.awssdk.services.secretsmanager.model.ResourceNotFoundException;

public class SecretsManagerUtil {

    public static String obtainSecret() {
        String secretName = "db_secret_name";
        String region = "us-east-1";

        SecretsManagerClient client = SecretsManagerClient.builder().region(Region.of(region)).build();
        GetSecretValueResponse response = null;

        try {
            response = client.getSecretValue(GetSecretValueRequest.builder().secretId(secretName).build());
        } catch (DecryptionFailureException e) {
            // Secrets Manager can't decrypt the protected secret text using the provided KMS key.
            // Deal with the exception here, and/or rethrow at your discretion.
            throw e;
        } catch (InternalServiceErrorException e) {
            // An error occurred on the server side.
            // Deal with the exception here, and/or rethrow at your discretion.
            throw e;
        } catch (InvalidParameterException e) {
            // You provided an invalid value for a parameter.
            // Deal with the exception here, and/or rethrow at your discretion.
            throw e;
        } catch (InvalidRequestException e) {
            // You provided a parameter value that is not valid for the current state of the resource.
            // Deal with the exception here, and/or rethrow at your discretion.
            throw e;
        } catch (ResourceNotFoundException e) {
            // We can't find the resource that you asked for.
            // Deal with the exception here, and/or rethrow at your discretion.
            throw e;
        }

        return response.secretString();
    }
}

Deserialize and print secret:

public class SecretPrinter {

private static final Logger logger = LoggerFactory.getLogger(SecretPrinter.class);

public void printSecret() {
    String json = SecretsManagerUtil.obtainSecret(); // secret in json format

    RdsSecret secret;
    try {
        secret = new ObjectMapper().disable(FAIL_ON_UNKNOWN_PROPERTIES).readValue(json, RdsSecret.class);
    } catch (IOException e) {
        logger.error("Couldn't parse secret obtained from AWS Secrets Manager!");
        throw new RuntimeException(e);
    }

    System.out.println("username: " + secret.getUsername());
    System.out.println("password: " + secret.getPassword());
}

static class RdsSecret {
    private String username;
    private String password;

    public String getUsername() {
        return username;
    }

    public void setUsername(String username) {
        this.username = username;
    }

    public String getPassword() {
        return password;
    }

    public void setPassword(String password) {
        this.password = password;
    }
}

}

Maven:

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>software.amazon.awssdk</groupId>
      <artifactId>bom</artifactId>
      <version>2.6.3</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependency>
  <groupId>software.amazon.awssdk</groupId>
  <artifactId>secretsmanager</artifactId>
</dependency>

Gradle:

implementation platform('software.amazon.awssdk:bom:2.6.3')
implementation 'software.amazon.awssdk:secretsmanager'

When a developer runs this locally, it is normal that the code fails because the machine does not have the required permissions to access the secret, right? That would be production-ready code for the deployed instance in the cloud which, itself, would have access to the secrets because the CodeCommit linked to the instance has the necessary permissions? (My current problem is that I'm trying to prevent developers from accessing the secrets locally, but still be able to run the code locally... what is the suggested way of doing that?) — payne, Jan 14 '20 at 18:55
Locally, `GetSecretValueRequest.builder().secretId(secretName).build()` crashes with `software.amazon.awssdk.services.secretsmanager.model.SecretsManagerException: The security token included in the request is invalid.`., yet I'm using the value provided by the code-snippet given to me when I created the Secret in the AWS Secrets Manager. — payne, Jan 14 '20 at 18:57
@payne This code is production ready. In my understanding (I was probably reading about this somewhere) AWS SDK under the hood uses metadata attached to each physical instance (machine) to check if particular machine has access to SecretsManager (and particular secret). Local machine doesn't have this metadata that's why local code will not be able to reach AWS SecretsManager. This might can be configured but to be honest I don't know how. Please, leave a comment here if you find some useful information in this regard. — Eugene Maysyuk, Jan 14 '20 at 19:03
@payne, in reply to comment one. If developers will be able to run the code locally, they will most likely be able to debug the code locally and evaluate plain password after retrieval from SecretsManager. — Eugene Maysyuk, Jan 14 '20 at 19:16
Just tested in a deployed environment: as expected, it got past the Exception I mentioned earlier, and now we're hitting `not authorized to perform: secretsmanager:GetSecretValue`. Thanks for your help! We will resolve this by adding the proper permissions. — payne, Jan 14 '20 at 21:14

score 3 · Answer 3 · answered Nov 05 '19 at 04:22

aws-secretsmanager-jdbc can be used to access AWS RDS via AWS secrete manager. https://github.com/aws/aws-secretsmanager-jdbc

Below is the content of my application.properties

spring.datasource.url=jdbc-secretsmanager:mysql://dev-xxxx-database.cluster-xxxxxxxxx.ap-southeast-1.rds.amazonaws.com:3306/dev_xxxxxx

spring.datasource.username=/secret/application
spring.datasource.driver-class-name=com.amazonaws.secretsmanager.sql.AWSSecretsManagerMySQLDriver

spring.jpa.database-platform = org.hibernate.dialect.MySQL5Dialect

Below is the secret in AWS secret manager.

By using this method, you don't need to obtain username and password manually and create data sources.

What about documentdb non sql databases? Does aws have something like that for documentdb as well? — Guchelkaben, Mar 20 '20 at 08:21

score 1 · Answer 4 · answered Mar 20 '19 at 08:02

1

just add the lib to your pom: https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-secretsmanager

answered Mar 20 '19 at 08:02

ido flax

528
1
10
19

score 1 · Answer 5 · answered Aug 14 '19 at 00:27

I recommend using aws secret manger jdbc wrapper to access the RDS (https://github.com/aws/aws-secretsmanager-jdbc). You don't need to deal with fetching the secret, decode and move around with password in text before passing to RDS client.

Just pass in secret id to RDS client and the jdbc wrapper will handle the rest.

Tanaya Maji · Answer 6 · 2022-09-23T14:14:37.087

1

I have faced the same issue. just delete the scope line i.e "test" from the dependency. It'll work

edited Sep 23 '22 at 14:14

answered Sep 23 '22 at 14:13

Tanaya Maji

11
2

score 0 · Answer 7 · answered Aug 23 '20 at 22:28

You're missing "aws-java-sdk-secretsmanager" dependency, you have to add this to your pom.xml and then import on your java class.

From Maven:

    <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-java-sdk-secretsmanager</artifactId>
        <version>1.11.355 </version>
    </dependency>

AWS Reference

If you are not using maven, you have add AWS SDK to your existing project.

Manage RDS access with AWS Secrets Manager

7 Answers7