Using multiple types of Vault secrets in one Spring boot application

Question

Consider the following application.properties file of a Spring Boot application.

# Spring configuration parameters
spring.application.name=MyApplication
server.port=${SERVICE_PORT:8080}

# SSL Configuration
server.ssl.key-store-type=JKS
server.ssl.key-store=classpath:keystore/keystore.jks
server.ssl.key-store-password=${KEYSTORE_PASSWORD}
server.ssl.key-alias=my-alias
server.ssl.enabled=true

# Security configuration
keycloak.realm = ${AUTHENTICATION_REALM}
keycloak.auth-server-url = https://${AUTHENTICATION_HOST}:${AUTHENTICATION_PORT}/
keycloak.ssl-required = external
keycloak.resource = ${AUTHENTICATION_CLIENT}
keycloak.credentials.secret = ${AUTHENTICATION_SECRET}
keycloak.use-resource-role-mappings = true
keycloak.bearer-only = true
keycloak.truststore=classpath:keystore/cacerts.jks
keycloak.truststore-password=${TRUSTSTORE_PASSWORD}

# Database configuration
spring.datasource.url=jdbc:mysql://${DB_HOST}:${DB_PORT:3306}/mydatabase
spring.datasource.driverClassName=com.mysql.cj.jdbc.Driver
spring.datasource.username=${DB_USER}
spring.datasource.password=${DB_PASSWORD}
spring.jpa.database-platform=org.hibernate.dialect.MySQL8Dialect
spring.jpa.generate-ddl=true
spring.jpa.hibernate.ddl-auto=update

From security point of view, I am investigating the possibilities to use HashiCorp Vault to safely store and manage the secrets required by the application. Depending on the kind of secrets, the following distinction can be made.

1. SERVICE_PORT, AUTHENTICATION_REALM, AUTHENTICATION_HOST, AUTHENTICATION_PORT, AUTHENTICATION_CLIENT, DB_HOST and DB_PORT contain no highly sensitive information and can be considered as non-changing over time. Would it be considered safe and common practise to store those values as environment parameters in let's say a docker-compose file?
2. Parameters like KEYSTORE_PASSWORD, TRUSTSTORE_PASSWORD and AUTHENTICATION_SECRET are sensitive but do not change (often) over time since changing those passwords would require to change/update the corresponding keystore files. Does it make sense to store those values as key-value secrets?
3. Parameters like DB_USER and DB_PASSWORD are highly susceptible to abuse and need to be handled very carefully. Due to the nature of those values, I would like to store them as dynamic generated secrets. Does this also make sense?

I found a series of tutorials and articles online describing how to integrate Vault into Spring Boot. Unfortunately, none of the found articles described the use of multiple secret engines in the same application.

1. Is it possible to use multiple secret engines (key-value and database) in the same Spring Boot application? If so, what should the bootstrap.yml file look like? I am struggling to find the right configuration to do so...
2. What would be the best approach? Let Spring Boot handle to retrieval of the secrets using spring-cloud-starter-vault-config or handle secret retrieval by some orchestration mechanism that spawns the application's different Docker containers. Currently I am using docker compose with a bunch of environment parameters containing all secrets needed by Spring which is off course a very bad idea!

EDIT01

Adding the following configuration to the application.properties file mentioned before solves the problem of accessing secrets in the KV secret engine.

# Vault Server Configuration
spring.cloud.vault.host=${VAULT_HOST:localhost}
spring.cloud.vault.port=${VAULT_PORT:8200}
spring.cloud.vault.scheme=http
spring.cloud.vault.connection-timeout=5000
spring.cloud.vault.read-timeout=15000
spring.cloud.vault.authentication=TOKEN
spring.cloud.vault.token=${VAULT_TOKEN}
spring.config.import=vault://secrets/my-application, vault://database

# Vault Common Secrets Configuration
spring.cloud.vault.kv.enabled=true
spring.cloud.vault.kv.backend=secrets

To access dynamic secrets for the database I added the spring-cloud-vault-config-databases dependency besides the spring-cloud-starter-vault-config in the pom.xml file and added the following configuration to application.properties.

# Vault Database Secrets Configuration
spring.cloud.vault.database.enabled=true
spring.cloud.vault.database.backend=database
spring.cloud.vault.database.role=ROLE_MANAGE_USERS
spring.cloud.vault.database.static-role=false
spring.cloud.vault.database.username-property=DB_USER
spring.cloud.vault.database.password-property=DB_PASSWORD

The configuration of the dynamic secret engine on Vault has been done and seems to work. I can use the UI to generate credentials allowing me to login and perform tasks on the MySQL database. So I assume everything there works as it should.

The Spring Boot application itself cannot retrieve database credentials resulting in the Access denied for user '${DB_USER}'@'172.19.0.1' (using password: YES) error message being thrown.

As explained in some tutorials I found, I tried to put every Vault related configuration in a bootstrap.properties file as well, but KV secrets no longer work under this configuration. I also tried to split KV and database secrets in application.properties and bootstrap.properties respectively, but this also does not seem to work...

I also tried to put ${} around DB_USER and DB_PASSWORD. Unfortunately, no effect.

Obviously, the configuration seems to be missing something (probably very basic) but I just don't seem to figure it out...

Thank you for reading my question and pointing me into the right direction.

Answer 1

I added my findings as answer for future reference and other users trying to achieve the same. It turns out the initial configuration has a few errors.

First, I had to remove

spring.datasource.username=${DB_USER}
spring.datasource.password=${DB_PASSWORD}

Secondly, I had to replace

spring.cloud.vault.database.username-property=DB_USER
spring.cloud.vault.database.password-property=DB_PASSWORD

with

spring.cloud.vault.database.username-property=spring.datasource.username
spring.cloud.vault.database.password-property=spring.datasource.password

This automatically adds the correct spring.datasource.username and spring.datasource.password properties to the configuration so they don't have to be defined explicitly anymore.

Finally, verify that your Vault configuration has two secret engines configured: one named database (for your database dynamic secrets) and one named secrets (for your static key-value secrets).

The secrets secret engine should contain a secret with the same name as the value specified in spring.config.import property (which is in this case vault://secrets/my-application). This secret engine defines the values used as KEYSTORE_PASSWORD, TRUSTSTORE_PASSWORD and AUTHENTICATION_SECRET.

The database secret engine should have the same name as the value of property spring.cloud.vault.database.backend (which is database in this case). Since this secret engine manages the database's dynamic secrets, appropriate usernames and passwords are created by Vault on-demand. Ensure you configure the correct database connection and role for each database in Vault. Also check the permissions for the database user in charge of creating the username and password credentials.

About the additional questions in my post:

1. Is it possible to use multiple secret engines (key-value and database) in the same Spring Boot application? If so, what should the bootstrap.yml file look like? I am struggling to find the right configuration to do so...

=> Yes you can and you don't need a bootstrap.yml file.

2. What would be the best approach? Let Spring Boot handle to retrieval of the secrets using spring-cloud-starter-vault-config or handle secret retrieval by some orchestration mechanism that spawns the application's different Docker containers. Currently I am using docker compose with a bunch of environment parameters containing all secrets needed by Spring which is off course a very bad idea!

=> Since I am using docker-compose currently, I decide to let Spring handle the retrieval of secrets. There is definitely rooms for improvement here...