0

I have configured my spring boot application to obtain the database credentials from the vault database backend. This is working fine when I run it with the vault in 'vault for development mode'. But, now I am trying to get it to work when the vault server is running in the production mode.

I am getting below exception in the spring boot application at the context loading.

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2023-01-31 19:22:12.511  WARN 32872 --- [    Test worker] LeaseEventPublisher$LoggingErrorListener : [RequestedSecret [path='database/creds/myrole', mode=RENEW]] Lease [leaseId='database/creds/myrole/sDABdI1ZfDyYiuzZ3WUmZN9o', leaseDuration=PT5M, renewable=true] Status 400 Bad Request: failed to revoke entry: resp: (*logical.Response)(nil) err: ERROR: role "v-root-myrole-o1PzAyuwybhLzZpPx1k0-1675173125" cannot be dropped because some objects depend on it (SQLSTATE 2BP01); nested exception is org.springframework.web.client.HttpClientErrorException$BadRequest: 400 Bad Request

org.springframework.vault.VaultException: Status 400 Bad Request: failed to revoke entry: resp: (*logical.Response)(nil) err: ERROR: role "v-root-myrole-o1PzAyuwybhLzZpPx1k0-1675173125" cannot be dropped because some objects depend on it (SQLSTATE 2BP01); nested exception is org.springframework.web.client.HttpClientErrorException$BadRequest: 400 Bad Request
at org.springframework.vault.client.VaultResponses.buildException(VaultResponses.java:63) ~[spring-vault-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.vault.core.VaultTemplate.doWithSession(VaultTemplate.java:391) ~[spring-vault-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.vault.core.lease.SecretLeaseContainer.doRevokeLease(SecretLeaseContainer.java:785) [spring-vault-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.vault.core.lease.SecretLeaseContainer.destroy(SecretLeaseContainer.java:503) [spring-vault-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:258) [spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:571) [spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:543) [spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:1072) [spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:504) [spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingletons(DefaultListableBeanFactory.java:1065) [spring-beans-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1060) [spring-context-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:1029) [spring-context-5.2.0.RELEASE.jar:5.2.0.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:978) [spring-context-5.2.0.RELEASE.jar:5.2.0.RELEASE]
...

Also I am getting below error in the vault (docker container) log as well.

2023-01-31T13:51:38.534Z [ERROR] expiration: failed to revoke lease: lease_id=database/creds/myrole/JhS1BcblrfdqHAWGnTSjTx8p error="failed to revoke entry: resp: (*logical.Response)(nil) err: ERROR: role \"v-root-myrole-PMfgb1jqWGBNlXIGKM2E-1675171846\" cannot be dropped because some objects depend on it (SQLSTATE 2BP01)"
2023-01-31T13:51:38.660Z [ERROR] expiration: failed to revoke lease: lease_id=database/creds/myrole/tX4nIeWS4tWS0bJGpMSB5uAA error="failed to revoke entry: resp: (*logical.Response)(nil) err: ERROR: role \"v-root-myrole-LpuSXIp0yzV6uqQpwllS-1675171653\" cannot be dropped because some objects depend on it (SQLSTATE 2BP01)"
2023-01-31T13:51:54.010Z [ERROR] expiration: failed to revoke lease: lease_id=database/creds/myrole/JhS1BcblrfdqHAWGnTSjTx8p error="failed to revoke entry: resp: (*logical.Response)(nil) err: ERROR: role \"v-root-myrole-PMfgb1jqWGBNlXIGKM2E-1675171846\" cannot be dropped because some objects depend on it (SQLSTATE 2BP01)"
2023-01-31T13:52:04.127Z [ERROR] expiration: failed to revoke lease: lease_id=database/creds/myrole/tX4nIeWS4tWS0bJGpMSB5uAA error="failed to revoke entry: resp: (*logical.Response)(nil) err: ERROR: role \"v-root-myrole-LpuSXIp0yzV6uqQpwllS-1675171653\" cannot be dropped because some objects depend on it (SQLSTATE 2BP01)"
2023-01-31T13:52:12.474Z [ERROR] secrets.system.system_e586bc2f: lease revocation failed: lease_id=database/creds/myrole/sDABdI1ZfDyYiuzZ3WUmZN9o error="failed to revoke entry: resp: (*logical.Response)(nil) err: ERROR: role \"v-root-myrole-o1PzAyuwybhLzZpPx1k0-1675173125\" cannot be dropped because some objects depend on it (SQLSTATE 2BP01)"

I tried to add below policy to the default policy as well, but it didn't resolve this issue.

path "sys/leases/revoke/*" {
  capabilities = ["update"]
}

Since it has this in the exception message, I tried to add read, update capabilities for "database/creds/myrole" in the default policy, and all of a sudden vault started throwing the below error as well. But, it was fixed after I removed the policy config and re-updated the password of the user from the database.

expiration: failed to revoke lease: lease_id=database/creds/myrole/tX4nIeWS4tWS0bJGpMSB5uAA error="failed to revoke entry: resp: (*logical.Response)(nil) err: error verifying connection: failed to connect to `host=host.docker.internal user=spring_user database=ax`: failed SASL auth (FATAL: password authentication failed for user \"spring_user\" (SQLSTATE 28P01))"

Appreciate if someone can help me resolve this issue, I have been trying to find a solution, but now I am confused.

udani
  • 1,243
  • 2
  • 11
  • 33

1 Answers1

0

The reason for the sql error was, all the roles created by the vault during the entire was available in the database. As the error mentioned, the vault is unable to drop the role as there were other objects tied to the role.

The 400 bad request error didn't come when I restarted the service after revoking privileges and dropping all the roles created by the vault.

Then adding the script below the Revocation Statement of my database backend role, helped to remove the previously created role automatically, when the lease period expired.

DROP OWNED BY "{{name}}"; REVOKE ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA ax_management FROM "{{name}}"; DROP ROLE "{{name}}";
udani
  • 1,243
  • 2
  • 11
  • 33