1

I am using a kubernetes service account to pass the DAX role to the spring boot application which is connecting to DAX cluster. For about 40 minutes, the service is able to connect to DAX and retrieve the response from DynamoDB tables. However I am seeing below exception after sometime:

extendedStackTrace":"software.amazon.dax.exceptions.DaxServiceException: [4.23.31.33] Connection requires authentication (Service: null, Status Code: -1, Request ID: null)\n\tat software.amazon.dax.exceptions.DaxServiceException.pickException(DaxServiceException.java:177) ~[amazon-dax-client-2.0.1.jar!/:?]\n\tat software.amazon.dax.channel.StatusCodeHandler.channelRead(StatusCodeHandler.java:43) ~[amazon-dax-client-2.0.1.jar!/:?]\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379

"thrown":{"commonElementCount":0,"localizedMessage":"Unable to call cluster endpoint: No endpoints available","message":"Unable to call cluster endpoint: No endpoints available","name":"software.amazon.awssdk.core.exception.SdkClientException"

This is how I have created a DAX Client Configuration. It is a bean present in @Configuration annotated class

  @Bean
  public ClusterDaxClient clusterDaxClient() {

    LOG.info("Started Creating DAX Cluster Client");

    ClusterDaxClient clusterDaxClient = null;
    try {
      clusterDaxClient =
          ClusterDaxClient.builder().overrideConfiguration(setClientConfiguration()).build();

      LOG.info("DAX Client Created Successfully");
      return clusterDaxClient;

    } catch (IOException e) {
      LOG.error("Error while connecting to DAX Client : ", e);
    }
    
    return null;
  }

  private software.amazon.dax.Configuration setClientConfiguration() {
    AwsCredentialsProvider provider =
        StaticCredentialsProvider.create(
            WebIdentityTokenFileCredentialsProvider.create().resolveCredentials());

    return software.amazon.dax.Configuration.builder()
        .credentialsProvider(provider)
        .region(Region.of(awsRegion))
        .url(daxEndpoint)
        .build();
  }

NOTE: I have verified that DAX cluster is in same VPC as EKS with same subnets and also Inbound rules allowed on Port 9111 in DAX Security Group.

Mohammad Hanan
  • 21
  • 6
  • `level":"ERROR","loggerName":"io.netty.util.ResourceLeakDetector","message":"LEAK: ByteBuf.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more information.\nRecent access records: \nCreated at:\n\tio.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:96)\n\tio.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:188)` Seeing this error also in logs – Mohammad Hanan May 21 '22 at 11:44
  • did you figure out what was causing this? I'm getting the same error (also after some time running only) and having a hard time tracking it down – Arthur Mar 09 '23 at 11:44

0 Answers0