I'm using the ES service on AWS. I've made good progress learning how to use it, particularly with the ES HighLevelRestClient. So now I wanted to secure the ES server, adding user authentication security. Amazon provides the "access key" and "secret key" values to use, analogous to "user/password" credentials. Unfortunately, they also provide their own AWSCredentials and AWS4Signer classes to create credentials & sign the request!
Their code examples are simple enough, and work fine with Elastic Co's Java "Low Level" RestClient. Here's a snippet analogous to what amazon recommends at https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-indexing-programmatic.html:
credentialsProvider = new AWSStaticCredentialsProvider (new BasicAWSCredentials ("access key", "secret key")); // while debugging
HttpRequestInterceptor interceptor =
new AWSRequestSigningApacheInterceptor (serviceName, signer, credentialsProvider);
RestClient lowLevelClient = RestClient.builder (HttpHost.create (esURL))
.setHttpClientConfigCallback (hacb -> hacb.addInterceptorLast (interceptor)).build ();
The good news is, this works fine! Unfortunately, the obvious extension to a signed version of the ES High Level Rest Client (as suggested in Add authentication in elasticsearch high level client for JAVA) doesn't work! That is:
RestClientBuilder builder = RestClient.builder (HttpHost.create (esURL))
.setHttpClientConfigCallback (hacb -> hacb.addInterceptorLast (interceptor));
highLevelClient = new RestHighLevelClient (builder);
always leads to the following error message from AWS:
{"message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details."}
I speculate that the ES high-level rest client constructs JSON queries which the AWS interceptor does not see when creating the signature. Since the equivalent method works fine for the Elastic Co cloud instance (see SO thread mentioned above), I'm guessing this is where AWS folks and ES folks are not talking to each other.
Is there some way I can use the AWS CredentialsProvider and Signer, but with the Apache HttpAsyncClientBuilder::setDefaultCredentialProvider method? Or is there a version of the ES Rest API in which the Java High Level Rest Client constructs the request fully before the AWS interceptor signs it?
Otherwise, I'll be forced to use IP address signing (ugh), start using the Low Level Rest Client (double ugh), or use some combination of ssh tunneling and AWS VPC security (many ughs).
