How to make calls to elasticsearch apis through NodeJS?

Question

I have been tasked with making a POST api call to elastic search api,

https://search-test-search-fqa4l6ubylznt7is4d5yxlmbxy.us-west-2.es.amazonaws.com/klove-ddb/recipe/_search

I don't have any previous experience with making api calls to AWS services.

So, I tried this -

axios.post('https://search-test-search-fqa4l6ubylznt7is4d5yxlmbxy.us-west-2.es.amazonaws.com/klove-ddb/recipe/_search')
            .then(res => res.data)
            .then(res => console.log(res));

But I was getting {"Message":"User: anonymous is not authorized to perform: es:ESHttpPost"}

I also checked out with some IAM roles and added AWSESFullAccess policies to my profile.

Still I can't make anything work out.

Please help me.

You'll need to add the execution role ARN of the lambda function to the ElasticSearch cluster access policy to allow `es:eshttppost` — Tom Nijs, Jun 21 '18 at 17:16
There is an execution role : es:* I think that covers all? Or should I specifically add one es:eshttppost — thedreamsaver, Jun 21 '18 at 17:55

score 7 · Accepted Answer · edited May 20 '19 at 15:05

The reason your seeing the error User: anonymous is not authorized to perform: es:ESHttpPost is because you're making requesting data without letting ElasticSearch know who you are - this is why it says 'Anonymous'.

There are a couple ways of authentication, the easiest being using the elasticsearch library. With this library you'll give the library a set of credentials (access key, secret key) to the IAM role / user. It will use this to create signed requests. Signed requests will let AWS know who's actually making the request, so it won't be received as anonymous, but rather, yourself.

Another way of getting this to work is to adjust your access policy to be IP-based:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "es:*",
            "Condition": {
                "IpAddress": {
                    "aws:SourceIp": [
                        "AAA.BBB.CCC.DDD"
                    ]
                }
            },
            "Resource": "YOUR_ELASTICSEARCH_CLUSTER_ARN"
        }
    ]
}

This particular policy will be wide open for anyone with the ip(range) that you provide here. It will spare you the hassle of having to go through signing your requests though.

A library that helps setting up elasticsearch-js with AWS ES is this one

A working example is the following:

const AWS = require('aws-sdk')
const elasticsearch = require('elasticsearch')
const awsHttpClient = require('http-aws-es')

let client = elasticsearch.Client({
    host: '<YOUR_ES_CLUSTER_ID>.<YOUR_ES_REGION>.es.amazonaws.com',
    connectionClass: awsHttpClient,
    amazonES: {
        region: '<YOUR_ES_REGION>',
        credentials: new AWS.Credentials('<YOUR_ACCESS_KEY>', '<YOUR_SECRET_KEY>')
    }
});

client.search({
    index: 'twitter',
    type: 'tweets',
    body: {
        query: {
            match: {
                body: 'elasticsearch'
            }
        }
    }
})
.then(res => console.log(res));

I have my IP address whitelisted. Still not knowing the issue. Can you share any link to examples of making an api call to elasticsearch. I cannot get these to work - https://github.com/aws-samples/amazon-elasticsearch-lambda-samples — thedreamsaver, Jun 22 '18 at 15:41
Have you tried the samples of the elasticsearch-js library from the link I provided? — Tom Nijs, Jun 23 '18 at 16:08
@thedreamsaver I've added a code sample of how to use the library. — Tom Nijs, Jun 23 '18 at 16:17
Yes, I finally got it working :D Thank you so much Tom Nijs. — thedreamsaver, Jun 25 '18 at 06:27

score 1 · Answer 2 · answered Oct 31 '20 at 13:58

The Elasticsearch npm package is going to be deprecated soon, use @elastic/elasticsearch and @acuris/aws-es-connection so you don't have to provide IAM Credentails to the function.

Here the code, I use:

'use strict';

const { Client } = require('@elastic/elasticsearch');
const { createAWSConnection, awsGetCredentials } = require('@acuris/aws-es- 
connection');

module.exports.get_es_interests = async event => {
const awsCredentials = await awsGetCredentials();
const AWSConnection = createAWSConnection(awsCredentials);
const client = new Client({
...AWSConnection,
node: 'your-endpoint',
});

let bodyObj = {};
try {
    bodyObj = JSON.parse(event.body);
} catch (jsonError) {
    console.log('There was an error parsing the JSON Object', jsonError);
    return {
        statusCode: 400
    };
}

let keyword = bodyObj.keyword;

const { body } = await client.search({
index: 'index-name',
body: {
  query: {
    match: {
      name: {
        query: keyword,
        analyzer: "standard"
      }
    }
  }
}
});

var result = body.hits.hits;

return result;
};

score 0 · Answer 3 · answered May 19 '21 at 22:57

Now there's https://github.com/gosquared/aws-elasticsearch-js

Import them in

const AWS = require('aws-sdk');
const ElasticSearch = require('@elastic/elasticsearch');
const { createConnector } = require('aws-elasticsearch-js');

Configure client using named profile that can be found on ~/.aws/config. You can verify this by doing: cat ~/.aws/config which should output something like:

[profile work]
region=ap-southeast-2

[default]
region = ap-southeast-1

const esClient = new ElasticSearch.Client({
  nodes: [
    '<aws elastic search domain here>'
  ],
  Connection: createConnector({
    region: '<region>',
    getCreds: callback =>
      callback(
        null,
        new AWS.SharedIniFileCredentials({ profile: '<target profile>' })
      )
  })
});

Then you can start using it like:

// this query will delete all documents in an index
await esClient.delete_by_query({
  index: '<your index here>',
  body: {
    query: {
      match_all: {}
    }
  }
});

References:

How to make calls to elasticsearch apis through NodeJS?

3 Answers3

Linked