Stack Overflow
Stack Overflow

Questions tagged [amazon-elasticsearch]

Amazon Elasticsearch Service is a managed Elasticsearch offering as part of Amazon Web Services.

248 questions
0
votes
1 answer

what is a good indexing strategy when dealing with AWS/CloudWatch logs?

I am new to Elasticsearch world and I'm working on a project to use Amazon Elasticsearch service (Elasticsearch and Kibana) to provide a log analytics system for all the CloudWatch logs from different AWS accounts. Setting up the stack and routing…
Nisman
  • 1,271
  • 2
  • 26
  • 56
0
votes
1 answer

How to create rolling index with date as the index name?

My elastic search index will ingest thousands of documents per second. The service which puts documents in the index doesn't creates a new index, instead it just gets current data in nodejs and indexes docs in "log-YYYY.MM.DD". So, we know index is…
Broken Phone
  • 41
  • 5
0
votes
2 answers

Encountered a retryable error. Will Retry with exponential backoff 413

Logstash keep encountering following error message that logs cannot be sent to AWS ElasticSearch. [2021-04-28T16:01:28,253][ERROR][logstash.outputs.amazonelasticsearch] Encountered a retryable error. Will Retry with exponential…
PPShein
  • 13,309
  • 42
  • 142
  • 227
0
votes
1 answer

Updated AWS Elasticsearch storage, but "Minimum free storage space" also increased

I'm a complete newbie with AWS and I'm having issues with AWS Elasticsearch. We have a large Magento 2 catalog with hundreds of thousands of products (although we only index product name and SKU data). Elasticsearch constantly reboots when indexing…
Kalvin Klien
  • 911
  • 1
  • 12
  • 32
0
votes
1 answer

AWS Quicksight Fine-grained access control document level security on Elasticsearch data

I have integrated AWS elasticsearch into AWS QuickSight, but we are attempting to only allow certain users to see certain data. For example Users in group A would only be able to see data for records in the elasticsearch index that have…
user3311613
  • 147
  • 12
0
votes
1 answer

Which one is better among DynamoDB and AWS ElasticSearchService for querying and storing logs?

I'm building a GUI tool for querying logs and looking for a cheaper option. DDB will fetch logs from an S3 bucket using lambda whereas ES will get the same logs streamed from CloudWatch. The thing is my queries are gonna be simple, not complex ones…
Broken Phone
  • 41
  • 5
0
votes
1 answer

How many instances do I need for Amazon Elasticsearch Service?

If I need a 3-node cluster do I need to have 3 instances? Or are they created in same instance? https://i.stack.imgur.com/4oRAI.png
kirk mads
  • 13
  • 4
0
votes
1 answer

Elasticsearch Snapshot Failing in AWS, preventing upgrade

My incremental Snapshots in Elasticsearch are now failing. I didn't touch anything, nothing seems to have changed, can't figure out what is wrong. I checked my Snapshots by doing: GET _cat/snapshots/cs-automated?v&s=id and finding the details of a…
Derrops
  • 7,651
  • 5
  • 30
  • 60
0
votes
1 answer

Roll up availability on AWS Elasticsearch Service?

Open Distro for Elasticsearch has Rollup implemented. But this is missing on the AWS Elasticsearch service. I do really need this feature. Is there anyway to implement this feature or does anybody have any solution which they are currently doing in…
Derrops
  • 7,651
  • 5
  • 30
  • 60
0
votes
1 answer

Where am I going wrong with Elasticsearch _delete_by_query?

I'm fairly new to Elasticsearch and trying to periodically delete documents using the _delete_by_query API (I fully appreciate I should probably be using time based indices to make this easier, and will be updating the indexing structure in due…
pr.lwd
  • 140
  • 10
0
votes
1 answer

Opendistro for Elasticsearch - ISM is not triggering jobs

I am trying to use ISM with opendistro on AWS Elasticsearch Service v7.8. I am trying to set up a basic rollover/delete policy and it seems that the policy is not triggering. I assume I am doing something wrong, but I can't seem to track it down...…
pallen
  • 3
  • 1
  • 6
0
votes
2 answers

AWS Elasticsearch publishing wrong total request metric

We have an AWS Elasticsearch cluster setup. However, our Error rate alarm goes off at regular intervals. The way we are trying to calculate our error rate is: ((sum(4xx) + sum(5xx))/sum(ElasticsearchRequests)) * 100 However, if you look at the…
c2tarun
  • 776
  • 2
  • 9
  • 27
0
votes
1 answer

Elasticsearch: Alternative to using filtered aliases

I'm trying to figure out possible solution to the following problem. In my current implementation I'm relying on so called filtered aliases where I can create a view of the same index for each user e.g.: POST /aliases { "actions": "add": { "index":…
rilush
  • 1
  • 1
0
votes
0 answers

Is there a native way to stream logs from Cloudwatch to an Elastic Cloud index

Like you can select a loggroup, and then stream this to an AWS ES trivially easy by selecting the loggroup and selecting Create Elasticsearch Subscription Filter can you do the same thing but to an Elasticsearch domain in Elastic Cloud? I know you…
Derrops
  • 7,651
  • 5
  • 30
  • 60
0
votes
0 answers

AWS-SDK Node JS High Latency

I am facing problem while using aws-sdk amazons elasticsearch in my node app. I am getting high latency when having performance testing using some tools like jmeter or seige. My requests initially show low latency but when the number of requests…
Nayeem Ur Rehman
  • 39
  • 1
  • 7
1 2 3
…
16 17