For questions about the Elasticsearch bulk API.
For more information, see here: https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html
Questions tagged [elasticsearch-bulk-api]
76 questions
1
vote
0 answers
Elasticsearch HTTP 500 Error {"message":null}
I am encountering error from _bulk endpoint but I can't solve since the error message sent by ES is null. I am calling the _bulk endpoint via ES REST API. This only happens intermittently.
I already tried replicating it via load testing but I still…
samanthamita
- 83
- 2
- 6
1
vote
0 answers
ElasticSearch illegal_argument_exception for client.bulk() Node.js
I try to send data to ES and I was using POST request to send data line by line and it worked fine with small files.
When I changed to use client.bulk(), I keep getting the following error message:
"status": 400,
"error":
{
"type":…
Yen Wu
- 11
- 2
1
vote
2 answers
How to handle errors with bulk requests
I am using Elasticsearch bulk API to send a lot of documents to index and delete at once. If there is an error for one document, other documents will be indexed or deleted successfully. And this leads to wrong state of data in elasticstore because…
Rohanil
- 1,717
- 5
- 22
- 47
1
vote
2 answers
How to bulk change JSON format for Elastic Search
I have the below format and was hoping to pre-process in bulk using elasticsearch.
{"title":"April","url":"https://simple.wikipedia.org/wiki/April", "abstract":"April is the 4th month of the year, and comes between March and May. It is one of four…
ESLearner
- 87
- 1
- 14
1
vote
0 answers
How to get "related documents" in a query in Elasticearch?
I have a problem in elasticsearch i am not being able to solve:
I have two different types in an index, one called 'keywords' and another one called 'products'.
Keywords have the string property 'keyword_content', and products have the text…
JavierAnselmi
- 23
- 5
1
vote
1 answer
How to insert an already created json-format string to Elasticsearch Bulk
In a python script,
I'm trying for elasticsearch.helpers.bulk to store multiple records.
I will get a json-format string from another software, and I want to attach it in the source part
I got the helpers.bulk format by this answer
part of my…
haeny
- 363
- 2
- 5
- 15
1
vote
1 answer
How can I force NEST to NOT populate request on Bulk API response?
I've been looking all over the place and haven't been able to find a suitable answer to this question. I've created a NEST client using this code:
var myIndex = "myTestIndex";
var myType = "myTestType";
var myClusterUri=…
Sgt_zippy
- 66
- 5
1
vote
1 answer
How to create an index with integer fields in Elasticsearch for the JSON file of format?
I am trying to create an index in Elasticsearch for the JSON file of format:
{ "index" : { "_index" : "entity", "_type" : "type1", "_id" : "0" } }
{ "eid":"guid of Event autogenerated", "entityInfo": { "entityType":"qualityevent", …
Mrunal Pagnis
- 801
- 1
- 9
- 26
1
vote
1 answer
elasticsearch bulk queue size
I'm monitoring the elasticsearch cluster while bulk importing data.
So when I run curl localhost:9200/_cat/thread_pool?v
it shows:
host ip bulk.active bulk.queue bulk.rejected index.active index.queue index.rejected…
milodky
- 443
- 2
- 7
- 18
1
vote
0 answers
ElasticSearch Load rate Degrades Over Time
I'm using Java 1.7 and the ES bulk API with the java client to load json documents to and embedded Elasticsearch 1.7.1 instance.
Over the many hours this takes to run, I notice a gradual degradation in the rate of documents loaded per minute. I'm…
Phil
- 141
- 2
- 7
1
vote
1 answer
ElasticSearch, use of Wildcard on index names while using bulk api
I'm using RabbitMQ River Plugin for Elasticsearch to Insert/Delete data to our ElasticSearch indexes. River plugin only supports Bulk API.
Our current data is using multiples indexes, an index by date, something like “indexName_yyyymmdd”
When…
jdiaz4517
- 269
- 1
- 3
- 8
1
vote
1 answer
Get all documents from an index of an elasticsearch cluster and index it in another elasticsearch cluster
My goal here is to get all documents from an index of an ES cluster and insert them in another ES cluster keeping the same metadata.
I had a look at mget API to retrieve data and Bulk API to insert it but this Bulk API needs a special…
Romolive
- 13
- 3
1
vote
1 answer
Bulk loading JSON object as document into elasticsearch
Is there a way to bulk load the data below into elasticsearch without modifying the original content? I POST each object to be a single document. At the moment I'm using Python to parse through individual objects and POST them one at a time.
{
…
codeBarer
- 2,238
- 7
- 44
- 75
0
votes
1 answer
Is there a way to identify when the elastic _bulk API created a new index vs when it operated on an existing index?
I am using the Elasticsearch _bulk api to insert some data into the index.
However, what I observed is that, if an index does not exist, this API will first create an index with dynamic mappings and then index the document.
Is there a way to find…
asten_ark
- 47
- 4
0
votes
0 answers
OpenSearch bulk query vs AWS Kinesis Firehose OpenSearch destination
Premise: I already noticed this similar question, but it doesn't exactly cover what I would like to understand.
Problem
As part of a project to build a scalable and reliable search solution, I am exploring different ways to bulk load data into…
ciurlaro
- 742
- 10
- 22