For questions about the Elasticsearch bulk API.
For more information, see here: https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html
Questions tagged [elasticsearch-bulk-api]
76 questions
0
votes
0 answers
Uploading a CSV file in sql using SP_OAMethod procedures to do a BULK sync with Salesforce using BULK API Version 2
My query is that I have a CSV file with data in the format as accepted by Salesforce and require to do a BULK sync using BULKApi Version 2. I now need to send this data from sql using SP_OAMethod procedures.
Tried to move this data into a DB table…
0
votes
1 answer
A mapper_parsing_exception occurred when using the bulk API of Elasticsearch
Elasticsearch version: 8.3.3
Indexing was performed using the following Elasticsearch API.
curl -X POST "localhost:9200/bulk_meta/_doc/_bulk?pretty" -H 'Content-Type: application/json' -d'
{"index": { "_id": "1"}}
{"mydoc": "index action, id 1…
john_smith
- 3
- 3
0
votes
1 answer
Unable to use _bulk API on Elasticsearch on Windows
I am new to windows . I was working my entire on linux. Can somebody help me here for _bulk api indexing
C:\ELK\master-1\elasticsearch\sample_data>curl -k -H 'Content-Type: application/x-ndjson' -X POST 'https://localhost:9200/bank/_bluk?pretty' -d…
Tushar singh
- 1
- 1
0
votes
1 answer
remove an item using bulk update
I have an object with different values in Elasticsearch. When I want to update it, I use bulk api like below:
inventory_edition.append(
{
'_op_type': 'update',
'_index': value_index,
'_id': object['inventory.id'],
'_source': {'doc':…
Shokouh Dareshiri
- 826
- 1
- 12
- 24
0
votes
1 answer
Efficient Bulk Loading Options for Elasticsearch in Python
I am trying to ingest a large amount of data into Elasticsearch using Python. For this purpose, I am using the bulk API helper and I have developed a function that looks something like this,
def __load(self, docs, index):
try:
# begin…
Minura Punchihewa
- 1,498
- 1
- 12
- 35
0
votes
1 answer
How can I write multiple updates in one BulkAll method in ElasticSearch NEST 7.13.2
Using ElasticSearch NEST .Net package 7.13.2 in Visual Studio 2019
For a list of products I am currently updating existing documents in my product index by using the following code:
var productIndex = "productindex";
foreach (var product in…
pdw
- 5
- 2
0
votes
1 answer
how Elasticsearch bulk API works with threadpool
While trying to index around 20000 records (each record has around 100kb size) using python's bulk API, I saw that max write queue size consumed is 6. So are my following understanding about bulk indexing correct (with default configuration of 500…
Divakar Patil
- 323
- 2
- 11
0
votes
1 answer
Elasticsearch multiple JSON insert bulk
I am trying to insert multiple JSON documents in Elastic search. I have done with the single document as the following curl sample
curl --request POST \
--url 'http://localhost:9200/articles/_doc/?pretty=' \
--header 'Content-Type:…
Munish Kapoor
- 3,141
- 2
- 26
- 41
0
votes
1 answer
getting [ERROR] 'The [dims] property must be specified for field [vector].' despite being set in mapping
I am trying to upload dense vectors to Elasticsearch endpoint.
created index with mapping as below:
mapping = {
"mappings": {
"properties" : {
"vector": {
"type": "dense_vector",
…
Vinay Sharma
- 319
- 1
- 5
- 13
0
votes
1 answer
How to use bulk operator in elasticsearch
I am using the elasticsearch client.
var elasticsearch = require('elasticsearch');
var client = new elasticsearch.Client({
host:"https://********",
log: 'trace',
});
I have JSON objects lets say
[{"name":"abc", "age": 23},{"name":"bcd",…
Rahul Saini
- 927
- 1
- 11
- 28
0
votes
1 answer
Upload new line JSON to Elasticsearch bulk API
I'm trying to upload a new line JSON to Elasticsearch using the Bulk API. The bulk JSON I'm uploading looks like this, with each JSON on a new line:
{"ip": "x.x.x.x", "seen": true, "classification": "malicious", "spoofable": false, "first_seen":…
ipconfuse
- 37
- 1
- 6
0
votes
0 answers
why index only one doc takes so long time
There is a scheduler task (every ten minutes) that synchronize changed data from Oracle to ElasticSearch, but occasionally there are SocketTimeOutException
java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection http-outgoing-113…
zhuguowei
- 8,401
- 16
- 70
- 106
0
votes
1 answer
Index json files in bulk elasticsearch 5.6
I have a folder with around 590,035 json files. Each file is a document that has to be indexed. If I index each document using python then it is taking more than 30 hours. How do I index these documents quickly?
Note - I've seen bulk api but that…
user10005853
0
votes
1 answer
Post request for bulk API is giving status code of 406, How to resolve it?
I am using Elastic search 6.1 version
My data is appending correctly and I am adding '\n' at the end of the request.
My code is as follows:
def insert_in_bulk(self, filee, rtype):
U = urljoin(self.args.host, "/_bulk")
body = []
f =…
Naga Venkatesh Gavini
- 151
- 3
- 15
0
votes
1 answer
How to insert an element into already present list in elastic search
Say I have documents stored like below.
document 1
{
id : '1',
title : "This is a test document1",
list : ["value1" , "value2"],
...
}
document 2
{
id : '2',
title : "This is a test…
Melvin Ruban
- 87
- 6