Python's official low-level client for Elasticsearch
Questions tagged [elasticsearch-py]
194 questions
4
votes
1 answer
Elasticsearch-py Index takes 4 arguments, 5 given?
I have the following call to the Elasticsearch-py client:
esClient.index(index=data['AppName'], id=data['RequestId'], body=data)
I get the following error when I run my code:
Traceback (most recent call last):
File…
danielschnoll
- 3,045
- 5
- 23
- 34
4
votes
4 answers
ElasticSearch Scroll API with multi threading
First of all, I want to let you guys know that I know the basic work logic of how ElasticSearch Scroll API works. To use Scroll API, first, we need to call search method with some scroll value like 1m, then it will return a _scroll_id that will be…
A l w a y s S u n n y
- 36,497
- 8
- 60
- 103
4
votes
1 answer
How set ignore_malformed in index level when creating an index through ElasticSearch DSL python wrapper?
According to docs, this should be sufficient:
"settings": {
"index.mapping.ignore_malformed": true
}
But how can I achieve this on python wrapper? My current code looks like this:
from elasticsearch_dsl import Index
index =…
Montenegrodr
- 1,597
- 1
- 16
- 30
4
votes
2 answers
Elasticsearch group by and distinct
I have a bunch of documents with the fields username and device_os as follows:
{ "username": "foo", "device_os": "Android", ....},
{ "username": "foo", "device_os": "iOS", ....},
{ "username": "bar", "device_os": "Android", ....},
{ "username":…
Vineet Goel
- 2,138
- 1
- 22
- 28
4
votes
1 answer
Elasticsearch delay in store and search immediately
I am using elasticsearch with python. and use dsl driver in python.
My script is as below.
import time
from elasticsearch_dsl import DocType, String
from elasticsearch import exceptions as es_exceptions
from elasticsearch_dsl.connections import…
Nilesh
- 20,521
- 16
- 92
- 148
4
votes
0 answers
Python query to Elasticsearch returns 0 hits
I'm trying to query elastic search for all the logs between two milliseconds timestamps.
When querying Elasticsearch directly via:
curl -XPOST ' -d ' {"query" : { "bool" : { "must" : { "range" : { "received_at" : { "gte" :…
Alex
- 151
- 13
4
votes
1 answer
sync/async insert or update ElasticSearch in Python
I'm using ElasticSearch bulk Python API, Does it provide both sync and Async api?
Jack
- 5,540
- 13
- 65
- 113
3
votes
2 answers
Connecting to AWS Elasticsearch within VPC from Lambda over HTTPS
I have a Lambda written in Python which writes some data to Elasticsearch hosted on AWS. The ES service is within a VPC, so I'm trying to use the internal DNS of the ES to connect to it. This is my code:
es_client = Elasticsearch(
…
drunkenfist
- 2,958
- 12
- 39
- 73
3
votes
1 answer
Elasticsearch how to query on ID field higher than x
I am trying to apply pagination to results by querying multiple times to get past the 10k barrier of Elasticsearch. Since the results of Elasticsearch can differ during multiple queries I want to use the generated ID to get the next results.
So for…
JasperFennet
- 53
- 10
3
votes
1 answer
elasticsearch python client - work with many nodes - how to work with sniffer
i have one cluster with 2 nodes.
i am trying to understand the best practise to connect the nodes, and check failover when there is downtime on one node.
from documentation:
es = Elasticsearch(
['esnode1', 'esnode2'],
# sniff before doing…
Eyal Ch
- 9,552
- 5
- 44
- 54
3
votes
1 answer
AuthorizationException: on Scan
I am trying to run what seems like a very simple Scan but I'm running into an odd exception. It seems like the generator created by the scan hits an exception and I have no idea why. I tried wrapping the interior of my for loop in a try/except but…
Obj3ctiv3_C_88
- 1,478
- 1
- 17
- 29
3
votes
2 answers
How to insert new or update old document using elasticsearch-py?
What is the most elegant way to insert a new document (if not already exists) or update (increase counter by 1) of an already existed document?
This one:
res = elasticsearch.update(
index='stories-test',
doc_type='news',
…
user706838
- 5,132
- 14
- 54
- 78
3
votes
0 answers
Python elasticsearch client returns true when type does not exists
I am trying to delete the mapping of a type but before doing that I am checking if the type exists as in the code below:
def delete_mapping(self, doc_type):
if self.elasticsearch.indices.exists_type(index='my_index', doc_type=doc_type):
…
Cris Pinto
- 293
- 3
- 18
2
votes
0 answers
OutOfMemoryError when bulk index large data into Elasticsearch
I'm trying to index a large amount of data into Elasticsearch. Data comes from a CSV file which is around 41 GB and contains around 100 million rows. I'm using Elasticsearch Python client for this task. The code looks more or less like this:
es =…
Triet Doan
- 11,455
- 8
- 36
- 69
2
votes
0 answers
Getting error while loading data into aws elasticsearch : RequestError(400, 'no handler found for uri [/:443/movies/_doc/5] and method [PUT]'
I'm trying to load the data into AWS elasticsearch using python
Here is what I'm doing
I have connected to AWS auth and Elasticsearch service using the elasticsearch module and used the elasticsearch endpoint as host.
here is the code:
awsauth =…
ankur suman
- 151
- 1
- 3
- 10