1

I am calling elasticsearch data using eland. The documentation is simple and I am able to implement it, but when searching the index it search using es_index_pattern which is basically a wildcard index-string.

from elasticsearch import ElasticSearch
import eland as ed

es = Elasticsearch(hosts="myhost", "port":0000)

search_body={
    "bool":{
            "filter":[
                {"exists": {"field": "customer_name"}},
                {"match_phrase": {"city": "chicago"}},
                ]
        }

    }

# I am able to get the results if I search the index through "elasticsearch" api. Tried this repetitively and it works every time
results = es.search(index="my_index", body=search_body)

# But, I do not get results (but ReadTimeoutError) if I connect to 'my_index' index via localhost Elasticsearch node using Eland
df = ed.DataFrame(es_client=es, es_index_pattern = 'my_index')

I have to hand type the error message becasue I cannot copy the error outside the environment I am using. Also, my host and port would be different

...
  File ".../elasticsearch/transport.py", line 458, in perform_request
    raise e
  File "......elasticsearch/transport.py", line 419, in perform_request
  File "..... /elasticsearch/connection/http_urllib3.py", line 275, in perform_request
    raise ConnectionTimeout("TIMEOUT", str(e), e)
elasticsearch.exceptions.ConnectionTimeout: ConnctionTimeout caused by - ReadTimeoutError(HTTPSConnectionPool(host=myhost', port=0000): Read timed out. (read timeout=10)

I think that elasticsearch is able to get results bc it's calling the exact index name and hence not running into timedout. But, Eland is rather using es_index_pattern thereby using my_index as wildcard i.e *my_index, therefore I must be running into ReadTimeOutError.

I looked inside the source code to see if there was anything I could do, so it did not search the index as a pattern but exact match. But, I see no option for searching the exact index both in the documentation and source code.

How do I search for exact index string in Eland?

Sources:

everestial007
  • 6,665
  • 7
  • 32
  • 72

1 Answers1

0

Regardless of whether you use a wildcard or not in your index pattern the way Eland communicates with Elasticsearch is identical. So that leads me to think that your requests are timing out either due to connectivity or the request being too large to process in time on Elasticsearch (do you have a complete traceback of your timeout error?)

Perhaps increasing your request_timeout value on the Elasticsearch client constructor would help here?

(Disclaimer: I'm the maintainer of Eland and work for Elastic)

sethmlarson
  • 923
  • 8
  • 21
  • Yes, the request is too large to process in time. And that is bc the index I am searching is being searched as wild card. If is search for the same index through pyelasticsearch (instead of eland) I am getting the results. But, as soon as I use eland the `ReadTimeoutError` kicks in bc/ the index say `abc` is being searched as `*abc*` – everestial007 Jan 12 '22 at 16:08
  • I just update the question with more details. Please let me know what are my options. – everestial007 Jan 12 '22 at 16:26
  • Do you have any update on this question? Though its not a good practice but I posted a question again; b/c question not answered on the first day are not generally answered. Please look https://stackoverflow.com/questions/70685986/how-to-search-exact-index-not-index-pattern-in-elasticsearch-eland-updated – everestial007 Jan 14 '22 at 15:18