How to return only a single field in a response ElasticSearch - Python?

Question

I am connecting to an AWS elasticsearch server and trying to fetch only one field in the response. In addition, I am not aware of the format but I am sure it works with the normal curl / postman requests.

The code :

    import collections
    from time import sleep

    import requests

    import simplejson as json
    from aws_requests_auth.aws_auth import AWSRequestsAuth
    from elasticsearch import Elasticsearch, RequestsHttpConnection, helpers
    from pyelasticsearch import ElasticSearch, bulk_chunks

    es = ElasticSearch(urls='http://'+es_host,
                      port=80)

    auth = AWSRequestsAuth(aws_access_key='XXXXXXXX',
                   aws_secret_access_key='XXXXXXXXX',
                   aws_host=es_host,
                   aws_region='us-west-2',
                   aws_service='es')

           es_client = Elasticsearch(host=es_host,
                          port=80,

        connection_class=RequestsHttpConnection,
                          http_auth=auth)

      print (es_client.info())
      print (es_client.count(index='X_data'))
      result = es_client.search(index='X_data', body={"terms":{'field':"pid"}})
      print (result)

This gives below format error. Any changes to fetch only PID field as results?

File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/base.py", line 105, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.RequestError: TransportError(400, u'parsing_exception')

score 1 · Answer 1 · edited Jul 11 '19 at 10:11

1

Use this:

client.search('X_data', _source=['pid'])

edited Jul 11 '19 at 10:11

jkdev

11,360
15
54
77

answered Jul 11 '19 at 08:43

DennisLi

3,915
6
30
66

Ryan Widmaier · Answer 2 · 2018-10-15T19:11:11.943

-1

You are missing an outer level in your body. Specifically for "query". It should look like this:

result = es_client.search(index='X_data', body={"query": {"term":{'field':"pid"}}})

edited Oct 15 '18 at 19:11

answered Oct 15 '18 at 18:57

Ryan Widmaier

7,948
2
30
32

That would still give me : File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/base.py", line 105, in _raise_error raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info) elasticsearch.exceptions.RequestError: TransportError(400, u'parsing_exception') – Viv Oct 15 '18 at 18:59
1

Sorry, typo. Try `term` query instead of `terms`. `terms` expects a list of values. Answer is updated. – Ryan Widmaier Oct 15 '18 at 19:11
Thats not helping, it only gets shard values (shards': {u'successful': 5, u'failed': 0, u'total': 5}) – Viv Oct 15 '18 at 19:19
I would argue it did help since it fixed your original issue.. I'm not aware of a way to not get the hits key in the response. What version of ES? Also what does your data look like? – Ryan Widmaier Oct 15 '18 at 19:29
Also.. the shards key starts with an underscore, so it looks like you are truncating string data maybe? Maybe something more basic with your terminal or program output? Full result print should look like `{u'hits': {u'hits': [], u'total': 0, u'max_score': None}, u'_shards': {u'successful': 5, u'failed': 0, u'skipped': 0, u'total': 5}, u'took': 0, u'timed_out': False}` – Ryan Widmaier Oct 15 '18 at 19:31
It has an underscore... just pasted some part of the string.. I am not getting any output though {u'hits': {u'hits': [], u'total': 0, u'max_score': None}, u'_shards': {u'successful': 5, u'failed': 0, u'total': 5}, u'took': 1, u'timed_out': False} – Viv Oct 15 '18 at 19:53
Probably there is an issue with your data, or your mapping. I would confirm the field you are searching on is keyword type. If that doesn't fix your problem, you should probably open a separate question. – Ryan Widmaier Oct 15 '18 at 19:58
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/181905/discussion-between-viv-and-ryan-widmaier). – Viv Oct 15 '18 at 20:27

How to return only a single field in a response ElasticSearch - Python?

2 Answers2