1

The situation is I load the df into es, the df has two columns:'url' & 'text'.

And I query the 'text' with values in a list which named 'forbidden_words'.

I wanna make the res can also present two columns, one is "url", one is "forbidden_words" which are searched in the text.

But the below code present that '_source' doesn't show anything...

If there is any help, would be many thanks!

for i in forbidden_words:
    dsll = {
       'query': {
           'match': {
               'text': i
               }
           },
       "_source": {
           "includes": forbidden_words,
           # "excludes": []
           }
       }
res = es.search(index='test', body=dsll)

The result of res:

    {'took': 25,
 'timed_out': False,
 '_shards': {'total': 5, 'successful': 5, 'skipped': 0, 'failed': 0},
 'hits': {'total': 26,
  'max_score': 3.211111,
  'hits': [{'_index': 'test',
    '_type': 'test',
    '_id': 'ml5utHcBcazm5fCndKUY',
    '_score': 3.211111,
    '_source': {}}, ....
   {'_index': 'test',
    '_type': 'test',
    '_id': 'oV5utHcBcazm5fCndKUY',
    '_score': 1.2800283,
    '_source': {}}]}}
zoe
  • 11
  • 2

2 Answers2

0

_source expects field names, not field values. So you could only say:

{
  "query": {
    "match": {
      "text": "xyz"
    }
  },
  "_source": {
    "includes": ["text", "url"]    <--
  }
}

If you only want to return the actual words that matched your query, take a look at highlighting:

{
  "query": {
    "match": {
      "text": "xyz"
    }
  },
  "_source": {
    "includes": ["text", "url"]
  },
  "highlight": {
    "fields": {
      "text": {}
    }
  }
}

Note that the highlighted values will not be inside of the _source anymore but inside highlight.

Joe - GMapsBook.com
  • 15,787
  • 4
  • 23
  • 68
  • I found out that the highlighted values would be sentences, but I wanna export forbidden_words which r queried in the text. So I used the for loop to do that. I'm appreciated for your help. All the best:) – zoe Feb 20 '21 at 08:38
  • You're welcome! If the above helped you, upvote it. If it solved your problem, accept the answer :) – Joe - GMapsBook.com Feb 20 '21 at 08:53
0
result = []
for i in forbidden_words:
    dsl = {   
       'query': {
           'match_phrase': {  
               'text': i
               }
           }
       }
    res = es.search(index='cn_web_crawler_test_linux', body=dsl, size=sizee)
    for j in res["hits"]["hits"]:
        if len(j) > 0:
            append_list = (i, j['_source']['url'], j['_source']['text'])
        result.append(append_list)
zoe
  • 11
  • 2