1

I am having some trouble understanding how the weighting is calculated in my implementation of Elasticsearch. It is my understanding that unless you use Dismax that the score of the document is the sum of all weights and not the score of the max field. Second the calculation are completely different from the documentation.

Looking at my provided query and explanation below I have three questions:

  1. How can the doc_count be different for both name and description for the same document?
  2. Why is the scoring done based on max field weight?
  3. If i have only 5 documents in the whole index that contain the search term why is the doc freq 6.

Thanks in advance.

Query

GET localhost_document/_search?explain=1&pretty=1&search_type=dfs_query_then_fetch

        {
          "query": {
            "multi_match" : {
              "query":    "lhc", 
              "fields": [  "Metadata.Name", "Metadata.Description^5" ] 
            }
          }
        }

Explaination

"_explanation": {
               "value": 28.427635,
               "description": "max of:",
               "details": [
                  {
                     "value": 28.427635,
                     "description": "weight(Metadata.Description:lhc in 0) [PerFieldSimilarity], result of:",
                     "details": [
                        {
                           "value": 28.427635,
                           "description": "score(doc=0,freq=1.0 = termFreq=1.0\n), product of:",
                           "details": [
                              {
                                 "value": 5,
                                 "description": "boost",
                                 "details": []
                              },
                              {
                                 "value": 5.3759904,
                                 "description": "idf, computed as log(1 + (docCount - docFreq + 0.5) / (docFreq + 0.5)) from:",
                                 "details": [
                                    {
                                       "value": 6,
                                       "description": "docFreq",
                                       "details": []
                                    },
                                    {
                                       "value": 1404,
                                       "description": "docCount",
                                       "details": []
                                    }
                                 ]
                              },
                              {
                                 "value": 1.0575776,
                                 "description": "tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 - b + b * fieldLength / avgFieldLength)) from:",
                                 "details": [
                                    {
                                       "value": 1,
                                       "description": "termFreq=1.0",
                                       "details": []
                                    },
                                    {
                                       "value": 1.2,
                                       "description": "parameter k1",
                                       "details": []
                                    },
                                    {
                                       "value": 0.75,
                                       "description": "parameter b",
                                       "details": []
                                    },
                                    {
                                       "value": 2.9529915,
                                       "description": "avgFieldLength",
                                       "details": []
                                    },
                                    {
                                       "value": 2.56,
                                       "description": "fieldLength",
                                       "details": []
                                    }
                                 ]
                              }
                           ]
                        }
                     ]
                  },
                  {
                     "value": 4.2207813,
                     "description": "weight(Metadata.Name:lhc in 0) [PerFieldSimilarity], result of:",
                     "details": [
                        {
                           "value": 4.2207813,
                           "description": "score(doc=0,freq=1.0 = termFreq=1.0\n), product of:",
                           "details": [
                              {
                                 "value": 5.7578497,
                                 "description": "idf, computed as log(1 + (docCount - docFreq + 0.5) / (docFreq + 0.5)) from:",
                                 "details": [
                                    {
                                       "value": 16,
                                       "description": "docFreq",
                                       "details": []
                                    },
                                    {
                                       "value": 5224,
                                       "description": "docCount",
                                       "details": []
                                    }
                                 ]
                              },
                              {
                                 "value": 0.7330482,
                                 "description": "tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 - b + b * fieldLength / avgFieldLength)) from:",
                                 "details": [
                                    {
                                       "value": 1,
                                       "description": "termFreq=1.0",
                                       "details": []
                                    },
                                    {
                                       "value": 1.2,
                                       "description": "parameter k1",
                                       "details": []
                                    },
                                    {
                                       "value": 0.75,
                                       "description": "parameter b",
                                       "details": []
                                    },
                                    {
                                       "value": 2.1161945,
                                       "description": "avgFieldLength",
                                       "details": []
                                    },
                                    {
                                       "value": 4,
                                       "description": "fieldLength",
                                       "details": []
                                    }
                                 ]
                              }
                           ]
                        }
                     ]
                  }
               ]
            }
         }

0 Answers0