Script writing to get Distinct value from Elasticsearch

Question

I want to create a custom distinct function where I can write simple distinctBy script of multiple field names to be distinct together. So is there anyway in ElasticSearch to achieve this.

Answer 1

What I did, is made use of Terms Aggregation using Script to construct keys from three different fields and then apply Terms Aggregation on this concatenated field to give what you want.

I've created a sample index with 3 fields, (field1, field2 and field3 of type keyword) with documents as below. You can check the query and the results part to see how they appear. Key point is the keys in the result part are distinct.

Sample Documents

POST myfieldindex/mydocs/1
{
  "field1": "Football",
  "field2": "Premier League",
  "field3": "Chelsea"
}

POST myfieldindex/mydocs/3
{
  "field1": "Football",
  "field2": "Premier League",
  "field3": "Liverpool"
}

POST myfieldindex/mydocs/3
{
  "field1": "Football",
  "field2": "Premier League",
  "field3": "ManCity"
}

Query

POST myfieldindex/_search
{  
   "size":0,
   "aggs":{  
      "myagg":{  
         "terms":{  
            "script":{  
               "source":"doc['field1'].value + params.param + doc['field2'].value + params.param + doc['field3'].value",
               "lang":"painless",
               "params":{  
                  "param":","
               }
            }
         }
      }
   }
}

Query Results

{
  "took": 0,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 3,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "myagg": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": [
        {
          "key": "Football,Premier League,Chelsea",
          "doc_count": 1
        },
        {
          "key": "Football,Premier League,Liverpool",
          "doc_count": 1
        },
        {
          "key": "Football,Premier League,ManCity",
          "doc_count": 1
        }
      ]
    }
  }
}

So you can see in the results as how key are constructed (keys are unique).

Let me know if this helps!