How to calculate total for each token in Elasticsearch

Question

I have a request into Elastic

{  
   "query":{  
      "bool":{  
         "must":[  
            {  
               "query_string":{  
                  "query":"something1 OR something2 OR something3",
                  "default_operator":"OR"
               }
            }
         ],
         "filter":{  
            "range":{  
               "time":{  
                  "gte":date
               }
            }
         }
      }
   }
}

I wanna calculate count for each token in all documents using elastic search in one request, for example:

something1: 26 documents
something2: 12 documents
something3: 1 documents

Answer 1

Assuming that the tokens are not akin to enumerations (i.e. constrained set of specific values, like state names, which would make a terms aggregation your best bet with the right mapping), I think the closest thing to what you want would be to use filters aggregation:

POST your-index/_search
{
  "query":{  
    "bool":{  
      "must":[  
      {  
        "query_string":{  
          "query":"something1 OR something2 OR something3",
          "default_operator":"OR"
         }
      }
      ],
      "filter":{  
        "range":{  
          "time":{  
            "gte":date
          }
        }
      }
    }
  },
  "aggs": {
    "token_doc_counts": {
      "filters" : {
        "filters" : {
          "something1" : { 
            "bool": { 
              "must": { "query_string" : { "query" : "something1" } }, 
              "filter": { "range": { "time": { "gte": date } } } 
            }
          },
          "something2" : { 
            "bool": { 
              "must": { "query_string" : { "query" : "something2" } }, 
              "filter": { "range": { "time": { "gte": date } } } 
            }
          },
          "something3" : { 
            "bool": { 
              "must": { "query_string" : { "query" : "something3" } }, 
              "filter": { "range": { "time": { "gte": date } } } 
            }
          }
        }
      }
    } 
  }
}

The response would look something like:

{
  "took": 9,
  "timed_out": false,
  "_shards": ...,
  "hits": ...,
  "aggregations": {
    "token_doc_counts": {
      "buckets": {
        "something1": {
          "doc_count": 1
        },
        "something2": {
          "doc_count": 2
        },
        "something3": {
          "doc_count": 3
        } 
      } 
    } 
  }
}

Answer 2

You can split your query into filters aggregation of three filters. For reference look here: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-filters-aggregation.html

Answer 3

What you would need to do, is to create a Copy_To field and have the mapping as shown below.

Depending on the fields that your query_string queries, you need to include some or all of the fields with copy_to field.

By default query_string searches all the fields, so you may need to specify copy_to for all the fields as shown in below mapping, where for sake of simplicity, I've created only three fields, title, field_2 and a third field content which would act as copied to field.

Mapping

PUT <your_index_name>
{
  "mappings": {
    "mydocs": {
      "properties": {
        "title": {
          "type": "text",
          "copy_to": "content" 
        },
        "field_2": {
          "type": "text",
          "copy_to": "content" 
        },
        "content": {
          "type": "text",
          "fielddata": true
        }
      }
    }
  }
}

Sample Documents

POST <your_index_name>/mydocs/1
{
  "title": "something1",
  "field_2": "something2"
}

POST <your_index_name>/mydocs/2
{
  "title": "something2",
  "field_2": "something3"
}

Query:

You'd get the required document counts for the each and every token using the below aggregation query and I've made use of Terms Aggregation:

POST <your_index_name>/_search
{
  "size": 0,
  "query": {
    "query_string": {
      "query": "something1 OR something2 OR something3"
    }
  },
  "aggs": {
    "myaggs": {
      "terms": {
        "field": "content",
        "include" : ["something1","something2","something3"]
      }
    }
  }
}

Query Response:

{
  "took": 7,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "myaggs": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": [
        {
          "key": "something2",
          "doc_count": 2
        },
        {
          "key": "something1",
          "doc_count": 1
        },
        {
          "key": "something3",
          "doc_count": 1
        }
      ]
    }
  }
}

Let me know if it helps!