Can I get a count for each of a list of domains?

Question

I am trying to do a terms filter query to get the number of documents I have for each domain in a list of domains:

GET /myindex/_count
{
   "query": {
      "filtered": {
         "filter": {
            "terms": {
                 "domain": ["w3.org"]
            }
         }
      }
   }
}

Returns 25. I have a list of a couple thousand domains and would like to do this all in 1 query. Is that possible? I've tried:

GET /myindex/_count
{
  "query": {
    "bool": {
      "must": {
        "terms": {
          "domain": [
            "w3.org",
            "google.com",
            ...,
          ]
        }
      }
    }
  }
}

but that gives me 1 number (whereas I need them broken down by each domain), e.g.:

w3.org: 25,
google.com: 143,
...

Answer 1

query does not return count for each matching term found, it will show you how many matching docs are found, basically hits in elasticsearch terminology. To get the numbers with each term you found you have to use aggregations, more data available here.

For your particular case you have to use Terms Aggregation, more data available here.

Your query will look like this

GET /myindex/_search
{
  "query": {
    "bool": {
      "must": {
        "terms": {
          "domain": [
            "w3.org",
            "google.com"
          ]
        }
      }
    }
  },
  "aggregations":{
    "domain_count":{
      "terms":{
        "field": "domain"
      }
    }
  }
}

The response would look something like this, where doc_count and key inside buckets is your required result.

{
    ...

    "aggregations" : {
        "domain_count" : {
            "doc_count_error_upper_bound" : 46,
            "buckets" : [
                {
                    "key" : "w3.org",
                    "doc_count" : 100
                },
                {
                    "key" : "google.com",
                    "doc_count" : 52
                },
                ...
            ]
        }
    }
}

Make sure you are using _search end-point instead of _count.

If you don't want to limit your query for particular domains like w3.org or google.com, you can give a match_all query. It will give you all possible domain values with their doc_count.

GET /myindex/_search
{
  "query": {
    "bool": {
      "must": {
        "match_all": {
        }
      }
    }
  },
  "aggregations":{
    "domain_count":{
      "terms":{
        "field": "domain"
      }
    }
  }
}