elastic search: how to terminate a multi search query once we get desired result

Question

We have elastic search document with a string field named "Type". This field can have different values from "A" to "Z". Multiple documents can have the same type i.e. multiple documents can have Type "A"

We want to write an elastic search query that returns us maximum 30 of these documents. We want output to be in different group based on the type. For example:

1. If we have 10 document of type A, 15 of type B, 20 of type C, I should get all 10 with type A, all 15 with type B and 5 of type C.
2. If we have 0 document of type A, 10 of type B, 15 of type C, 20 of type D, I should get all 10 with type B, 15 of type C and 5 of type D.
3. Worst case: If we don't have any document of type A ... Y and 30 document of type Z, I should get 30 document of type Z.

I have written a very basic multi-search query for this (total 26 queries) i.e.

    POST _msearch/
    {"index":"<index_name>","type":"<type>"}
    {"from":0,"size":30,"query":{"bool":{"must":[{"terms":{"type":["A"]}}]}}}
    {"index":"<index_name>","type":"<type>"}
    {"from":0,"size":30,"query":{"bool":{"must":[{"terms":{"type":["B"]}}]}}}
    ...
    {"index":"<index_name>","type":"<type>"}
    {"from":0,"size":30,"query":{"bool":{"must":[{"terms":{"type":["Z"]}}]}}}

I am worried about multi-search query execution i.e. for case 1 and case 2, we got sufficient output i.e. 30 documents from first few queries, then why should we execute rest of the multi-search queries? Is there any way to stop multi-search query operation once we get desired number of result i.e. terminate multi search once we get 30 or more result.

Please note:

1. I have given very simple condition here i.e. the condition of different multi search is more complex than just on basis of type.
2. We want multiple collections in output i.e. type A, type B etc. all in different collections. (due to this constraint we had to rule out painless script option)

Answer 1

It looks like you can achieve what you want with a single search with size and a sort, plus optionally using bool to combine queries into one.

Can I terminate a Multi Search query prematurely?

No. From the documentation of Multi Search we can conclude this. It executes several search requests, with certain level of concurrency, and sends back the result only when all the queries have finished.

It is much like Bulk API, a way to easily execute parallel requests.

Can I get matching documents but in a custom defined order?

Yes, this is what sort is for. To achieve the behavior described in the original post it is literally enough to use this call:

POST /<index_name>/<index_type>/_search?sort=type:asc&size=30

Can I make one search request over several indices and still use sort?

Yes, you just need to define the list of indexes:

POST /multisearch1,mutlisearch2/<index_type>/_search?sort=type:asc&size=30

Or a wildcard expression:

POST /multisearch*/<index_type>/_search?sort=type:asc&size=30

Can I do sort in arbitrary order?

Yes, for example using Script Based Sorting. If, for instance, your wish to see the type in your results in this order: X, C, A, you could write a script like this:

POST /<index_name>/<type>/_search
{
  "size": 30,
  "sort": {
    "_script": {
      "type": "number",
      "script": {
        "lang": "painless",
        "source": """
int r = 1;
if(doc['type'].value  == 'X') { 
  r = 100;
} else if(doc['type'].value  == 'C') { 
  r = 10;
} else if(doc['type'].value  == 'A') { 
  r = 5;
}
  r;
"""
      },
      "order": "desc"
    }
  }
}

This will also work with multiple collections (like the query above).

Can I do this if I have a complex query that depends on the value of type?

Yes, no problem, use bool query for example:

POST /<index_name>/<type>/_search
{
  "query": {
    "bool": {
      "should": [
        {
          "bool": {
            "must": [
              {
                "term": {
                  "type": "A"
                }
              },
              {
                "match": {
                  "description": "Quick fox"
                }
              }
            ]
          }
        },
        {
          "bool": {
            "must": [
              {
                "term": {
                  "type": "X"
                }
              },
              {
                "match": {
                  "description": "Quick bear"
                }
              }
            ]
          }
        }
      ]
    }
  },
  "size": 30,
  "sort": {
    "_script": {
      "type": "number",
      "script": {
        "lang": "painless",
        "source": """
int r = 1;
if(doc['type'].value  == 'X') { 
  r = 100;
} else if(doc['type'].value  == 'C') { 
  r = 10;
} else if(doc['type'].value  == 'A') { 
  r = 5;
}
  r;
"""
      },
      "order": "desc"
    }
  }
}

Hope that helps!