2

How can I do Bucket Sort in composite Aggregation?

I need to do Composite Aggregation with Bucket sort.

I have tried Sort with aggregation. I have tried composite aggregation.

Sanker
  • 69
  • 1
  • 7

1 Answers1

2

I think this question, is in continuation to your previous question, so considered the same use case

You need to use Bucket sort aggregation that is a parent pipeline aggregation which sorts the buckets of its parent multi-bucket aggregation. And please refer to this documentation on composite aggregation to know more about this.

Adding a working example with index data, mapping, search query, and search result

Index Mapping:

{
  "mappings":{
    "properties":{
      "user":{
        "type":"keyword"
      },
      "date":{
        "type":"date"
      }
    }
  }
}

Index Data:

{
  "date": "2015-01-01",
  "user": "user1"
}
{
  "date": "2014-01-01",
  "user": "user2"
}
{
  "date": "2015-01-11",
  "user": "user3"
}

Search Query:

The size parameter can be set to define how many composite buckets should be returned. Each composite bucket is considered as a single bucket, so setting a size of 10 will return the first 10 composite buckets created from the values source. The response contains the values for each composite bucket in an array containing the values extracted from each value source. Defaults to 10.

{
  "size": 0,
  "aggs": {
    "my_buckets": {
      "composite": {
       "size": 3,               <-- note this
        "sources": [
          {
            "product": {
              "terms": {
                "field": "user"
              }
            }
          }
        ]
      },
      "aggs": {
        "mySort": {
          "bucket_sort": {
            "sort": [
              {
                "sort_user": {
                  "order": "desc"
                }
              }
            ]
          }
        },
        "sort_user": {
          "min": {
            "field": "date"
          }
        }
      }
    }
  }
}

Search Result:

"aggregations": {
    "my_buckets": {
      "after_key": {
        "product": "user3"
      },
      "buckets": [
        {
          "key": {
            "product": "user3"
          },
          "doc_count": 1,
          "sort_user": {
            "value": 1.4209344E12,
            "value_as_string": "2015-01-11T00:00:00.000Z"
          }
        },
        {
          "key": {
            "product": "user1"
          },
          "doc_count": 1,
          "sort_user": {
            "value": 1.4200704E12,
            "value_as_string": "2015-01-01T00:00:00.000Z"
          }
        },
        {
          "key": {
            "product": "user2"
          },
          "doc_count": 1,
          "sort_user": {
            "value": 1.3885344E12,
            "value_as_string": "2014-01-01T00:00:00.000Z"
          }
        }
      ]
    }
ESCoder
  • 15,431
  • 2
  • 19
  • 42
  • so... what if I have more user than the size value (by default size value in 10 right )? I need to sort the entire aggregation!!! – Sanker Oct 29 '20 at 13:11
  • @Sanker you can add the `size` parameter. Please go through my updated part of the question. – ESCoder Oct 29 '20 at 13:24
  • I know about the size parameter. that's ok. so What I need is: Aggregate all the data in Elasticsearch and sort them by data of max @timestamp. at this stage basically I need entire data in ES aggregated and sorted so I can get the latest 1000 bucket from the aggregation and next 1000 and so on.... – Sanker Oct 29 '20 at 13:43
  • do you have any kind of solution for this?? – Sanker Oct 29 '20 at 13:43
  • @Sanker As a simple workaround, if you need 1000 buckets you can set the `size` parameter to that value. If your use case is different from this, I would recommend you to ask a separate question with this modified use case (so that the community can help you). Mixing different use cases in the same question becomes difficult to answer. And, if my answer helped you, it would be great if you can upvote and accept it TIA :) – ESCoder Oct 29 '20 at 13:48
  • @Sanker any update and luck here? Would be great if you can upvote and accept my answer if it helped you :) – ESCoder Oct 31 '20 at 04:52