How can i get the ALL lastest record with each group by Elasticsearch query?

Question

I have reference from this how-to-get-latest-values-for-each-group-with-an-elasticsearch-query

and now i do the search, but the aggregations only return 10 doc for me, how can it show all match result? I ONLY show two since its too long for the return repsonse , thanks!

my ES query is :

{
    "size" :1,
    "aggs": {
    "group": {
        "terms": {
            "field": "studentId"
        },
        "aggs": {
            "group_docs": {
                "top_hits": {
                    "size": 1,
                    "sort": [
                        {
                            "timestamp": {
                                "order": "desc"
                            }
                        }
                    ]
                }
            }
        }
    }
  }
}

and the result:

{
    "took": 32,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 10000,
            "relation": "gte"
        },
        "max_score": 1.0,
        "hits": [
            {
                "_index": "data",
                "_type": "class",
                "_id": "N-wsrHYB4zCrGLTdS7Ur",
                "_score": 1.0,
                "_source": {
                    "studentId": 144,
                    "timestampstring": "2020-09-02 05:58:04.828",
                    "type": "data"
                }
            }
        ]
    },
    "aggregations": {
        "group": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 99670,
            "buckets": [
                {
                    "key": 131,
                    "doc_count": 579,
                    "group_docs": {
                        "hits": {
                            "total": {
                                "value": 579,
                                "relation": "eq"
                            },
                            "max_score": null,
                            "hits": [
                                   {
                                    "_index": "data",
                                    "_type": "class",
                                    "_id": "SVVj4HYBlaUrIoJst3-o",
                                    "_score": null,
                                    "_source": {
                                        "studentId": 131,
                                        "timestampstring": "2021-01-08 13:06:34.413",
                                        "type": "data"
                                    },
                                    "sort": [
                                        1609340059767
                                    ]
                                }
                            ]
                        }
                    }
                },
                {
                    "key": 147,
                    "doc_count": 529,
                    "group_docs": {
                        "hits": {
                            "total": {
                                "value": 529,
                                "relation": "eq"
                            },
                            "max_score": null,
                            "hits": [
                                {
                                    "_index": "data",
                                    "_type": "class",
                                    "_id": "SVVj4HYBlaUrIoJst3-o",
                                    "_score": null,
                                    "_source": {
                                        "studentId": 147,
                                        "timestampstring": "2021-01-08 13:06:34.413",
                                        "type": "data"
                                    },
                                    "sort": [
                                        1610082394413
                                    ]
                                }
                            ]
                        }
                    }
                }
            ]
        }
    }
}

ESCoder · Accepted Answer · 2021-01-08T11:15:09.643

You need to add the size param in the terms aggregation

The size parameter can be set to define how many term buckets should be returned out of the overall terms list.

{
  "size": 1,
  "aggs": {
    "group": {
      "terms": {
        "field": "studentId",
        "size": 100                // note this
      },
      "aggs": {
        "group_docs": {
          "top_hits": {
            "size": 1,
            "sort": [
              {
                "timestamp": {
                  "order": "desc"
                }
              }
            ]
          }
        }
      }
    }
  }
}

Update 1:

You can use stats bucket aggregation, to get the count of unique studenid

{
  "size": 1,
  "aggs": {
    "group": {
      "terms": {
        "field": "studentId",
        "size": 100 // note this
      },
      "aggs": {
        "group_docs": {
          "top_hits": {
            "size": 1,
            "sort": [
              {
                "timestamp": {
                  "order": "desc"
                }
              }
            ]
          }
        }
      }
    },
    "bucketcount": {
      "stats_bucket": {
        "buckets_path": "group._count"
      }
    }
  }
}

thanks you! can i also ask how to count how can i count how many "unique" studentId can i get in this query? @ESCoder — Man Man Yu, Jan 08 '21 at 11:08
@ManManYu please go through my updated answer, and let me know if this resolves your issue ? — ESCoder, Jan 08 '21 at 11:15
yes but i need to wait three minute to press the "Tick": sign ,,,, =P — Man Man Yu, Jan 08 '21 at 11:18

How can i get the ALL lastest record with each group by Elasticsearch query?

1 Answers1