C# Elastic aggregate query with size and date range limit

Question

I have the following query which groups by a primary key mainjobid and then groups by a supplied time interval. I would like to restrict the aggregation and hits to a time range supplied by two datetime objects. Setting the Size limits the number of hits but I cannot figure out how to do the same thing for aggregates and how to set a global date range for both?

SearchResponse<LogLine> response = await _elasticsearchClient.SearchAsync<LogLine>(r =>
{
    r.QueryLuceneSyntax(request);
    r.Aggregations(arr =>
    {
        arr.Terms("group_by_mainjobid", i =>
        {
            i.Field(p => p.Fields.MainJobId);
            i.Aggregations(agg =>
            {
                agg.DateHistogram("group_by_time", e =>
                {
                    e.Field(p => p.TimeStamp).CalendarInterval(interval);
                });
            });
        });
    });
    r.Index(Indices.Parse("log-index-*"));
    r.Size(512);
});

Update: I was able to restrict the aggregates into a singular date range but the hits are unaffected. I tried to update the query string to include the timestamp range but that's not working.

SearchResponse<LogLine> response = await _elasticsearchClient.SearchAsync<LogLine>(r =>
{
    r.QueryLuceneSyntax(request);
    r.Aggregations(arr =>
    {
        arr.DateRange("range", darr =>
        {
            darr.Field(p => p.TimeStamp);
            darr.Ranges(desc =>
            {
                desc.From(new FieldDateMath(DateMath.FromString(from.ToString("o", CultureInfo.InvariantCulture))));
                desc.To(new FieldDateMath(DateMath.FromString(to.ToString("o", CultureInfo.InvariantCulture))));
            });
            darr.Aggregations(rarr =>
            {
                rarr.Terms("group_by_mainjobid", i =>
                {
                    i.Field(p => p.Fields.MainJobId);
                    i.Aggregations(agg =>
                    {
                        agg.DateHistogram("group_by_time", e =>
                        {
                            e.Field(p => p.TimeStamp).CalendarInterval(interval);
                        });
                    });
                });
            });
        });
    });
    r.Index(Indices.Parse("log-index-*"));
    r.Size(logSize);
});

score 1 · Accepted Answer · answered Sep 01 '22 at 13:33

In c# for me to make a query to elastic between two dates and have both aggregations and hits respect it using the Lucene query language was to include the following snippet in my query string. No other changes were needed.

$"@timestamp:[\"{from:o}\" TO \"{to:o}\"]"

score 0 · Answer 2 · answered Aug 23 '22 at 12:58

0

You've used 2 aggregations but to restrict the overall results "globally" like you mentioned you should use a query before the aggregations.

Use the q.DateRange(...) query.

You have example in the docs: https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/date-range-query-usage.html

Basically it's using the "range" query in ES.

Time and Date mappings in ES is format based so please note that your range needs the correct format (Done when you are mapping your types). Be aware of automatic mapping, I usually do my mapping manually.

More here: https://www.elastic.co/guide/en/elasticsearch/reference/2.0/mapping-date-format.html#built-in-date-formats

Docs are your friend :)

answered Aug 23 '22 at 12:58

SimplyCode

318
2
9

The DateRange function I can use seems very different from the documented one. Let me see if I can figure this out. – BillHaggerty Aug 23 '22 at 13:18
Ok, looks like im half way there. My aggregations all seem to be restricted to this time range but the hits are not. – BillHaggerty Aug 23 '22 at 13:44
@BillHaggerty You need to restrict it *before* the aggregation and not inside the aggregation – SimplyCode Aug 25 '22 at 11:21

C# Elastic aggregate query with size and date range limit

2 Answers2