This works for me correctly:
search = Search(using=client, index='my_index').script_fields(
special={'script': {
'source': "<a formula that produces a double>"}
}
)
And now I want to attach a sum the field special across all hits. I tried this:
search = Search(using=client, index='my_index').script_fields(
special={'script': {
'source': "<a formula that produces a double>"}
}
).aggs.metric('total', 'sum', field='special')
But when I check the aggs attibute of the response, I get
{'total': {'value': 0.0}}
The response itself tells me that special is populated correctly for each entry, and it's always positive. It looks a bit to me like special might not be around at the time the sum occurs, so that elasticsearch doesn't see it and so it makes a sum of 0. I experimented with pipeline in place of metric but that didn't change things. What I'm proposing should be possible, right?
I could do the sum in my script rather than through elasticsearch, but I'm betting the sum on the ES side will be much faster.
Update
Based on this discussion at elasticsearch forum I think I found the way it's supposed to work, but it still isn't working properly for me. You provide the script inside the aggregation like this:
s=Search(using=client, index='my_index').aggs.pipeline('total', 'sum', script={'source': "<the special formula>"})
The problem now is that the output is not correct. On a test index with one item of a given time, I tried to report that field in the sum aggregation, and instead of the expected result (the value of the field) I got something that was different by 128.
Update
I received an answer to this last problem in the elasticsearch help forum: https://discuss.elastic.co/t/script-aggregation-yields-wrong-but-close-answer/133744/3?u=rschwieb
In a nutshell, the data stored in the field was interpreted as a double by elasticsearch, but the aggregation used the type determined by the mapping for the index (which said 'float' instead). After re-indexing the field as a double, the numbers are as expected.
