0

I am trying to generate the summary from some data available in Elasticsearch using Logstash in which i have fields like loan_account_no., customer_id, batch, rank, channel and many more. I am creating the primary key (_id) by the combining (loan_account_no, customer_id and batch) and using fingerprint to hash it. so its generating unique records in the index. So here in the data, "channel" is categorical variable which has 3 categories(''voicebot,'smarttext','email').In the summary i am generating the new fields based on the channel variable. The fields are as follows:-

1) voicebot_rank, 2) smarttext_rank, 3) email_rank,

and i am trying to update these fields in such a way :

(i) Assume i already have a data present for the particular(_id) including the new fields in summary index for the channel 'voicebot' and voicebot_rank is 5. (ii) What if the next time the data comes for the the same (_id) with the rank 6 for the channel 'voicebot' and i am trying to update rank by comparing the record present in the elasticsearch by new record through logstash. (iii) Trying same for the other fields too.

NOTE:- Input and output both are elasticsearch.

So here i have to compare the ranks of same (_id) and if existing rank is smaller than the previous existing rank then have to update the rank. current have to up

Shishir Dubey
  • 1
  • 2
  • Have you looked in the [`elasticsearch`](https://www.elastic.co/guide/en/logstash/current/plugins-filters-elasticsearch.html) filter on logstash? You would need to query elasticsearch to get the values of the previous document and see if you need to update it or not. – leandrojmp Dec 06 '19 at 14:41
  • Hey @leandrojump, Thanks for your response. could you please let me know how to use it. As my input index is different and my output index is different. and i want to compare the incoming data with output index and then update. – Shishir Dubey Dec 07 '19 at 05:51

0 Answers0