I see from this question that document_id can easily be used in Logstash to replace a duplicate record in ElasticSearch. How would I add/increment a count value for e.g. repeating syslog messages? Instead of just replacing the record I want to increment the count so I know how many times it has repeated.
Asked
Active
Viewed 737 times
2 Answers
2
Depending on what you are using to view the data, it might just be as simple as looking at the _version field of the documents. ES will automatically update that value when something changes for the document. Kibana doesn't show the _version field (https://github.com/elasticsearch/kibana/issues/1234), but it's there.
Alcanzar
- 16,985
- 6
- 42
- 59
-
Thanks, this may do the trick. I will need to do some testing to verify that the version number starts at 1 and increments by exactly 1 each time (and not under any other conditions), but this is definitely a good start. A related question might be whether there is a method for telling ES to increment a field rather than specifying a value. Ultimately I'm trying to decide if ES can function well as an event database (in which case records might be updated by other criteria besides a repeating message), but that by itself might be too broad of a question to post? – cfiske Aug 14 '14 at 16:35
-
Elasticsearch can do [upserts](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-update.html), but I don't think you can do that through Logstash – Alcanzar Aug 14 '14 at 16:53
-
Aha, you beat me to it. :) But yeah, will have to investigate the Logstash angle. Thanks again! – cfiske Aug 14 '14 at 16:55
1
I think I have found what I need. An upsert will insert if a record doesn't exist, and update if it does. And enabling the _source field will allow incrementing of an existing field. That combination gives me the ability to start with a count of 1, and increment by 1 if the record already exists. Thanks @Alcanzar, your answer got my brain going in the right direction to find this.
cfiske
- 184
- 6