Elasticsearch query data removing continuous duplicate data

Question

i have a problem.

I would like to get a list of data but has some same values that i would like to get rid of.

This is a list of a alerts turned on and off, some times i get a "falseTick" and store this data, i need to make a history of this but i can't just get the amount of true values becouse maybe some of this "true" is a "falseTick".

This is the example of data that i index into elastic:

[
  { id: "asdfghjkl", value: true, date: July 24th 2020, 12:18:38 pm },
  { id: "asdfghjkl", value: true, date: July 24th 2020, 12:18:37 pm },
  { id: "asdfghjkl", value: false, date: July 24th 2020, 12:18:36 pm },
  { id: "asdfghjkl", value: true, date: July 24th 2020, 12:18:35 pm },
  { id: "asdfghjkl", value: false, date: July 24th 2020, 12:18:34 pm },
  { id: "asdfghjkl", value: false, date: July 24th 2020, 12:18:33 pm },
]

and this is what i would like to get

[
  { id: "asdfghjkl", value: true, date: July 24th 2020, 12:18:38 pm },
  { id: "asdfghjkl", value: false, date: July 24th 2020, 12:18:36 pm },
  { id: "asdfghjkl", value: true, date: July 24th 2020, 12:18:35 pm },
  { id: "asdfghjkl", value: false, date: July 24th 2020, 12:18:34 pm }
]

Real Tick Example:

Sensor connect and record the current value "false" > 
addElastIndex({ id: "asdfghjkl", value: false, date: July 24th 2020, 12:18:38 pm }) > 
Sensor detect a change "true" > 
addElastIndex({ id: "asdfghjkl", value: true, date: July 24th 2020, 12:18:38 pm })

False Tick Example:

Sensor connect and record the current value "true" > 
addElastIndex({ id: "asdfghjkl", value: true, date: July 24th 2020, 12:18:38 pm }) > 
Sensor dissconnects >
Sensor connect and record the current value "true" > 
addElastIndex({ id: "asdfghjkl", value: true, date: July 24th 2020, 12:18:38 pm })   # This value should be considered a falseTick becouse wasn't a real True, becouse wasn't detected by the sensor

What's the pattern here? What do you consider a `falseTick` again? — Joe - GMapsBook.com, Jul 24 '20 at 13:44
Got it. So the next incoming value must be the opposite of the latest previous value? — Joe - GMapsBook.com, Jul 24 '20 at 14:27
Also, do you wish to get the list _after_ you've ingested the docs or do you want to prevent them from indexing in the first place? — Joe - GMapsBook.com, Jul 24 '20 at 14:34
Yes, i would like to make a query that looks for turn on off pattern, simple as make a history of when the sensor has detected something. :D.I can prevent that with my sensor app, but not allways, becouse some times i will program this things and others someone will inject their data — Josep Soler, Jul 24 '20 at 14:36

Elasticsearch query data removing continuous duplicate data

0 Answers0