1

When the logs appear to have the same timestamps, some of it appears out of order. Reading through some of the topics, I couldn't find a suitable solution. But I believe there must be something done about it since this issue was reported as early as Logstash 1.

Basically, I am using logstash to listen on a certain port for incoming tcp using the json lines codec [using the logstash-logback-encoder here]. For now, the filter is empty and I'm outputting the data to elasticsearch and stdout (codec rubydebug).

The console log in my IDE:

2017-Aug-30 15:15:30.191 [main] INFO  com.sbsatter.logbackLogstash.App - Testing LOG Order;
Expected Order: 1 => 10 
2017-Aug-30 15:15:30.193 [main] INFO  com.sbsatter.logbackLogstash.App - 1 
2017-Aug-30 15:15:30.194 [main] INFO  com.sbsatter.logbackLogstash.App - 2 
...
2017-Aug-30 15:15:30.195 [main] INFO  com.sbsatter.logbackLogstash.App - 9 
2017-Aug-30 15:15:30.195 [main] INFO  com.sbsatter.logbackLogstash.App - 10

The terminal which launched logstash reads accordingly:

{
    "@timestamp" => 2017-08-30T09:15:30.197Z,
          "port" => 47820,
      "@version" => "1",
          "host" => "127.0.0.1",
          "time" => "2017-08-30 15:15:30.191+0600",
       "message" => "Testing LOG Order;\nExpected Order: 1 => 10"
}
{
    "@timestamp" => 2017-08-30T09:15:30.198Z,
          "port" => 47820,
      "@version" => "1",
          "host" => "127.0.0.1",
          "time" => "2017-08-30 15:15:30.193+0600",
       "message" => "1"
}
{
    "@timestamp" => 2017-08-30T09:15:30.198Z,
          "port" => 47820,
      "@version" => "1",
          "host" => "127.0.0.1",
          "time" => "2017-08-30 15:15:30.194+0600",
       "message" => "2"
}
.....
{
    "@timestamp" => 2017-08-30T09:15:30.216Z,
          "port" => 47820,
      "@version" => "1",
          "host" => "127.0.0.1",
          "time" => "2017-08-30 15:15:30.195+0600",
       "message" => "9"
}
{
    "@timestamp" => 2017-08-30T09:15:30.224Z,
          "port" => 47820,
      "@version" => "1",
          "host" => "127.0.0.1",
          "time" => "2017-08-30 15:15:30.195+0600",
       "message" => "10"
}

BUT, kibana shows the following :frowning: : result in kibana

While the difference is not great in here, however, when used with real-time logs, changed order makes the log appear nonsense. How can I fix this issue?

Note, I have asked this question in the elasticsearch forum as well. I've dived through the documentation to find anything close to this but to no avail.

sbsatter
  • 591
  • 3
  • 22
  • Would you share the link of your cross-posted duplicate here, and post something over there too to point here, so that future respondents do not replicate something that someone has already posted? – halfer Aug 30 '17 at 12:26
  • 1
    hi @halfer, I have done this a long time ago. Do you know of any solution? – sbsatter Sep 06 '17 at 05:03
  • Did you ever figure out anything with this? – David Jun 16 '22 at 22:21
  • Hi @David, the best I can remember is I think using the default timestamp field helped, although I'm not sure if this was the case with logstash-logback-encoder. I didn't find this module too helpful so I think I moved away to another measure. – sbsatter Jun 20 '22 at 19:39

0 Answers0