ElasticSearch / Kibana timestamp - field or _source?

Question

My single raw document in ES/Kibana looks like this:

{
  "_index": "fluentd.apache.access.20191202",
  "_type": "_doc",
  "_id": "8jql724B3OrCHx56JnWb",
  "_score": 1,
  "_source": {
    "serveraddress": "....",
    "clientaddress": "...",
    "@timestamp": "2019-12-02T11:00:30+0000",
    "method": "GET",
    "url": "....",
    "status": 200,
    "bytes": 19820,
    "referrer": "....",
    "agent": "...",
    "@log_name": "apache.access"
  },
  "fields": {
    "@timestamp": [
      "2019-12-02T11:00:30.000Z"
    ]
  }
}

Why do the @timestamps differ regarding their format?

2019-12-02T11:00:30+0000 (UTC timezone, no milliseconds)
2019-12-02T11:00:30.000Z (milliseconds + "Z" for UTC timezone)

What does my index mapping have to look like to have the time quickfilter available in Discover?

Currently it looks like this and it doesn't show me the timespan quickfilter:

...,
"@timestamp": {
  "type": "date",
  "format": "yyyy-MM-dd'T'HH:mm:ssZ"
},
...

Index pattern:

When you created your index pattern, which date field did you pick? That's the field that will be used by the time picker in the Discover view. — Val, Dec 10 '19 at 12:05
It doesn't look like you picked the `@timestamp` field when creating that index pattern otherwise there would be a small hour clock showing next to its name. See [here](https://www.elastic.co/guide/en/kibana/current/index-patterns.html#_manage_your_index_pattern) — Val, Dec 10 '19 at 12:10
@Val there is clock symbol in the Discover view and Kibana seems to understand the format: @ timestamp Dec 2, 2019 @ 12:00:30.000 — Daniel W., Dec 10 '19 at 12:15
No it just means it's a date field, but it's not THE date field — Val, Dec 10 '19 at 12:16
Go ahead, create a new index pattern that matches your fluentd indexes and pick the `@timestamp` field as THE time field and you'll see the difference — Val, Dec 10 '19 at 12:17
@Val Yea! That worked! I had a lot back and forth and recreating the index pattern solved the problem! Thanks for the hints! — Daniel W., Dec 10 '19 at 12:19

score 1 · Accepted Answer · answered Dec 10 '19 at 12:21

1

You need to recreate the index pattern and make sure to select the @timestamp field as THE time field for that index pattern.

That's the field Kibana uses for selecting documents based on the time selected int he time picker.

answered Dec 10 '19 at 12:21

Val

207,596
13
358
360

ElasticSearch / Kibana timestamp - field or _source?

1 Answers1