3

I am trying to populate dashboard in kibana with Elasticsearch data on date fields . I have log file with dates and i find that i don't have @timestamp in it.

Here is mapping :

PUT test2
{
  "settings": {
    "index.mapping.ignore_malformed": true 
  },
  "mappings": {
    "my_type": {
      "properties": {
    "Size": {"type": "integer","ignore_malformed": true },
    "Copy Size": {"type": "integer","ignore_malformed": true }
    "Email Sent Time": {"type": "date"},
    "Creation Time": {"type": "date"},
    "Modification Time": {"type": "date"}
      }
    }
  }
}

How to add default timestamp? To create area chart in kibana.

Nusrath
  • 499
  • 1
  • 4
  • 16
  • Does this answer your question? [How to make elasticsearch add the timestamp field to every document in all indices?](https://stackoverflow.com/questions/17136138/how-to-make-elasticsearch-add-the-timestamp-field-to-every-document-in-all-indic) – Boaz Nov 20 '22 at 15:34

1 Answers1

8

Once upon a time, Elasticsearch used to support adding default timestamps automatically to all documents you put in an index. The mapping was something like this when creating your index:

"mappings" : {
    "_default_":{
        "_timestamp" : {
            "enabled" : true,
            "store" : true
        }
    }
}

However, as you can see at this link, it was deprecated by version 5.x. Today, it is recommended to populate a regular date field with the current timestamp on application side.

kahveci
  • 1,429
  • 9
  • 23
  • do you have idea to convert "28-09-2016 12:21:06" to "2015-06-18T09:03:25.877Z" this format in elastic search. – Nusrath Dec 22 '18 at 07:17
  • How do you read data from the log file you mentioned and insert it to Elasticsearch? Do you use Filebeat or Logstash? – kahveci Dec 22 '18 at 11:00
  • I have a CSV file with python I am inserting data into ES – Nusrath Dec 23 '18 at 09:03
  • 1
    In this case, I am afraid, you cannot make any trick on Elastic Stack side. However, you can still handle it on application side easily. Basically, all you need to do is to convert your date type fields into a supported ISO 8601 date and time format (e.g. "yyyy-MM-dd'T'HH:mm:ss.SSSZZ") and then pass it to Elasticsearch. Please refer to this document for all default ISO formats supported: https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-date-format.html – kahveci Dec 23 '18 at 22:11