How to map a Firestore date object to a date in elasticsearch

Question

I am using a cloud function to send a Firebase firestore document to elasticsearch for indexing. I am trying to find a way to map a firebase timestamp field to an elasticsearch date field in the index.

The elasticsearch date type mapping supports formats for epoch_millis and epoch_seconds but the firestore date type is an object as follows:

"timestamp": {
  "_seconds": 1551833330,
  "_nanoseconds": 300000000
},

I could use use the seconds field but will lose the fractional part of the second.

Is there a way map the timestamp object to a date field in the index that calculates the epoch_millis from the _seconds and _nanoseconds fields? I recognize that precision will be lost (nanos to millis).

I ended up adding another field before indexing the document. Not ideal but it works. — Duane, Jun 13 '19 at 00:46

score 3 · Accepted Answer · answered Aug 05 '20 at 18:23

If you don't mind losing the fractional part of the second, you could set a mapping on your index like this, which is what I ended up doing:

"mappings": {
      "date_detection": false,
      "dynamic_templates": [
        {
            "dates": {
              "match": ".*_seconds",
              "match_pattern": "regex",
              "mapping": {
                  "type": "date",
                  "format": "epoch_second"
              }
            }
        }
      ]
}

It will convert any timestamps (even nested in the document) to dates with second precision.

Wondering, why did you disable date_detection? Also what data type do you set in your mapping definitions? string or date? — Piotr, Oct 15 '21 at 11:40

How to map a Firestore date object to a date in elasticsearch

1 Answers1