Elasticsearch Reindex remote from 1.x to 5.x _timestamp

Question

There has been a post about this, but there were no responses. I have an old index in ES 1.x, that I want to migrate to 5.x. The source documents have the _timestamp field enabled. I am using the remote-reindex api and need to migrate the _timestamp field. Have not been able to do this. I have tried using ingest pipelines as in the following

PUT _ingest/pipeline/p1
{
  "processors": [
      {
        "set" : {
          "field" : "new_timestamp",
          "value" : "{{_source._ingest.timestamp}}"
        }
      }
    ]
}

Then I run the reindex as

  POST _reindex
{
  "size": 1,
  "source": {
    "remote": {
      "host": "http://xxx:9200"

    },
    "index": "twitter2"
  },
  "dest": {
    "index": "twitter2",
    "pipeline": "p1"
  }
}

I have also tried to use inline scripts as

POST _reindex
{
  "source": {
    "remote": {
      "host": "http://xxx:9200"

    },
    "index": "twitter2"
  },
  "dest": {
    "index": "twitter2"
  },
  "script": {
    "lang": "painless",
    "inline": "ctx._source.new_timestamp=ctx._source.timestamp"
  }
}

But both these presented null fields in the results. Has anyone been able to solve this please?

There is a similar question with an answer here: https://stackoverflow.com/questions/43077970/reindex-data-from-elasticsearch-1-x-to-elastichsearch-with-old-timestamp-to-new/43084221?noredirect=1#comment75669766_43084221 — Val, Jun 03 '17 at 05:45
The answer on that thread did not work for the poster. Also, I have tried some other options in trying to make it work, and described them in the above. — user2689782, Jun 05 '17 at 13:54
Do you have a field in your document that could be equal or close to the `_timestamp` field ? — Val, Jun 05 '17 at 14:08
You mean in my source document? Unfortunately no. I just don't know why this does not work. I am following documentation — user2689782, Jun 05 '17 at 15:59
One idea could be to first modify your 1.5 index by adding the `_timestamp` field as a regular field. I've provided an answer here: https://stackoverflow.com/questions/44379396/elasticsearch-1-x-add-field-copy-of-timestamp — Val, Jun 06 '17 at 05:48
OK, I think I have a reason why it was not working. The source document had timestamp enabled, but not stored. Not totally sure if this is the only reason. — user2689782, Jun 06 '17 at 22:26

Elasticsearch Reindex remote from 1.x to 5.x _timestamp

0 Answers0