How to _reindex elasticsearch data to new mapping (from flat fields to objects)?

Question

I have an old index (elasticsearch index) has more than 20K objects, this index has fields

{
    "title": "Test title",
    "title_ar": "عنوان تجريبي",
    "body": "<p>......</p>"
}

I want to _reindex them to convert all data to new mapping like this

{
    "title_1": {
        "en": "Test title",
        "ar": "عنوان تجريبي"
    },
    "body": "<p>......</p>"
}

What is the best elasticsearch pipeline processor to make this conversion available in _reindex API?

You will not be able to do that without recreating your index, because the `title` field already exists as `text` and you cannot change it to be an `object` — Val, Jun 26 '19 at 14:33
I create a new index with new mapping, I can name it `title_1` if it is not possible. the most important thing is converting it to object — Abdallah Sabri, Jun 26 '19 at 14:43
Then if you create a rand new index, it's definitely possible and easy to do — Val, Jun 26 '19 at 14:44

score 2 · Answer 1 · answered Jun 26 '19 at 14:47

2

I suggest to simply use the reindex API to do this:

POST _reindex
{
  "source": {
    "index": "old_index"
  },
  "dest": {
    "index": "new_index"
  },
  "script": {
    "source": "ctx._source.title = [ 'en' : ctx._source.title, 'ar': ctx._source.title_ar]",
    "lang": "painless"
  }
}

If in your old_index index you have this:

{
    "title": "Test title",
    "title_ar": "عنوان تجريبي",
    "body": "<p>......</p>"
}

In your new index, you'll have this:

{
    "title": {
        "en": "Test title",
        "ar": "عنوان تجريبي"
    },
    "body": "<p>......</p>"
}

answered Jun 26 '19 at 14:47

Val

207,596
13
358
360

I get the following error ` "script_stack": [ "ctx._source.title = [ 'en' : ctx._source.title, 'ar': ctx._source.title_ar]", " ^---- HERE" ],` – Abdallah Sabri Jun 26 '19 at 14:51
there is no `.title` field in `ctx._source.title`, but it is a mandatory field – Abdallah Sabri Jun 26 '19 at 14:55
Not sure I understand your second comment – Val Jun 26 '19 at 14:57
I get `ScriptException[runtime error]; nested: NullPointerException;` on the `title_1` in the following statement `"source": "ctx._source.title_1 = ....` – Abdallah Sabri Jun 26 '19 at 15:06
You don't need to create the `title_1` field since you're creating a new index you can keep `title` – Val Jun 26 '19 at 15:08
this is fantastic. +1 – JBone Jun 26 '19 at 17:40
@Val, if I keep it `title` or `title_1` I still get the same error `ScriptException[runtime error]; nested: NullPointerException;`, the error is `_source` is null. I cannot get any field inside it – Abdallah Sabri Jun 29 '19 at 05:30
Please update your question with the full/exact error you get – Val Jul 08 '19 at 16:30

How to _reindex elasticsearch data to new mapping (from flat fields to objects)?

1 Answers1