Elasticsearch: Issues reindexing - ending up with more than one type

Question

ES 6.8.6 I am trying to reindex some indexes to reduce the number shards.

The original index had a type of 'auth' but recently I added a template that used _doc. When I tried:

curl -X POST "localhost:9200/_reindex?pretty" -H 'Content-Type: application/json' -d'
{
  "source": {
      "index": "auth_2019.03.02"
  },
  "dest": {
      "index": "auth_ri_2019.03.02",
      "type": "_doc"

  }
}
'

I get this error:

"Rejecting mapping update to [auth_ri_2019.03.02] as the final mapping would have more than 1 type: [_doc, auth]"

I understand that I can't have more than one type and that types are depreciated in 7.x. My question is can I change the type during the reindex operation.

I am trying to tidy everything up in preparation to moving to 7.x.

Try to set `type` inside `dest` as [per documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html) — leandrojmp, Aug 05 '20 at 22:43
Thanks!, actually I tried that and, rereading the doc, I see that the type of the source index is supposedly ignored. The question now becomes where could it be picking up the 'auth' type which is the type from the source index. Will update the question... — Russell Fulton, Aug 06 '20 at 00:38
Hmmm... adding "type": "auth" to source made the error go away and now I get an empty index created but with the correct setup. Progress. — Russell Fulton, Aug 06 '20 at 00:59
Doh! there is a good reason that the dest index was empty. So was source! Problem solved. What I had to do was add explicit type entries for both source and dest. — Russell Fulton, Aug 06 '20 at 01:09

score 1 · Answer 1 · answered Aug 06 '20 at 01:19

It looks like you have to write a script to change the document during the reindex process.

From the docs,

Like _update_by_query, _reindex supports a script that modifies the document.

You are indeed able to change type.

Think of the possibilities! Just be careful; you are able to change: _id, _type, _index, _version, _routing

For your case add


  "script": {
    "source": "ctx._type = '_doc'",
    "lang": "painless"
  }

Full example

{
  "source": {
      "index": "auth_2019.03.02"
  },
  "dest": {
      "index": "auth_ri_2019.03.02",
  },
  "script": {
    "source": "ctx._type = '_doc'",
    "lang": "painless"
  },
}

Russell Fulton · Answer 2 · 2020-08-07T18:57:46.450

Firstly thanks to leandrojmp for prompting me to reread the docs and noticing the example where they had type specified for both source and dest.

I don't understand why but adding a type to the source specification solved the problem.

This worked:

curl -X POST "localhost:9200/_reindex?pretty" -H 'Content-Type: application/json' -d'
{
  "source": {
      "index": "auth_2019.03.02",
      "type": "auth"
  },
  "dest": {
      "index": "auth_ri_2019.03.02",
      "type": "_doc"

  }
}
'

Elasticsearch: Issues reindexing - ending up with more than one type

2 Answers2