Elasticsearch Delete Mapping Property

Question

I am trying to figure out an approach to delete all entries for a specific property in an elasticsearch index and remove all type mappings for that property.

I have been looking at the following two doc pages: put mapping and delete mapping

From second link:

"Allow to delete a mapping (type) along with its data. The REST endpoint is /{index}/{type} with DELETE method."

What I think I need is a /{index}/{type}/{property}?

Do I need to recreate the whole index to accomplish this, i.e. moving and manipulating data between types?

For Example, calling GET on the mapping:

curl -XGET 'http://.../some_index/some_type/_mapping'

result:

{
  "some_type": {
    "properties": {
      "propVal1": {
        "type": "double",
        "index": "analyzed"
      },
      "propVal2": {
        "type": "string",
        "analyzer": "keyword"
      },
      "propVal3": {
        "type": "string",
        "analyzer": "keyword"
      }
    }
  }
}

after this delete operation on propVal3 would return:

curl -XGET 'http://.../some_index/some_type/_mapping'

result:

{
  "some_type": {
    "properties": {
      "propVal1": {
        "type": "double",
        "index": "analyzed"
      },
      "propVal2": {
        "type": "string",
        "analyzer": "keyword"
      }
    }
  }
}

and all data for propVal3 would be removed through the index.

score 22 · Accepted Answer · answered Apr 22 '13 at 21:16

22

You can not do that. Just forget that this value exists... ;-) If you really need to remove it, you will have to reindex your documents.

answered Apr 22 '13 at 21:16

dadoonet

14,109
3
42
49

Thanks for the response. I feel I am going to need to do this. What is the best strategy for doing this via "reindexing"? I realize this is going to be a performance nightmare... but what strategy would you take here? Is moving to another type (removing property in process) and back again an option? – Ryan R. Apr 22 '13 at 21:44
Created a followup to this question: http://stackoverflow.com/questions/16159902/elasticsearch-reindex-or-flag-deleted-type-property – Ryan R. Apr 23 '13 at 02:30
Why does the documentation say it is possible? Or did I misunderstand it? – brupm Nov 19 '13 at 21:23
Could you provide the link to the doc where we say that? – dadoonet Nov 20 '13 at 04:43

score 16 · Answer 2 · answered Aug 10 '16 at 12:57

You can use the new _reindex api for this, you could even PUT a new _mapping to the dest index before running the reindex so you can change the properties of the fields in your index.

To do a reindex and removing a property, you can do this:

POST /_reindex
{
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter",
  },
  "script": {
    "inline": "ctx._source.remove('whatever')"
  }
}

if you would use this in combination with the _aliases API you can modify indexes without having any 'downtime'

This answer took me a while to find. But this is the only way I was able to completely delete a filed of null values in elastic. Thank you. — j7skov, Jan 11 '22 at 16:17

score 4 · Answer 3 · answered Apr 22 '13 at 21:15

4

It's not currently possible to remove a property from a mapping. In order to remove all values of a property from all records, you need to reindex all records with this property removed.

answered Apr 22 '13 at 21:15

imotov

28,277
3
90
82

1

@TheRedPea using [reindex API](https://www.elastic.co/guide/en/elasticsearch/reference/2.3/docs-reindex.html) for example or one of many reindex scripts available for elasticsearch. – imotov May 24 '16 at 17:52

score 0 · Answer 4 · answered Sep 24 '22 at 21:22

You can choose whats documents fields you will reindex to a new index. For example:

POST _reindex { "source": { "index": "my-source-index", "_source": ["host.hostname", "host.ip", "another_field"] }, "dest": { "index": "my-dest-index" } }

Reference: https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html#docs-reindex-filter-source

score 0 · Answer 5 · answered Nov 15 '22 at 07:55

0

Just want to add another approach for the case of "removing property from index".

POST pulse/_update_by_query
{
  "query": {
    "match_all": {}
  },
  "script": {
    "source": "ctx._source.remove(\"file_id\")",
    "lang": "painless"
  }
}

answered Nov 15 '22 at 07:55

Karma Blackshaw

880
8
20

Elasticsearch Delete Mapping Property

5 Answers5

Linked