7

How can I store elasticsearch settings+mappings in one file (like schema.xml for Solr)? Currently, when I want to make a change to my mapping, I have to delete my index settings and start again. Am I missing something?

I don't have a large data set as of now. But in preparation for a large amount of data that will be indexed, I'd like to be able to modify the settings and some how reindex without starting completely fresh each time. Is this possible and if so, how?

jbattle
  • 143
  • 2
  • 10

2 Answers2

16

These are really multiple questions disguised as one. Nevertheless:

How can I store elasticsearch settings+mappings in one file (like schema.xml for Solr)?

First, note, that you don't have to specify mapping for lots of types, such as dates, integers, or even strings (when the default analyzer is OK for you).

You can store settings and mappings in various ways, in ElasticSearch < 1.7:

  1. In the main elasticsearch.yml file
  2. In an index template file
  3. In a separate file with mappings

Currently, when I want to make a change to my mapping, I have to delete my index settings and start again. Am I missing something?

You have to re-index data, when you change mapping for an existing field. Once your documents are indexed, the engine needs to reindex them, to use the new mapping.

Note, that you can update index settings, in specific cases, such as number_of_replicas, "on the fly".

I'd like to be able to modify the settings and some how reindex without starting completely fresh each time. Is this possible and if so, how?

As said: you must reindex your documents, if you want to use a completely new mapping for them.

If you are adding, not changing mapping, you can update mappings, and new documents will pick it up when being indexed.

czerasz
  • 13,682
  • 9
  • 53
  • 63
karmi
  • 14,059
  • 3
  • 33
  • 41
  • Sorry for the multiple questions. It was all one scenario for me since I hadn't figured out re-index without starting the entire process over. Thanks for your response. I'm currently modifying mappings specifically using the NGram and edgeNGram tokenizers and in an attempt to get just right have had to re-write them. – jbattle Oct 20 '11 at 19:25
  • I've placed my mappings+settings inside of a shell script so as to make the creation and configuration of my index a faster process. – jbattle Oct 20 '11 at 19:43
  • Yes, in most scenarios, it makes sense to wrap index creation in custom procedure (could be shell script, class method in Ruby, etc). I just wanted to point to many ways ElasticSearch gives you to "wrap" the mapping/settings logic on the search engine level. – karmi Oct 22 '11 at 07:29
  • +1 for this nice answer. One thing which should be pointed out is that I wouldn't store things into configuration files and use the API to update them like both of you mentioned e.g. while index creation. It makes it simple to change one settings on all nodes! – Karussell Oct 22 '11 at 08:45
0

Since Elasticsearch 2.0:

It is no longer possible to specify mappings in files in the config directory.

Find the documentation link here.

It's also not possible anymore to store index templates within the config location (path.conf) under the templates directory.

The path.conf (/etc/default/elasticsearch by default on Ubuntu) stores now only environment variables including heap size, file descriptors.

You need to create your templates with curl.

If you are really desperate, you could create your indexes and then backup your data directory and then use this one as your "template" for new Elasticsearch clusters.

czerasz
  • 13,682
  • 9
  • 53
  • 63