0

iam using elastic search 7.0. I have a model which has to be saved to elastic.

When the index did not exist, then i try to save the documents directly to elastic:

final IndexRequest indexRequest = new IndexRequest("seminar_map", "seminar", id)
                    .source(new Gson().toJson(object), XContentType.JSON);
            indexRequest.id(id);
final IndexResponse indexResponse = client.index(indexRequest, RequestOptions.DEFAULT);

Everything works fine and the documents will be saved to elastic.

But i want to have a custom analyzer and i need to change the mapping types.

So i tried to set up the index and the mapping before saving any documents with:

try {
            CreateIndexRequest request = new CreateIndexRequest(index);
            request.settings(Settings.builder()
                    .loadFromSource(Strings.toString(jsonBuilder()
                            .startObject()
                            .startObject("analysis")
                            .startObject("analyzer")
                            .startObject("case_insensitive_analyzer")
                            .field("tokenizer", "keyword")
                            .field("type", "custom")
                            .field("filter", new String[]{"lowercase"})
                            .endObject()
                            .endObject()
                            .endObject()
                            .endObject()), XContentType.JSON)
            );

            XContentBuilder builder = XContentFactory.jsonBuilder();
            builder.startObject();
            {
                builder.startObject("properties");
                {
                    builder.startObject("seminar_nummer");
                    {
                        builder.field("analyzer", "case_insensitive_analyzer");
                    }
                    builder.endObject();
                }
                builder.endObject();
            }
            builder.endObject();
            request.mapping("_doc",builder);

            return client.indices().create(request, RequestOptions.DEFAULT);

But i get the error:

Elasticsearch exception [type=illegal_argument_exception, reason=The mapping definition cannot be nested under a type [_doc] unless include_type_name is set to true.]

How can i fix that error?

And is the plan correct to create a index and a mapping with not all parameter and then save the documents with more parameters than created. So will elastic add the other missing parameters to the mappings or do i have to set the complete mapping within the creating index part?

Florian Ernst
  • 3
  • 2
  • 4

3 Answers3

2

The error comes from the _doc in the JSON you are creating and elastic complains about it because of this.

Two options from here:

  1. Add include_type_name=true to the URL

OR

  1. Replace this line request.mapping("_doc",builder); with request.source(builder);,

See the docs for more details:

  1. https://www.elastic.co/guide/en/elasticsearch/reference/7.5/removal-of-types.html#_schedule_for_removal_of_mapping_types (note the difference between 6.8 and 7.X)

  2. https://www.elastic.co/guide/en/elasticsearch/client/java-rest/master/java-rest-high-put-mapping.html#CO385-1

ibexit
  • 3,465
  • 1
  • 11
  • 25
  • Hey, - 1. Option: How can i add the paramter to the url using the `RestHighLevelClient`? - 2. Option: Changing to `request.source(builder)` throws the exception `Elasticsearch exception [type=illegal_argument_exception, reason=unknown setting [index.properties.seminar_nummer.type] please check that any required plugins are installed, or check the breaking changes documentation for removed settings]` Are there any changes within the hierachy of the parameters? – Florian Ernst Feb 06 '20 at 08:36
  • 1
    Regarding the url parameter: you need to go one abstraction level down and use the Low Level Rest Client. – ibexit Feb 06 '20 at 09:07
  • Is your example still the same? Check if you provided a valid mapping type to the seminar_nummer field, for example type: text. Have a look into the docs, how a valid mapping should look like and rebuild it using the client: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-mapping.html – ibexit Feb 06 '20 at 09:11
  • The analyzer parameter should work. Using postman its working to set the mapping of seminar_nummer to: `"seminar_nummer": { "type": "text", "analyzer": "case_insensitive_analyzer", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }` And changing the parameter to **type : text** its the same exception – Florian Ernst Feb 06 '20 at 09:18
  • Not the analyser, you need a mapping type for each field/property definition under builder.startObject("seminar_nummer") – ibexit Feb 06 '20 at 09:20
  • Changed to `builder.startObject("properties"); { builder.startObject("seminar_nummer"); { builder.field("type", "text"); } builder.endObject(); } builder.endObject();` same exception – Florian Ernst Feb 06 '20 at 09:22
  • Ok, as you are using the create index api, which supports setting and mappings, you need to nest the properties under "mappings" in order to distinguish the mappings from the settinng (you don't set, what's ok, as the defaults are applied) – ibexit Feb 06 '20 at 09:25
  • Just to clarify the comment above: youn don't need to set settings, as long you don't need something different from the default index settings. Providing only a mapping is pretty fine. – ibexit Feb 06 '20 at 09:33
  • Thanks. I used the lower rest client, to create the index with the mapping. Its the same like using postman. So thank you very much for the tip. It was not possible for me to use the RestHighLevelClient to create a index with a custom analyzer and a custom mapping. – Florian Ernst Feb 07 '20 at 11:55
0

Types are deprecated in Elastic 7.x and will be removed in future versions of elastic. Changing request.mapping("_doc",builder) to request.mapping(builder) should work.

For the second question, Yes elastic search will add the missing fields that are not mentioned in the mapping when we save the documents.

Kumar V
  • 1,570
  • 1
  • 12
  • 19
  • Hey thanks for your answer. The problem is, that `request.mapping(builder)` is not working,, because the methods needs to have 2 parameters. `CreateIndexRequest mapping(String type, Object... source) {` – Florian Ernst Feb 06 '20 at 08:14
  • It has a method that takes only builder object. https://www.elastic.co/guide/en/elasticsearch/client/java-rest/7.x/java-rest-high-create-index.html – Kumar V Feb 06 '20 at 14:45
0

i also tried to seperate the create index with settings and the mapping part. The create index + settings is working well.

But the step add the mapping are not working:

final PutMappingRequest requestMapping = new PutMappingRequest("seminar_map");

            XContentBuilder builder = XContentFactory.jsonBuilder();
            builder.startObject();
            {
                    builder.startObject("properties");
                    {
                        builder.startObject("seminar_nummer");
                        {
                            builder.field("analyzer", "case_insensitive_analyzer");
                        }
                        builder.endObject();
                    }
                    builder.endObject();
            }
            builder.endObject();
            requestMapping.source(builder);

            client.indices().putMapping(requestMapping, RequestOptions.DEFAULT);

Dies gibt aber leider den Fehler: org.elasticsearch.action.ActionRequestValidationException: Validation Failed: 1: mapping type is missing;

But when iam adding a type to the request: requestMapping.type("_doc") it throws the exception 

Elasticsearch exception [type=illegal_argument_exception, reason=Types cannot be provided in put mapping requests, unless the include_type_name parameter is set to true.]

So it doesnt work with a type and doesnt work without a type :-D

Florian Ernst
  • 3
  • 2
  • 4