Spring Data Elasticsearch with ES 7.2.1 | GeoPoint mapping failure while indexing

Question

I am using ES 7.2.1 to store large amount of location based data and querying for near-by locations. For location coordinates, I am using GeoPoint fields from my java codebase.

ES: 7.2.1
Spring Data Elasticsearch: 4.0.0.DATAES-690-SNAPSHOT
MVN org.elasticsearch: 7.2.1

Template:

curl -X PUT "localhost:9200/_template/store_locator_template?pretty"
{
  "order": 1,
  "index_patterns": [
    "store_locator_*"
  ],
  "settings": {
  },
  "mappings": {
      "properties": {
      "esId": {
        "type": "keyword"
      },
      "geoPoint": {
        "type": "geo_point"
      },
      "storeName": {
        "type": "keyword"
      }
    }
  }
}

When trying to insert data via bulkIndex(), I am getting this error:

org.springframework.data.elasticsearch.ElasticsearchException: 
Bulk indexing has failures. Use ElasticsearchException.getFailedDocuments() 
for detailed messages [{QObQeXEBqxAg6uMFyeNZ=ElasticsearchException
[Elasticsearch exception 
[type=illegal_argument_exception, reason=
mapper [geoPoint] of different type, 
current_type [geo_point], merged_type [ObjectMapper]]]}]

Entity:

@Getter
@Setter
@ToString
@EqualsAndHashCode(of = "esId", callSuper = false)
@NoArgsConstructor
@Document(indexName = "store_locator_index", replicas = 0, createIndex = false)
public class EsEntity {

  @Id
  @Field(type = FieldType.Text)
  private String esId;

  @GeoPointField
  private GeoPoint geoPoint;

  @Field(type = FieldType.Text)
  private String storeName;
}

UPDATE: If I use the below code, it works fine. it puts the mapping as required and spring data es does no complain!

//clazz -> entity class with @Document annotation

boolean indexCreated = false;
if (!elasticsearchOperations.indexExists(clazz)) {
    indexCreated = elasticsearchOperations.createIndex(clazz);
}
if (indexCreated) {
    elasticsearchOperations.refresh(clazz);
    elasticsearchOperations.putMapping(clazz);   --> Does the MAGIC
}

... And the mapping generated from the above code is:

{
   "esentity":{              ---> Why is this here??
      "properties":{
         "esId":{
            "type":"keyword",
            "index":true
         },
         "geoPoint":{
            "type":"geo_point"
         }
      }
   }
}

It is adding a type, by the name of my entity class, to the mapping!

====================
Also.....

Everything seems to be working for:
ES: 6.4.3
Spring Data Elasticsearch: 3.1.X

I am able to (via template) insert document with GeoPoint field. The index is generated automatically when doc is inserted via code. The same set of code works fine with no error!!!!

Here's my template:

curl -X PUT "localhost:9200/_template/store_locator_template?pretty"
{
  "order": 1,
  "index_patterns": [
    "store_locator_*"
  ],
  "settings": {
  },
  "mappings": {
    "store_locator_index": {
      "properties": {
      "esId": {
        "type": "keyword"
      },
      "geoPoint": {
        "type": "geo_point"
      },
      "storeName": {
        "type": "keyword"
      }
    }
   }
  }
}

Here's the mapping:

{
  "mapping": {
    "properties": {
      "esId": {
        "type": "keyword"
      },
      "geoPoint": {
        "type": "geo_point"
      }
    }
  }
}

can you please share the code for your entity class? – P.J.Meisch Apr 14 '20 at 20:05 — P.J.Meisch, Apr 14 '20 at 20:05

score 0 · Answer 1 · answered Apr 16 '20 at 21:53

There are some things that don't match in the code you show:

In the first template you show, you define the storeName to be of type keyword, but on the entity you have it as type text.

A field annotated with @Id is always a type keyword, the @Field annotation defining it as type text is ignored.

I used the following versions: ES 7.3.0 (don't have 7.2.1 on my machine), Spring Data 4.0 current master, client libs set to 7.3.0.

When I don't have the template defined, but create the index with the code you showed:

    boolean indexCreated = false;

    Class<EsEntity> clazz = EsEntity.class;
    if (!elasticsearchOperations.indexExists(clazz)) {
        indexCreated = elasticsearchOperations.createIndex(clazz);
    }
    if (indexCreated) {
        elasticsearchOperations.refresh(clazz);
        elasticsearchOperations.putMapping(clazz);
    }

I get the following index:

{
  "store_locator_index": {
    "aliases": {},
    "mappings": {
      "properties": {
        "esId": {
          "type": "keyword"
        },
        "geoPoint": {
          "type": "geo_point"
        },
        "storeName": {
          "type": "text"
        }
      }
    },
    "settings": {
      "index": {
        "refresh_interval": "1s",
        "number_of_shards": "1",
        "provided_name": "store_locator_index",
        "creation_date": "1587073075464",
        "store": {
          "type": "fs"
        },
        "number_of_replicas": "0",
        "uuid": "72aZqWDtS7KLDMwdkgVtag",
        "version": {
          "created": "7030099"
        }
      }
    }
  }
}

The mapping looks like it should, there is no type info in the mapping (this was written by the Spring Data Elasticsearch 3.2 version when using ES 6, but i not used anymore)

When I add the template you showed and then do a bulk insert with the following code:

    EsEntity es1 = new EsEntity();
    es1.setEsId("1");
    es1.setGeoPoint(new GeoPoint(12, 34));
    es1.setStoreName("s1");
    IndexQuery query1 = new IndexQueryBuilder().withId("1").withObject(es1).build();

    EsEntity es2 = new EsEntity();
    es2.setEsId("2");
    es2.setGeoPoint(new GeoPoint(56, 78));
    es2.setStoreName("s2");
    IndexQuery query2 = new IndexQueryBuilder().withId("2").withObject(es2).build();

    elasticsearchOperations.bulkIndex(Arrays.asList(query1, query2), IndexCoordinates.of("store_locator_index"));

then the following index is created (note that store_name is type keywordnow, coming from the template):

{
  "store_locator_index": {
    "aliases": {},
    "mappings": {
      "properties": {
        "_class": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "esId": {
          "type": "keyword"
        },
        "geoPoint": {
          "type": "geo_point"
        },
        "storeName": {
          "type": "keyword"
        }
      }
    },
    "settings": {
      "index": {
        "creation_date": "1587073540386",
        "number_of_shards": "1",
        "number_of_replicas": "1",
        "uuid": "LqzXMC5uRmKmImIzblFBOQ",
        "version": {
          "created": "7030099"
        },
        "provided_name": "store_locator_index"
      }
    }
  }
}

and the two documents are inserted as they should:

{
  "took": 22,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 2,
      "relation": "eq"
    },
    "max_score": 1.0,
    "hits": [
      {
        "_index": "store_locator_index",
        "_type": "_doc",
        "_id": "1",
        "_score": 1.0,
        "_source": {
          "_class": "com.sothawo.springdataelastictest.EsEntity",
          "esId": "1",
          "geoPoint": {
            "lat": 12.0,
            "lon": 34.0
          },
          "storeName": "s1"
        }
      },
      {
        "_index": "store_locator_index",
        "_type": "_doc",
        "_id": "2",
        "_score": 1.0,
        "_source": {
          "_class": "com.sothawo.springdataelastictest.EsEntity",
          "esId": "2",
          "geoPoint": {
            "lat": 56.0,
            "lon": 78.0
          },
          "storeName": "s2"
        }
      }
    ]
  }
}

So I cannot find an error in the code, but you should check the templates and existing indices if there are conflicting entries.

Thanks. storeName is text, while posting a question I somehow edited it wrong to keyword. — Black Diamond, Apr 17 '20 at 05:59

Spring Data Elasticsearch with ES 7.2.1 | GeoPoint mapping failure while indexing

1 Answers1