Store Key-Value structure in Elasticsearch via Spring-Data

Question

I need information how to store the best way a Document (Java POJO) with the Spring-Data-Elasticsearch @Document Annotation which includes a Map

@Document(indexName = "downloadclienterrors", type = "downloadclienterror")
public class DownloadClientErrorLogElasticsearch {

    @Id
    private Long id;

    @Field(type = FieldType.String, index = FieldIndex.not_analyzed)
    private String host;
    @Field(type = FieldType.String, index = FieldIndex.not_analyzed)
    private String shortMessage;
    @Field(type = FieldType.String, index = FieldIndex.not_analyzed)
    private String fullMessage;

    @Field(type = FieldType.Date)
    private String clientTimestamp;

    private Integer level;

    private Map<String, String> additionalFieldList;

    ...
}

Like the POJO is created in this 1st class I can store it via my repository in the elastic search instance.

This is the way how I add then data to it, I wanna be flexible which JSON fields I add, because that's flexible from my client software.

    additionalFieldList.put("url", "http://www.google.de");
    additionalFieldList.put("user_agent", "Browser/1.0.0 Windows");

My problem is that I need also the fields in the additionalFieldList marked as .not_analyzed. (f.e additionalFieldList.url, additionalFieldList.user_agent). I would like to have the same behaviour like with the FieldIndex.not_analyzed annotation on a String also on my Map but of course only for the value in the map.

    @Field(type = FieldType.String, index = FieldIndex.not_analyzed)
    private Map<String, String> additionalFieldList;

But that doesn't work when I try to store the document. I receive a ugly Exception.

When someone knows a way, or how it would be better to design such a document in elasticsearch, because I am quit fresh and new in this area I would love to hear some comments.

Thanks before and grey greetings from Hamburg,

Tommy Ziegler

score 0 · Answer 1 · edited May 10 '15 at 00:40

You can use @Mapping annotation to configure dynamic_templates.

Just put your mapping file in your classpath and annotate your POJO with @Mapping

Mapping example

JSON

{
    "downloadclienterrors": {
        "dynamic_templates": [
            {
                "additionalFieldList": {
                    "path_match": "additionalFieldList.*",
                    "mapping": {
                        "type": "string",
                        "index": "not_analyzed"
                    }
                }
            }
        ]

        ...

    }
}

POJO

@Mapping(mappingPath = "/downloadclienterrors.json")
@Document(indexName = "downloadclienterrors", type = "downloadclienterror")
public class DownloadClientErrorLogElasticsearch {

    ...

}

score 0 · Answer 2 · answered May 17 '18 at 09:25

What you have to do is to create a another class additional and add additionalFieldList there.

something like this-

public class additional {

      private Map<String, String> additionalFieldList;

}

and then use this class in your pojo

@Document(indexName = "downloadclienterrors", type = "downloadclienterror")
public class DownloadClientErrorLogElasticsearch {

    @Id
    private Long id;

    @Field(type = FieldType.String, index = FieldIndex.not_analyzed)
    private String host;
    @Field(type = FieldType.String, index = FieldIndex.not_analyzed)
    private String shortMessage;
    @Field(type = FieldType.String, index = FieldIndex.not_analyzed)
    private String fullMessage;

    @Field(type = FieldType.Date)
    private String clientTimestamp;

    private Integer level;

    @Field(type = FieldType.Nested)
    private additional additional;

    ...
}

Store Key-Value structure in Elasticsearch via Spring-Data

2 Answers2