Do I have to specify a mapping in order to use the document.search controller in Kuzzle?

Question

I am using Kuzzle in order to build a simple slack-like application. I have a simple collection with documents only containing other documents ids. I wanted to query a list of ids from this collection and got stuck with an empty hits array in the response, no matter what id values I tried in the query. It got me to read about Elasticsearch query syntax, mapping, and I found out I needed to specify a mapping.

When the collection is created without specifying a mapping, document.search:

• returns an empty array of hits if given a "query" body property such as the following:

  {"query": {"terms": {"id": <ids array>}}}
  

• throws the following if given a "sort":

  // {"sort": [{"id": {"order": "desc"}}]}
  Error: No mapping found for [id] in order to sort on
  

• and returns an empty bucket list for an "aggregations" like this :

  "aggregations": {
      "groupById": {
          "terms": {
              "field": "id"
          }
      }
  }
  

I don't understand what is happening behind this. Is it normal that only one case is being treated as an error? I was working on the query part, and it was not obvious that the error didn't come from the content of my ids array!

Adding a simple mapping on "id" with the "keyword" type makes all cases work as intended, but I read about dynamic mappings in Elasticsearch. Is there a way to use them? Or is there another solution for querying a (the only) parameter in my document?

I'm using kuzzle-sdk v7.1.4

Many thanks!

Answer 1

Elasticsearch mappings are a way to tell the underlying engine how to index the fields of your documents.

For example, a text field will not be indexed the same way as a keyword field and you won't be able to make the same kind of query.

keyword fields are indexed "as is", so you can do very fast search on exact terms (with the term query) but text fields are analyzed with various method to be able to do a fuzzy match query for example.

If there is no declared mapping for a field, Elasticsearch won't be able to access it in any query.

You can read our Elasticsearch Cookbook for more informations.

About the mappings dynamic policy, by default this policy is set to false, meaning that ES will not infer newly introduced fields types.

We choose to have this default value because you cannot modify the type of a field so it's better to define it yourself than having poorly accurate types inferred by ES.

You can change this policy for the entire mapping but also for only a nested object for example (which is considered as a best practice)

{
  "dynamic": "false",
  "properties": {
    "id": { "type": "keyword" },
     "metadata": {
       "dynamic": "true",
       "properties": {
         // field types will be automatically inferred in the "metadata" object
       }
     }
  }
}

Then if you create the following document:

{
  "id": "your-unique-uuid",
  "name": "aschen",
  "metadata": {
    "avatar": "http://url.com"
  }
}

• id field was already declared as keyword, you can make query on this field.

• name field was not declared and the dynamic policy is false so you cannot make query on this field.

• metadata.avatar field was not declared but the dynamic policy for this nested object is true so ES will add a text type for this field, you can make query on this field.

One last thing, you are using an id field inside your document but Elasticsearch already generate unique identifier (_id) for every documents so you may want to use this instead.

I am a developer on Kuzzle core