7

In a Elasticsearch index, I have a few fields which are referencing main categories' ids (e.g. sector_id, country_id, etc...).

These fields are solely use for filtering (using the term/terms filters) and for creating buckets in terms aggregations (among others).

Each one of them is currently using the smallest suitable numeric datatype (e.g. byte, short, etc..)

Is this the best datatype to be used on these for heavy aggregations?

Or should these be using the keyword datatype?

Thanks in advance for any advice!

Concordia Discors
  • 369
  • 3
  • 15

1 Answers1

4

If the values of those fields are numeric, you should go for a numeric type, if they are strings, then go for the keyword type.

One thing to bear in mind is that if you want to run range queries and/or range aggregations on those fields at some point, you should prefer using a numeric type up front so that those values can be sorted numerically and not lexically.

For instance: if you have country ids such as 1, 2, 3, ..., 10, 11, 12, ..., 20, ... and they are mapped as keyword (i.e. string) then if you run a range query (or aggregation) on them with from: 1, to: 3, you'll also get 11, 12, 13, etc since in the string world, 11 is lexically smaller than 3.

Val
  • 207,596
  • 13
  • 358
  • 360
  • Thanks a lot for the reply @Val! In this instance, I don't intend to do any kind of numerical/range operation on these fields (they are solely used as unique identifier), and am more concerned about the actual performance of heavy aggregations. Would you still suggest to stick to the numeric types in regard to performance? Thanks again! – Concordia Discors Nov 24 '16 at 10:24
  • Since the answer depends on many more factors than simply the type of the field, and if your aggregations are really that heavy, I think it's worth giving it a shot and try both solutions (what you should do anyway), you'll see immediately what works best. – Val Nov 24 '16 at 13:06
  • 1
    There's also the possibility to have some hybrid approach (numeric+keyword) depending on your use case. See more details here: https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-search-speed.html#_pre_index_data – Val Nov 25 '16 at 05:11
  • Thanks @Val I think I found the answer in the page you were referencing (I had a look at it before posting my question but obviously skimmed through it too quickly!): _The fact that some data is numeric does not mean it should always be mapped as a numeric field. Typically, fields storing identifiers such as an ISBN or any number identifying a record from another database, might benefit from being mapped as keyword rather than integer or long._ From the [Mappings Section](https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-search-speed.html#_mappings). – Concordia Discors Nov 25 '16 at 11:23