1

I'm currently using the phillbaker/elasticsearch Terraform provider to generate indices on an AWS OpenSearch cluster running Elasticsearch 7.10 .

For most of my indices this works fine, however, I have one large index that has more than the default 1000 field limit. Based on the providers documentation there isn't an argument for the field limit, I've tried putting it in the mapping but that causes provider to fail to parse my mapping JSON file.

This is the terraform block for the index:

resource "elasticsearch_index" "large_index" {
  name               = "large-index"
  number_of_shards   = 1
  number_of_replicas = 2
  mappings           = file("${path.root}/elastic-search-mappings/large-index.json")
  analysis_analyzer  = jsonencode({
    email_analyzer = {
      tokenizer = "uax_url_email"
    }
  })
}

Is there another way to configure this setting with this provider?

I'm including the error from putting the setting into the mapping and the original error:

Error: elastic: Error 400 (Bad Request): Failed to parse mapping [_doc]: Root mapping definition has unsupported parameters:  [total_fields : {limit=2000}] [type=mapper_parsing_exception]

Error: elastic: Error 400 (Bad Request): Limit of total fields [1000] has been exceeded [type=illegal_argument_exception]
Jesse
  • 61
  • 4
  • you try this: [What does "Limit of total fields 1000 in index has been exceeded" means in Elasticsearch](https://stackoverflow.com/a/55373088/18778181) – rabbitbr Dec 28 '22 at 16:56
  • @rabbitbr Thank you, but configuring that setting using the REST API requires the index to be created and that would mean configuring the mappings and settings after running `terraform apply` which would also mean I would lose the benefits of IaC for this index. – Jesse Dec 28 '22 at 17:21
  • you can use like this: "settings": { "index": { "mapping": { "total_fields.limit": 2000 } }, .... – rabbitbr Dec 28 '22 at 18:20

1 Answers1

0

Try create index with configuration:

{
  "settings": {
    "index": {
     "mapping": {
       "total_fields": {
         "limit": 2000
       }
     }
   },
    "analysis": {
      "analyzer": {
       ...
    }
  },
  "mappings": {
    "properties": {
    .....

   }
}
rabbitbr
  • 2,991
  • 2
  • 4
  • 17
  • This works only for creating indices with Elastic Search's REST API. I'm trying to use the `phillbaker/elasticsearch` provider which does not support configuring settings in a json format. – Jesse Dec 28 '22 at 21:49