1

I am migrating from 2.x to 5.x I am adding values to the index like this 

indexInto (indexName / indexType) id someKey source foo

however I would also want to fetch all values by field:

 def getValues(tag: String) ={
client execute {
search(indexName / indexType) query ("_field_names", tag) aggregations (termsAggregation( "agg") field tag size 1)
}

But I am getting this exception :

RemoteTransportException[[8vWOLB2][172.17.0.5:9300][indices:data/read/search[phase/query]]]; nested: IllegalArgumentException[Fielddata is disabled on text fields by default. Set fielddata=true on [my_tag] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory.];

I am thought maybe to use keyword as shown here , but the fields are not known in advanced (sent by the user) so I cannot use perpend mappings

igx
  • 4,101
  • 11
  • 43
  • 88

1 Answers1

1

By default all the unknown fields will be indexed/added to elasticsearch as text fields which are not specified in the mappings. If you will take a look at mappings of such a field, you can see there a field is enabled with for such fields with type 'keyword' and these fields are indexed but not analyzed.

GET new_index2/_mappings
{
  "new_index2": {
    "mappings": {
      "type": {
        "properties": {
          "name": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      }
    }
  }
}

so you can use the fields values for the text fields for aggregations like the following

POST new_index2/_search
{
  "aggs": {
    "NAME": {
      "terms": {
        "field": "name.fields",
        "size": 10
      }
    }
  }
}

Check name.fields

So your scala query can work if you can shift to fields value.

def getValues(tag: String) = {
   client.execute {
      search(indexName / indexType)
        .query("_field_name", tag)
        .aggregations {
           termsAgg("agg", "field_name.fields")
        }.size(1)
   }
}

Hope this helps. Thanks

sksamuel
  • 16,154
  • 8
  • 60
  • 108
user3775217
  • 4,675
  • 1
  • 22
  • 33
  • thanks , I think that the mapping method should be PUT right ? should I add also analyzer ? – igx Mar 14 '17 at 08:19
  • mapping goes along with setting, the one in my answer is to fetch mappings for already created index. its the mappings endpoint for each index to get all types and mappings associated with it – user3775217 Mar 14 '17 at 08:47
  • analyzer, i don't think so you need to add analyzer. This field was created in the index on the fly, i didn't defined any mappings for this field as you mentioned 'fields are not known in advance'. So this is refering to what mappings look like if you have not defined them in your schema settings – user3775217 Mar 14 '17 at 08:50
  • please correct me if I am wrong but in your example I need to add a mapping to the field in advanced and my problem is that the fields names are known only in runtime . – igx Mar 14 '17 at 09:13
  • yes, you are correct. if you don't add mapping to the fields then by default these mappings will be added for this field { "new_index2": { "mappings": { "type": { "properties": { "name": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } } } } } } }. You don't have to add them yourself. You just have to use .fields while aggregating. – user3775217 Mar 14 '17 at 09:21
  • thanks, it did solve the exception hence it is marked as the correct answer. but maybe you can assist also here http://stackoverflow.com/questions/42791776/elasticsearch-5-x-getting-distinct-values-of-field – igx Mar 14 '17 at 16:43