ElasticSearch 7.6.3 Java HighLevel Rest Client : Auto Suggest across multiple fields - How to implement

Question

We have a index with the following fields and there is a requirement to give an Auto Suggest to the user by searching the data across all text and keyword mapping fields in the index

{


"settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1,
    "analysis": {
      "filter": {
        "autocomplete_filter": {
          "type": "edge_ngram",
          "min_gram": 1,
          "max_gram": 20
        }
      },
     }
  },

  "mappings": {
    "properties": {
      "id": {
        "type": "text"
      },
      "title": {
        "type": "text"
      },
      "date": {
        "type": "date",
        "format": "yyyy-MM-dd'T'HH:mm:ss.SSSZ"
      },
      "subject": {
        "type": "text"
      },
      "title_suggest": {
            "type": "completion",
            "analyzer": "simple",
            "preserve_separators": true,
            "preserve_position_increments": true,
            "max_input_length": 50
        },
        "subject_suggest": {
            "type": "completion",
            "analyzer": "simple",
            "preserve_separators": true,
            "preserve_position_increments": true,
            "max_input_length": 50

        }
      "fieldOr": {
        "type": "text"
      },
      "fieldsTa": {
        "type": "text"
      },
      "notes": {
        "type": "text"
      },
      "fileDocs": {
        "type": "nested",
        "properties": {
          "fileName": {
            "type": "text",
            "analyzer": "autocomplete", 
            "search_analyzer": "standard"
          },
          "fileContent": {
            "type": "text",
            "analyzer": "autocomplete", 
            "search_analyzer": "standard" 
          },
          "docType": {
            "type": "keyword"
          },
          "opinionId": {
            "type": "integer"
          }
        }
      },   
      "fileMeta": {
        "type": "nested",
        "properties": {
          "url": {
            "type": "text"
          },
          "name": {
            "type": "text"
          }
        }
      }
    }
  }
}

I have tried the Completion Suggest but it works with 1 fields. I have created 2 fields with *-suggest in the index and tried to create the Suggest using the completionSuggest

SuggestBuilders.completionSuggestion("my_index_suggest").text(input);

But it supports 1 field only. I am using ES 7.6.3 with Java HighLevel Rest Client and it works for 1 fields. What changes I need to do to support across multiple fields . Is this possible via JSON search ? If yes then I can create a json using Xcontentbuilder and do a Auto suggest ?

if you are not restricted to just use the ES completion suggestor then, I can tell you multiple ways to do it , lmk if you are intested — Amit, Feb 25 '20 at 15:32
HI Opster ES Ninja, Can you please suggest the other ways possible so that I can also try out them. We are using Spring Boot with angular frontend and if you see the index user can upload files like docx , pdf etc and we may need to Auto suggest based on the file content also . We are currently using Tika parser to get the content and save in the index under fileContent field. — sivacham, Feb 26 '20 at 05:57
Added my answer, pls have a look at it and let me know if need any clarification, also read my blog on autocomplete https://opster.com/elasticsearch-glossary/elasticsearch-autocomplete-troubleshooting-guide/ and why I used prefix queries to implement it in your case. — Amit, Mar 21 '20 at 06:29

score 2 · Accepted Answer · answered Feb 25 '20 at 15:03

2

Use the copy_to and copy all your desired fields to one field and perform your suggestion on top of it.

Example from the documentation for copy_to is,

PUT my_index
{
  "mappings": {
    "properties": {
      "first_name": {
        "type": "text",
        "copy_to": "full_name" 
      },
      "last_name": {
        "type": "text",
        "copy_to": "full_name" 
      },
      "full_name": {
        "type": "text"
      }
    }
  }
}

answered Feb 25 '20 at 15:03

aswath86

551
5
14

HI WinnieDaPooh, If you observe the index we are using a field named Filecontent where the content from uploaded documents from Angular FE is parsed using Tika parser and stored.If I use the copy_to option it will create a giant field and do you think we can perform a efficient Auto suggest using this merged field.(User can upload multiple files like docx/pdf etc) – sivacham Feb 26 '20 at 06:04

score 1 · Answer 2 · answered Mar 21 '20 at 06:28

For illustration purpose, I am using my own index mapping which has just two fields name and address and I would be making autocomplete queries using prefix on both the fields and you can include more fields similarly.

Index mapping

{
  "employee": {
    "mappings": {
      "properties": {
        "address": {
          "type": "text"
        },
        "name": {
          "type": "text"
        }
      }
    }
  }
}

Search query using Rest high-level client

 public SearchResponse autosuggestSearch() throws IOException {
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        BoolQueryBuilder qb = QueryBuilders.boolQuery();

        PrefixQueryBuilder namePQBuilder = QueryBuilders.prefixQuery("address", "usa");
        PrefixQueryBuilder addressPQBuilder = QueryBuilders.prefixQuery("address", "usa");
        qb.should(namePQBuilder);
        qb.should(addressPQBuilder); //Similarly add more fields prefix queries.
        sourceBuilder.query(qb);

        SearchRequest searchRequest = new SearchRequest("employee").source(sourceBuilder);
        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
        System.out.println("Search JSON query \n" + searchRequest.source().toString()); //Generated ES search JSON.
        return searchResponse;
    }

For this example generated search JSON

{
  "query": {
    "bool": {
      "should": [
        {
          "prefix": {
            "address": {
              "value": "usa",
              "boost": 1.0
            }
          }
        },
        {
          "prefix": {
            "address": {
              "value": "usa",
              "boost": 1.0
            }
          }
        }
      ],
      "adjust_pure_negative": true,
      "boost": 1.0
    }
  }
}
}

I have marked @WinnieDaPooh as the answer since I have used the copy_to in the index . I have also used Opster Elasticsearch Ninja suggestion and created the boolquery with some modifications to get a exact match for my auto -suggest so that users do not get un-necessary search responses. Thanks to bot its working now :) — sivacham, Mar 25 '20 at 13:11

ElasticSearch 7.6.3 Java HighLevel Rest Client : Auto Suggest across multiple fields - How to implement

2 Answers2

Index mapping

Search query using Rest high-level client

For this example generated search JSON