Custom search using Mongodb

Question

I have a MongoDB as my database and the backend is written in node.js. I am trying to implement a search for a table which returns me all results with the string entered AND string matching.

For example searching "foo" will return (In that order)

foo maker moo
doo foo doo //The order of word search does not matter as long as it puts the word search first
foobar
fooboo

Currently I have this but I am convinced there is a better way to do it without searching the db twice:


    async function(req, res) {
        var customerName = req.params.customerName;
   //word match
        var customers1 = await Models.DummyContactTable.find({
            customerName: {
                $regex: "^" + customerName,
                $options: 'i'
            },
            IsActive: true
        });
    //String match
   
     var customers2 = await Models.DummyContactTable.find({
          $and: [
              {
                customerName: {
                  $regex: customerName, $options: 'i'
                }
              },
              {
                customerName: {
                  $not: {
                    $regex: "^" + customerName,
                  }
                },
                IsActive: true
              }
          ]
        });
    //Since sometimes we get duplicates, doing a filter and find to de-dup
      var customers = customers1.concat(customers2.filter((customer) => !customers1.find(f => f.uuid === customer.uuid)));

Using user input as regular expressions is a security vulnerability. — D. SM, Aug 29 '20 at 03:30
I will eventually add the checks to the user input before sending it off to the db but for this demo code I just want to get the search right. — user1152262, Aug 29 '20 at 04:30
Problems like this are often solved with ngrams. Have you looked at Mongodb Atlas Search? You could use the autocomplete operator to return all the results quickly in one pass. https://docs.atlas.mongodb.com/atlas-search/ — Doug, Aug 30 '20 at 21:07
Just spent sometime researching that. It would be good if you can provide a working sample I can quickly try to see if that would work if possible otherwise I would have to do a lot of trial and error to see what would work. — user1152262, Aug 31 '20 at 00:14
@user1152262 I added an answer with an Atlas Search example. Hope that helps. — Doug, Aug 31 '20 at 19:08

score 1 · Answer 1 · answered Aug 31 '20 at 19:07

If you were using Atlas Search, you could write a query like this:

{ 
  $search: {
    autocomplete: { 
      path: "customerName",
      query: "foo"
}}}

// atlas search index definition
{
  "mappings": {
    "fields": {
      "customerName" : { 
        "type" : "autocomplete"
}}}

If you needed to control the result scores, you could use compound

{
  $search: {
    compound: {
      should: [ 
        {autocomplete: {path: "customerName", query: "foo" }},
        {text: {path: "customerName", query: "foo" , score: { boost: { "value" : 3" }}}}
    ]}}}

In this case, we're using the text operator to split on word boundaries using the lucene.standard analyzer, and boosting those results above. Results from Atlas Search are automatically sorted by score with top results first. Queries are optimized for performance and this query would be done in one pass.

There are a lot of other knobs in the docs to turn depending on your sorting and querying needs (such as using different analyzers, prefix searches, phrase searches, regex, etc).

score 0 · Answer 2 · answered Aug 29 '20 at 04:39

0

If you want those kinds of ordering rules I would load up all of your customer names into an application that does the search and perform search & sort entirely in the application. I don't expect even Atlas search to provide this kind of flexibility.

(I don't think the queries you provided achieve the ordering you want either.)

answered Aug 29 '20 at 04:39

D. SM

13,584
3
12
21

The order of the first two dont matter as they can be interchanged, what matters is that the "words" come before the sentences, – user1152262 Aug 31 '20 at 00:06
If that is the case update your question to clearly state the requirements. – D. SM Aug 31 '20 at 02:31

Custom search using Mongodb

2 Answers2