Searching for whole URL using elastic4s

Question

I'm using elastic4s in order to index and search data in ES. Part of the document I'm storing contains a url field which I need to search for (the entire url). The problem occurs when I search for a document containing the url field and get 0 results.

For the purpose of this search I define a mapping ahead of inserting the data into the index:

client.execute {
  create index <my_index> mappings {
    <my_type> as {
      "url" typed StringType analyzer NotAnalyzed
    }
  }
}

I'm inserting the data:

client.execute { index into <my_index> -> <my_type> fields (
  "url" -> "http://sample.com"
  )
}.await

And I search for the documents:

val filter =
"""
  |{
  |    "filtered" : {
  |        "query" : {
  |            "match_all" : {}
  |        },
  |        "filter" : {
  |            "bool" : {
  |              "should" : [
  |                { "bool" : {
  |                  "must" : [
  |                     { "term" : { "url" : "http://sample.com" } }
  |                  ]
  |                } }
  |              ]
  |            }
  |        }
  |    }
  |}
""".stripMargin

client.execute { search in <my_index> -> <my_type> rawQuery { filter } }.await

I get 0 results after executing this query. What am I doing wrong?

Thanks!

anatolyr · Answer 1 · 2015-09-24T11:42:17.140

Problem solved.

In the mapping, instead of doing:

client.execute {
  create index <my_index> mappings {
    <my_type> as {
      "url" typed StringType analyzer NotAnalyzed
    }
  }
}

I should have done:

client.execute {
  create index <my_index> mappings {
    <my_type> as {
      "url" typed StringType index "not_analyzed"
    }
  }
}

or

client.execute {
  create index <my_index> mappings {
    <my_type> as {
      "url" typed StringType index NotAnalyzed
    }
  }
}

score 0 · Answer 2 · answered Sep 22 '15 at 10:53

0

I think your query can be simplified a bit, to something like:

{
  "query": {
    "term": {
      "url": {
        "value": "http://example.com"
      }
    }
  }
}

Weirdly, I would expect your query to return all documents, due to the nesting of your boolean queries. should means documents that match any query in the array are scored higher than those not matching. So, a should containing only one must should return all documents, with matching documents scored higher than non matching.

answered Sep 22 '15 at 10:53

rahilb

726
3
15

You're right regarding simplifying the query. The actual query is a bit bigger, I just wanted it to maintain it's structure for purposes of reproduction. At first, it wasn't returning results because I didn't define a mapping for the 'url' field and did not tell ES it was a non analyzed field. So, the standard ES analyzer broke the url into several tokens and didn't match on the entire field. After telling it it was not a analyzed field, it should have matched on the entire field. – anatolyr Sep 24 '15 at 07:43

Searching for whole URL using elastic4s

2 Answers2