0

I am evaluating a couple different libraries to see which one will best fit what I need.

Right now I am looking at Bleve, but I am happy to use any library.

I am looking to index full files except specific ones which are in XML format. For those I only want Bleve to index specific tags as most of the tags are worthless to search. I am trying to evaluate if this is possible but, being new to Bleve, I am not sure what part I need to customize.

The documentation is very good, but I can't seem to find this answer. All I need is an explanation with keywords and steps, no code is required, I just need a push as I have spent hours spinning my wheels with google searches and I am getting no where.

themihai
  • 7,903
  • 11
  • 39
  • 61
Krum110487
  • 611
  • 1
  • 7
  • 20

1 Answers1

1

There are probably many ways to approach this. Here's one.

Bleve indexes documents which are collections of key/value metadata pairs.

In your case, a document could be represented by 2 key/value pairs: name of .xml file (to uniquely identify the document) and content of the file.

type Doc struct {
    Name string
    Body string
}

The issue is that body is XML and Bleve doesn't support XML out-of-the-box.

A way to address it would be to pre-process XML file by stripping unwanted tags and content. You can do it using encoding/xml standard library.

For an example of a similar task you can see the code of https://github.com/blevesearch/fosdem-search/

In there they index file in custom format (https://github.com/blevesearch/fosdem-search/blob/master/fosdem.ical) by parsing it into a format they can submit to Bleve for indexing (https://github.com/blevesearch/fosdem-search/blob/master/ical.go).

Krzysztof Kowalczyk
  • 3,513
  • 28
  • 28