Whoosh non-relational data

Question

I have django e-commerse site and I have simple EAV for product values.

Elasticsearch can index json data.

How to indexing with whoosh producs with different number of keys ? And how about facet search?

Answer 1

I was trying to solve this problem. I translated my EAV model instances into plain dicts (so every dict contains both usual and optional (EAV) fields) and passed it to whoosh. Passing different number of fields to whoosh is okay:

You don’t have to fill in a value for every field. Whoosh doesn’t care if you leave out a field from a document.

Be careful though! When you create a whoosh Schema, you should list all possible fields you will use, including all those EAV ones from all instances.

The results were disappointing. I compared facets by whoosh (created as described above) and facets written by myself (using django orm, python code, etc). I expected whoosh to be certainly faster than any hand-written solution, but that wasn't true. After measuring everything I found whoosh facets working 1.5 times slower than my own ones! Moreover, using django's orm gave me more control on the results (whoosh returned IDs and I had to translate them into django instances, and my own facets returned instances by default).

Of course, implementation matters, and I could possibly make some optimizations, but anyway I was expecting rough whoosh implementation to be much faster than orm.