I am using django-oscar e-commerce, backed with whoosh implementation for haystack searching. My shop contains of almost 70k items:
>>> from catalogue.models import Product
>>> Product.objects.all().count()
69084
but apparently the index contains only 21000 docs:
>>> from whoosh.index import open_dir
>>> from whoosh.query import Every
>>> ix = open_dir('whoosh_index')
>>> len(list(ix.searcher().documents()))
21000
>>> len(ix.searcher().search(Every('text')))
21000
Do you have any idea, why the search engine could have indexed only part of my documents (products)? I think this is no coincidence that a round number of 21000 was indexed (and not a random number, such as 24861 for instance) - but I may be wrong at that. Any ideas, where to look for the solution for this problem?