3

I use haystack with whoosh to search my company's code, every file per index. Most of the time, it works fine. However, when I search with some common keyword like 'port', the seaching takes 5 mins to finish, and the CPU is 100%. I think this is because the result count is too large(5000+ for keyword port). Indeed, I don't need so many results, maybe 100 or so is enough. How can I set the limit? or timeout is also OK.

Danny Beckett
  • 20,529
  • 24
  • 107
  • 134
guoqiao
  • 1,309
  • 12
  • 14

3 Answers3

2

Yes you can, there's nothing special about the default SearchView class and its methods. Here's an example how-to:

yourapp urls

# Django specific
from django.conf.urls import *

# App specific
from yourapp.search.views import AdvancedSearchView

urlpatterns = patterns('yourapp.views',
    url(r'^$', AdvancedSearchView(), name='yourapp_search'),
)

views

# Django specific
from django.conf import settings

# App specific
from haystack.views import SearchView

LIMIT = getattr(settings, 'HAYSTACK_MAX_RESULTS', 50)

class AdvancedSearchView(SearchView):
    def get_results(self):
        return self.form.search()[:LIMIT]

project urls

urlpatterns += patterns('',
    url(r'^search/$', include('yourapp.search.urls'))
)

You can override the global in your settings:

settings.py

HAYSTACK_MAX_RESULTS = 15
Hedde van der Heide
  • 21,841
  • 13
  • 71
  • 100
0

Maybe you should have a look at time limited searches. Combined with an limited Collector instance, you should be able to solve your problem.

ercpe
  • 342
  • 4
  • 9
0

Have a look at django's pagination - if you implement that, it will only show a subset of results per page and shouldn't use so much CPU. That said, you might want to try using another backend if you have thousands of items in your search index - I've found whoosh ideal for small indexes (up to about 1000 items) but a bit slow beyond that.

Greg
  • 9,963
  • 5
  • 43
  • 46
  • I know django's pagination, and I use it in py app. The problem is that before I can show my results, whooh takes too much time to query them. What I really want is something like this: results = foo.search(max_result_count=100). I want whoosh to just return as long as it hit the 100th results. But now, It keep going on to get all results as large as 5000+. – guoqiao Mar 29 '12 at 01:42
  • Ok, I see what you mean - is this even possible with Whoosh? I'd be confident that if you use haystack to request a subset of results, the whoosh backend would do it in the most efficient way possible. Might need to look at a different backend - Xapian is similiar to whoosh but I think it will handle this situation a bit more efficiently. – Greg Apr 11 '12 at 23:56
  • If you want to do this depending on the backend you will have to dig deeper in Haystack's logic, but I have added an example to simply overwrite the default SearchView. – Hedde van der Heide Dec 20 '12 at 15:00