3

How can I configure my django app to write the haystack/whoosh index to an S3 bucket?

I can't find any information on how to get going with this topic. Incidentally I am using Heroku for deployment.

Django storages looks promising, but documentation is very thin. Unsure if I could set WHOOSH_PATH as a storage object anyway?

Seeing as the haystack whoosh backend file has rules on setup like:

if self.use_file_storage and not os.access(settings.HAYSTACK_WHOOSH_PATH, os.W_OK):
            raise IOError("The path to your Whoosh index '%s' is not writable for the current user/group." % settings.HAYSTACK_WHOOSH_PATH)

line 90- haystack/backends/whoosh_backend.py

EDIT: I gave up on Whoosh (& heard it slower than Solr anyway); though it is perhaps common knowledge, Websolr is a fantastically easy way to get search on your Heroku-deployed Django app.

null
  • 1,137
  • 2
  • 11
  • 29

2 Answers2

2

Disclaimer: I am not familiar with Haystack & Whoosh beyond the quick look I have taken at their sites, but you did ping me on Twitter, so here are my thoughts :)

That being said... it sounds to me like the latency to S3 (even from an EC2 instance) would be prohibitive to storing your search indexes in there.

If it were I, I would save myself the trouble and (if you have not done so already), take a look at the Heroku addons page. You can use any of the following for somewhere between free and cheap for small projects:

  • IndexTank
  • Solr
  • Sphinx

I hope the helps a little!

Adam

Adam Charnock
  • 1,596
  • 1
  • 13
  • 17
  • Disclaimer accepted, won't sue you Adam ;) Thanks for the info, Solr seems like the option I should have gone for all along.. – null Oct 12 '11 at 02:04
0

I don't have recipe for this but you might be interested in looking at how whoosh works around the same situation with Google App Engine.

http://whoosh.ca/whoosh/src/44ed3d8ad098/src/whoosh/filedb/gae.py

Tommaso Barbugli
  • 11,781
  • 2
  • 42
  • 41