Strange error adding to Whoosh index

Question

Can anyone help me with this strange error I'm getting when adding a new document to a Whoosh index?

Here's the code:

def add_to_index(self, doc):
    ix = index.open_dir(self.index_dir)
    writer = AsyncWriter(ix) # use async writer to prevent write lock errors
    writer.add_document(**self.get_doc_args(doc))
    writer.commit()

def get_doc_args(self, doc):
    return {
        'id':        u""+str(doc['id']),
        'org':       doc['org__id'],
        'created':   doc['created_date'],
        'date':      doc['received_date'],
        'from_addr': doc['from_addr'],
        'subject':   doc['subject'],
        'body':      doc['messagebody__cleaned_message']
    }

I get the following error:

TypeError('ord() expected a character, but string of length 0 found',)
Traceback (most recent call last):
  File "/usr/local/lib/python2.6/dist-packages/celery/execute/trace.py", line 36, in trace
    return cls(states.SUCCESS, retval=fun(*args, **kwargs))
  File "/usr/local/lib/python2.6/dist-packages/celery/app/task/__init__.py", line 232, in __call__
    return self.run(*args, **kwargs)
  File "/usr/local/lib/python2.6/dist-packages/celery/app/__init__.py", line 172, in run
    return fun(*args, **kwargs)
  File "/mnt/deploy/prod/chorus/src/chorus/../chorus/search/__init__.py", line 131, in index_message
    MessageSearcher().add_to_index(message)
  File "/mnt/deploy/prod/chorus/src/chorus/../chorus/search/__init__.py", line 29, in add_to_index
    writer.commit()
  File "/usr/local/lib/python2.6/dist-packages/whoosh/writing.py", line 423, in commit
    self.writer.commit(*args, **kwargs)
  File "/usr/local/lib/python2.6/dist-packages/whoosh/filedb/filewriting.py", line 501, in commit
    new_segments = mergetype(self, self.segments)
  File "/usr/local/lib/python2.6/dist-packages/whoosh/filedb/filewriting.py", line 78, in MERGE_SMALL
    reader = SegmentReader(writer.storage, writer.schema, seg)
  File "/usr/local/lib/python2.6/dist-packages/whoosh/filedb/filereading.py", line 63, in __init__
    self.termsindex = TermIndexReader(tf)
  File "/usr/local/lib/python2.6/dist-packages/whoosh/filedb/filetables.py", line 590, in __init__
    super(TermIndexReader, self).__init__(dbfile)
  File "/usr/local/lib/python2.6/dist-packages/whoosh/filedb/filetables.py", line 502, in __init__
    OrderedHashReader.__init__(self, dbfile)
  File "/usr/local/lib/python2.6/dist-packages/whoosh/filedb/filetables.py", line 379, in __init__
    HashReader.__init__(self, dbfile)
  File "/usr/local/lib/python2.6/dist-packages/whoosh/filedb/filetables.py", line 187, in __init__
    self.hashtype = dbfile.read_byte()
  File "/usr/local/lib/python2.6/dist-packages/whoosh/filedb/structfile.py", line 219, in read_byte
    return ord(self.file.read(1))

Strangely, the exact same code using a standard writer (i.e. not AsyncWriter) works just fine. What am I missing here? Note that in production I have to use AsyncWriter in order to avoid LockErrors.

I should probably add that I've verified all the inputs are unicode strings. — kuhnza, Oct 31 '11 at 02:52

score 2 · Answer 1 · answered Jun 29 '15 at 11:09

2

This error is caused by some kind of index corruption. In my case the machine crashed by another reason while index was being rebuild.

You can easily solve it by deleting whoosh_index folder contents completely and rebuilidng index.

answered Jun 29 '15 at 11:09

n3storm

704
8
21

Deleting the whoosh index and rebuilding it fixed it for me as well. – Mahn Mar 23 '17 at 18:06

score -4 · Accepted Answer · answered Oct 31 '11 at 23:03

-4

Ended up finding a solution; it's called Solr :-)

answered Oct 31 '11 at 23:03

kuhnza

731
1
5
15

As a comment is Ok, but is not a solution to the error you specified. – n3storm Mar 29 '17 at 10:44

Strange error adding to Whoosh index

2 Answers2