Sphinx morphology stem_en not working

Question

I have a single-field Sphinx index with stemming set up as follows:

index main_sphinxalert
{
            # Options:
            type = rt
            path = /var/lib/sphinxsearch/data/main_sphinxalert
            morphology = stem_en

            # Fields:
            rt_field = query
}

I then insert:

mysql> INSERT INTO main_sphinxalert VALUES(201,'tables');
Query OK, 1 row affected (0.00 sec)

And select:

mysql> SELECT * FROM main_sphinxalert WHERE MATCH('tables');
+------+--------+
|  id  | weight |
+------+--------+
| 201  |  1709  |
+------+--------+
1 row in set (0.00 sec)

But can't select on a stem:

mysql> SELECT * FROM main_sphinxalert WHERE MATCH('table');
Empty set (0.00 sec)

Can anyone tell me what gives?

Are you sure your SQL query is running against Sphinx's stemmed index and not against MySQL's native full text index? — jsalonen, Apr 22 '13 at 12:33
Did you change the config after first creating the index? Sphinx ignores changes to the config after teh index has been created. — barryhunter, Apr 22 '13 at 12:34
@barryhunter - Good call - yes I did. I was under the impression that restarting the sphinxsearch service would cause it to re-read the config file. Is there a way to force a re-read without creating a new RT index? — awidgery, Apr 22 '13 at 12:55
@jsalonen Yes definitely connected to Sphinx's index using Sphinx's MySQL interface eg. `mysql -h 127.0.0.1 -P 9306` — awidgery, Apr 22 '13 at 12:56
According to [this post](http://www.sphinxconnector.net/Tutorial/IntroductionToRealTimeIndexes) you have to delete the index files. This worked for me (I had to stop the sphinxsearch service to do so) as I could afford to lose data. If someone knows of a better way without data loss or manually re-indexing please leave a comment and I'll update my answer. — awidgery, Apr 22 '13 at 13:08
No, there is no better way. You have to completely destroy the index files (which does really require searchd being shutdown), so that they can then be recreated from scratch. (basically you cant change the settings of an index, once its been created at all) — barryhunter, Apr 22 '13 at 13:49
btw, re 'data-loss' you should definitly not use sphinx as your primary data. Only input a copy of the data. For the most part you can't get the data back out (indexing building is a one way conversion of data). So the index should always be considered disposable. — barryhunter, Apr 22 '13 at 13:52

score 4 · Accepted Answer · answered Apr 22 '13 at 13:14

Turns out Sphinx will not read modifications to config files for a realtime index after it's been created: see here for info.

The way around this is to:

sudo service sphinxsearch stop

Then delete all index files (for me held in /var/lib/sphinxsearch/data/):

sudo rm main_sphinxalert.*

Then restart service:

sudo service sphinxsearch start

This means you lose your existing index, which will then have to be manually rebuilt from your database.

Sphinx morphology stem_en not working

1 Answers1