Sphinx: Can one update the limit for SQL query size used for indexing?

Question

I seem to have hit a certain Sphinx head case. I'm indexing a certain table, which will produce ≈ 140 indexed fields per record (trust me, they are all important). For 27 * 3 of them, the sub-query which produces it is in itself already quite big. This results in a huge massive query being generated to my development.sphinx.conf (17 lines). Which produces results, I've tested it directly in the db. But which can't index. It complains

"ERROR: index 'vendor_song_core': sql_query_range: : macro '$start' not found in match fetch query."

, but what this really means is that the deamon is not loading the full query. Apparently it is too long for it. Is my assumption right? And if so, can I work around it (like, a magical max_query_length field I can update somewhere)?

Can you paste your source and index config? – Miro Dec 10 '12 at 19:00 — Miro, Dec 10 '12 at 19:00

score 0 · Answer 1 · answered Dec 13 '12 at 15:46

0

Answer copied from the Sphinx forum... http://sphinxsearch.com/forum/view.html?id=10403

Move the 'long' query definition into a mysql VIEW.

Then the sql_query can be really short :)

I.e. the view itself, contains all the column names, the sql_query can just use "SELECT * FROM". Similly if joining lots of tables - that can all move into the view.

answered Dec 13 '12 at 15:46

barryhunter

20,886
3
30
43

the question (not answer) was posted by me in the sphinx forum. And I cannot put it into a view. It all has to do with "sorting" columns, which take a certain "customized" query which can't be really translated into a view. But anyway, I already worked a solution around. I still think maybe sphinx should take that into account that the indexing query does have an unstated limit, and should work around the issue accordingly. – ChuckE Dec 13 '12 at 21:48

score 0 · Accepted Answer · answered Jan 17 '13 at 13:54

It seems there is no real way of doing this. Sphinx defines the limit for the query size directly in its source code, so the only way of doing this is either by editing its source code and compile it locally, or do as barryhunter stated, as long as it is possible for you to define such a view. More details about this issue can be addressed on the link provided by barryhunter.

Sphinx: Can one update the limit for SQL query size used for indexing?

2 Answers2