Foreach Loop slow with API search

Question

I have an array of IDs that I am doing foreach loop and searching each ID in a SOLR index using Php Apache SOLR client. Its slow like a dead turtle. Any help appreciated in optimizing this

    foreach ( $f_games as $game_id ){   
        $game_type = BKT_PLUGIN_CLASS::tv_regions($game_id);
        //Do my stuff
        $count++;
    }

Where

BKT_PLUGIN_CLASS::tv_regions

is my class method for SOLR API search ( which works fine, no issues there ).

So its doing what i want it to do. It takes each ID and goes to SOLR and brings the result of that item and I do what I want to do and increase count. With only 200+ IDs, it takes more than 2 minutes to spit out results.

score 2 · Accepted Answer · answered Mar 15 '16 at 11:32

Use Result Grouping in Solr - that way you can get x number of hits for each region, all rolled up into a single response. Tweak the number of groups and number of hits for each group to match your need.

Filter the list by having a fq with all the values in, so that it returns the documents you need, then group by the value you'd normally search for.

score 0 · Answer 2 · edited May 23 '17 at 12:15

0

Why are You pinging API for each game? You are loosing lot of time just to connect there ... Can't You just pass all IDs and just count result? I don't know SOLR well but for me it's insane that it's not possible (so I assume that it is doable) How to do an IN query in Solr? How can I search on a list of values using Solr/Lucene?

edited May 23 '17 at 12:15

Community

1
1

answered Mar 14 '16 at 16:53

barat

1,040
8
14

I can't. It lets me search and get 10 results at a time even if I send all 200+ IDs at a time, I get 10 back. – Wali Hassan Mar 14 '16 at 18:24
1

@WaliHassan add `&rows=XXX` to your query where XXX is the maximum number of results you want. – Rick Sullivan Mar 14 '16 at 21:32

Foreach Loop slow with API search

2 Answers2