Retrieve more than 1000 results Google App Engine Search API

Question

I need to be able to retrieve all records available in the index. Seems like 1000 is the limit. Is there something else I can do?

Answer 1

I was also facing some similar issue in one of my projects so researched over internet and got one idea that instead of using the Search API, I created a workaround scenario. What I did is I have only one attribute in my table which needs to have a pattern based search. I am sharing my code as well here.

Objectify Entity Class

@Entity
public class NewsFeed {
@Id
@Index
private Long feedID;
private String title;
private Set<String> titleKeywords;

// getters and setter
}

Logic for storing the keyword in the same table. I have split all the title words for my entity into keywords, and stored them in a Set Object.

NewsFeed newsFeed = new NewsFeed();
newsFeed.setTitle(title);
newsFeed.setTitleKeywords(getKeywordsSet(newsTitle));
//  save entity here

Method for extracting keywords from title(field to be searched)

public Set<String> getKeywordsSet(String title) {
    Set<String> keywords = new HashSet<String>();
    String titleNews = title.toLowerCase();
    String[] array = titleNews.split(" ");
    for (int i = 0; i < array.length; i++) {
        // replacing all special characters here
        String word = array[i].replaceAll("\\W", "");
        keywords.add(word);
    }
    return keywords;
}

Listing all the feeds from our DB, and finally matching the parameter to be searched, by the logic below.

public List<NewsFeed> getFilterJsonArray(String param){
// Listing all the objects of entity
    List<NewsFeed> list = newsFeedDao.listOrderedFeeds();
    List<NewsFeed> matchedObject = new ArrayList<NewsFeed>();
    for (NewsFeed newsFeed : list) {

/**
* main logic for pattern matched keywords
**/
        if (isAnElementInSet(newsFeed.getTitleKeywords(), param.toLowerCase())) {
            matchedObject.add(newsFeed);
        }
    }
    return matchedObject;
}

public boolean isAnElementInSet(Set<String> keywords, String param) {
    String []params = param.split(" ");
    if (keywords.size() > 0) {
        for(String splittedParam : params){
            if (keywords.contains(splittedParam)) {
                return true;
            } else{
                for (String keyword : keywords) {
                    if(keyword.contains(splittedParam)){
                        return true;
                    }
                }
                return false;
            }
        }
        return true;
    }else{
        return false;
    }
}

I know this cannot be the best solution for searching things but this solution worked very fine for me. I just shared it here so as to get improvements in this logic as well.