Complex Google App Engine Search

Question

A couple quick questions related to GAE search and datastore:

(1) Why is it that I can inequality filter on more than one property using the search service, but that I can only inequality filter on at most one property when querying the datastore? It seems odd that this limitation would exist in one service but not the other.

(2) I intend to use google app engine search to query very many objects (thousands or hundreds of thousands, maybe more). I plan to be doing many inequalities, for example: "time created" before x, "price" greater than y, "rating" less than z, "latitude" between a and b, "longitude" between c and d etc. This seems like a lot of filters and potentially expensive. Is App Engine Search an appropriate solution for this?

Thanks so much.

Answer 1

1) The SearchService basically gives you an API to perform the sorts of things you can't using the datastore. If you could do them on the datastore, you wouldn't really need the SearchService. While not a very satisfactory answer, many of the common operations you might do with a traditional RDBMS were not really even possible before the Search API was available.

2) is a bit harder. Currently the search api doesn't handle failure conditions very well, usually you'll get a SearchServiceException without a meaningful message. The team seem to have been improving this over the last year or so, although fixes in this space seem to have been coming very slowly. From the tickets I've raised, failures are usually a result of queries running too long. This is usually represented as queries that are too complex. You can actually tune queries quite a lot with combinations of the query string as well as the parameters you apply to your search request. The downside is that its all totally black box, I haven't seen any guides or tools on optimising queries. When they fail, they just fail.

The AppEngine search api is designed to solve the problems you describe, whether in your case it does may be hard to determine. You could set up some sample queries and deploy to a test environment to see if it even basically works for your typical set of data. I would expect that it will work fine for the example you gave. I have successfully been running similar searches in large scale production environments.