It seems surprisingly difficult to run search queries programmatically via an API against the major engines.
- Google doesn't have a general purpose API for its search, apparently and surprisingly. They have a "custom search engine" which is designed to for adding a Google-powered search box to a given site and to return results only from a couple of domains. Their signup page demands the entry of the sites to search. I tried entering ".google.com/" and some variations here, but that's not giving me the same results (in particular no hits when the web search is giving me results) on some obscure terms that I care about.
- Bing search does have an API, but the API doesn't report the total number of hits, unlike their web results. Getting the total number of results is a requirement for my application.
- DuckDuckGo has an API, but it doesn't seem to query the same database as the web search.
- Blekko has an API, but it's rate limited at 1 request/second. I haven't tried asking what they pricing structure is.
- I haven't tried Yahoo.
Note that I'm happy and willing to pay for this, but still I can't find a service. Any help is appreciated.