Reserved words in Google Custom Search API

Question

I am working with the Google Search API, and I am running into some trouble. This request (in Python, using the requests library) works fine

res = requests.get("https://www.googleapis.com/customsearch/v1", params={
    "cx": <key1>,
    "key": <key2>,
    "alt": "json",
    "num": 2,
    "q": "cat sock ship hero monkey baby match"
})

and returns results with the syntax according to the documentation

However, this request does not work:

res = requests.get("https://www.googleapis.com/customsearch/v1", params={
    "cx": <key1>,
    "key": <key2>,
    "alt": "json",
    "num": 2,
    "q": "cat sock ship hero monkey footnoteref baby match"
})

it returns this:

{'kind': 'customsearch#search',
 'queries': {'request': [{'count': 2,
    'cx': '<key>',
    'inputEncoding': 'utf8',
    'outputEncoding': 'utf8',
    'safe': 'off',
    'searchTerms': 'cat sock ship hero monkey baby footnoteref match',
    'title': 'Google Custom Search - cat sock ship hero monkey baby footnoteref match',
    'totalResults': '0'}]},
 'searchInformation': {'formattedSearchTime': '0.22',
  'formattedTotalResults': '0',
  'searchTime': 0.218722,
  'totalResults': '0'},
 'spelling': {'correctedQuery': 'cat sock ship hero monkey baby footnote ref match',
  'htmlCorrectedQuery': 'cat sock ship hero monkey baby <b><i>footnote ref</i></b> match'},
 'url': {'template': 'https://www.googleapis.com/customsearch/v1?q={searchTerms}&num={count?}&start={startIndex?}&lr={language?}&safe={safe?}&cx={cx?}&sort={sort?}&filter={filter?}&gl={gl?}&cr={cr?}&googlehost={googleHost?}&c2coff={disableCnTwTranslation?}&hq={hq?}&hl={hl?}&siteSearch={siteSearch?}&siteSearchFilter={siteSearchFilter?}&exactTerms={exactTerms?}&excludeTerms={excludeTerms?}&linkSite={linkSite?}&orTerms={orTerms?}&relatedSite={relatedSite?}&dateRestrict={dateRestrict?}&lowRange={lowRange?}&highRange={highRange?}&searchType={searchType}&fileType={fileType?}&rights={rights?}&imgSize={imgSize?}&imgType={imgType?}&imgColorType={imgColorType?}&imgDominantColor={imgDominantColor?}&alt=json',
  'type': 'application/json'}}

The only difference between the two queries is that the latter has the word "footnoteref" in it. I did not find in the documentation anything about this word and its impact on the API's behavior. What is happening? Is there a way to disable this behavior, or a list of reserved words? For now, I am just going to remove the offending word from the query, but I am afraid I am going to play a whack-a-mole game of removing words each time other offending word pops out.

Are you absolutely sure there is a page in the domain you're searching that contains all of these words? If you go to google.com, put `cat sock ship hero monkey footnoteref baby match site:` in the search bar, do you get any results? — Mike Patrick, Nov 17 '17 at 23:30
@MikePatrick If I put those words on the google site by hand, it returns results for the corrected words. But, still, I don't search in a custom site, I am trying to use the api as a human would do, without a site. — user4052054, Nov 18 '17 at 00:31
I see. I get a single result when I try this, using the API explorer *or* python w/`requests`. I did notice that adding `site:*` to the query in the browser was initially giving me an empty set, but is now returning this SO question as the only result. Using the API, the single result I get is this PDF: http://purl.access.gpo.gov/GPO/LPS9006 Odd behavior indeed. If you add `"fileType": "pdf"` to your `params`, do you still come up empty? — Mike Patrick, Nov 18 '17 at 15:27

score 4 · Accepted Answer · answered Nov 23 '17 at 10:19

I searched on google for both "cat sock ship hero monkey footnoteref baby match" and "cat sock ship hero monkey baby match".

You said that "cat sock ship hero monkey footnoteref baby match" doesn't return anything, and that's because Google actually suggests a different search: 'cat sock ship hero monkey baby footnote ref match'.

When you don't have results, you should remove a word from the search (I will start with the last word) and try again. Or you should just try with the suggested search, like: 'cat sock ship hero monkey baby footnote ref match'.

The search works fast, I suggest you to implement the following technique:

a) Your search contains less than 3-4 words. You should repeat the search but add a new word from google's 'correctedQuery' suggestion.
b) Your search contains more than 4 words. You should remove the last word or a "link word" like "for", "and".. and repeat the search.

Good luck.

Reserved words in Google Custom Search API

1 Answers1