2

I have been trying to search videos using bing search engine. But every-time I try I get error HTTPError:HTTPError 403:Forbidden

import urllib
import urllib2
import json

def main():
    query = "'pyscripter'"
    print bing_search(query, 'Video')

def bing_search(query, search_type):
    #search_type: Web, Image, News, Video
    key= 'LsE7jElMmTDfbrnCEmrCmCEBbaPxMG5BvKr9CsfmSNS'
    query = urllib.quote(query)
    #create credential for authentication
    user_agent = 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; FDM; .NET CLR 2.0.50727; InfoPath.2; .NET CLR 1.1.4322)'
    credentials = (':%s' % key).encode('base64')[:-1]
    auth = 'Basic %s' % credentials
    url = 'https://api.datamarket.azure.com/Data.ashx/Bing/Search/'+search_type+'?Query=%27'+query+'%27&$top=5&$format=json'
    request = urllib2.Request(url)
    request.add_header('Authorization', auth)
    request.add_header('User-Agent', user_agent)
    request_opener = urllib2.build_opener()
    response = request_opener.open(request)
    response_data = response.read()
    json_result = json.loads(response_data)
    result_list = json_result['d']['results']
    print result_list
    return result_list

if __name__ == '__main__':
    main()

The error shown is: 

Traceback (most recent call last):
File "<module1>", line 30, in <module>
File "<module1>", line 7, in main
File "<module1>", line 22, in bing_search
File "C:\Python27\lib\urllib2.py", line 410, in open
    response = meth(req, response)
File "C:\Python27\lib\urllib2.py", line 523, in http_response
'http', request, response, code, msg, hdrs)
File "C:\Python27\lib\urllib2.py", line 448, in error
    return self._call_chain(*args)
File "C:\Python27\lib\urllib2.py", line 382, in _call_chain
result = func(*args)
File "C:\Python27\lib\urllib2.py", line 531, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 403: Forbidden

Before trying this I worked with YouTube search API which worked fine. But the only problem was that it was limited to the videos present in YouTube database. What I want is the list of URL's of all the videos related to the keyword present in internet. So I started with Bing search engine. Any help regarding this would be appreciated.

Aaron Misquith
  • 621
  • 1
  • 7
  • 11
  • I ran this code and got this error instead: `urllib2.HTTPError: HTTP Error 401: The authorization type you provided is not supported. Only Basic and OAuth are supported` – nofinator Sep 12 '14 at 17:22
  • I suggest to use Google Chrome, get the network traffic as a cURL command, and try it. Then, use Python Request will make your life easier. – Lovato Sep 12 '14 at 17:46
  • @Lovato I have no knowledge on using cURL. This was an offshoot of [this program](http://xiaoyao-web-hacks.blogspot.de/2012/09/use-python-to-access-windows-azure-bing.html) – Aaron Misquith Sep 12 '14 at 18:18
  • @nofinator I'm using Basic in this right? The error remains the same. – Aaron Misquith Sep 12 '14 at 18:20
  • You dont need to know curl. Just copy the cURL command from Chrome, and paste it to your terminal, and see if it works. It MUST work. Then you mimic all headers, payload and stuff to (as I suggested) Requests module. I did it myself last week helping other user here. – Lovato Sep 12 '14 at 18:27
  • Here is my help to him: http://stackoverflow.com/questions/25723465/post-requests-to-asp-site-dont-bring-a-good-result/25723579?noredirect=1#comment40216035_25723579 – Lovato Sep 12 '14 at 18:27

1 Answers1

0

I had save issue,

A web server may return a 403 Forbidden HTTP status code in response to a request from a client for a web page or resource to indicate that the server can be reached and understood the request, but refuses to take any further action. Status code 403 responses are the result of the web server being configured to deny access, for some reason, to the requested resource by the client.

in my case, I forgot to activate "bing search" subscription, so go to "https://datamarket.azure.com/dataset/bing/search" and activate "bing search" subscription

user1455502
  • 21
  • 2