How to fix urllib.error.HTTPError: HTTP Error 429: Too Many Requests

Question

I use this code for get description about hotel from booking website.

Code:

from flask import Flask, request, jsonify, Response
from functools import wraps
from geopy.geocoders import Nominatim
import requests
from bs4 import BeautifulSoup
from googlesearch import search
flask = Flask(__name__)
@flask.route('/hotel', methods=['POST'])
@requires_auth
def postJsonHandler():
   query = "www.booking.com/hotel/eg/" + data['Hotel Name']
        print(query)
        for j in search(query, tld="co.in", num=1, stop=1):
            print(j)
            r = requests.get(j)
            soup = BeautifulSoup(r.text, "lxml")
            answ = soup.find("div", {"id": "property_description_content"}).text
            print(answ)
        result = ('Hotel Name: ' + data['Hotel Name'], "latitude: " + str(loc.latitude), "longitude: " + str(loc.longitude), "Address: " + str(loc.address), "Description: " + answ)
        return jsonify(result), 200

But i get this error:

urllib.error.HTTPError: HTTP Error 429: Too Many Requests.

Any kind of help please?

Answer 1

You have made too many requests to the remote server, which is now rejecting your requests outright to protect itself from a potential DDoS attack.

So the solution is (most likely) to wait until the remote server forgives your transgression...

Answer 2

I solve this error by set:

time.sleep(1)

before for loop

Answer 3

Adding a User-Agent to the request header could solve your problem.