0

I have table of URLs that I want to check are still valid. So loop through them using a simple try / except:

try:
    status = requests.get(url).status_code
except:
    status = '999'

This works for most cases but a handful of URLs stop the flow. For example the following URL just hangs:

requests.get('https://www.moneycorpcard.com/').status_code

So I used another answer "Correct way to try/except using Python requests" to come up with the following:

try:
    status = requests.get(card[2]).status_code
except requests.exceptions.Timeout:
    status = 'timeout'
except requests.exceptions.TooManyRedirects:
    # URL was bad 
    status = 'bad url'
except requests.exceptions.RequestException as e:
    status = e
    sys.exc_clear()
    pass

But this too hangs at same URL (above).

Any ideas on what is happening with Requests and this particular URL above, and how to get Requests to keep moving when it encounters these types of situations?

curtisp
  • 2,227
  • 3
  • 30
  • 62

1 Answers1

1

Try to use timeout parameter for your get method call as described here http://docs.python-requests.org/en/master/user/quickstart/#timeouts

Volodymyr Gubarkov
  • 2,173
  • 20
  • 20
  • Setting timeout value did the job. Updated to: `status = requests.get(card[2], timeout=30).status_code` Now it moves through all URLs. – curtisp Nov 11 '17 at 17:52