0

Making a GET request with the python module Requests ends up with strange url:

>>> import requests
>>> r = requests.get("http://t.co/Uspy071j")
>>> print r.url
"http://feeds.feedburner.com/%257Er/LesArdoises/%257E3/bD2JuJagz5I/roxino-cest-tout-vert.html?utm_source=twitterfeed&utm_medium=twitter"

This url ends with an error 400. But using RestKit for the same url, the final_url return the correct value:

>>> import restkit
>>> r = restkit.request("http://t.co/Uspy071j", follow_redirect=True)
>>> print r.final_url
"http://lesardoises.com/6277/roxino-cest-tout-vert.html?utm_medium=twitter&utm_source=twitterfeed"

What is the problem with Requests ?

Piotr Dobrogost
  • 41,292
  • 40
  • 236
  • 366
Namlook
  • 181
  • 11
  • Using `curl` and following three `301` redirections, some HTML is finally retrieved. –  Jan 31 '12 at 12:49

2 Answers2

2

It will work properly if you install the current master branch from https://github.com/kennethreitz/requests.git instead of the latest tagged release.

Requests is incorrectly quoting the tildes in the last URL. Instead of requesting http://feedproxy.google.com/~r/LesArdoises/~3/bD2JuJagz5I/roxino-cest-tout-vert.html?utm_source=twitterfeed&utm_medium=twitter it is requesting http://feeds.feedburner.com/%257Er/LesArdoises/%257E3/bD2JuJagz5I/roxino-cest-tout-vert.html?utm_source=twitterfeed&utm_medium=twitter

I can reproduce this with the latest Requests release (0.10.1) but it seems to be fixed in the unreleased master (and develop) branch.

The commit that fixed this bug was https://github.com/kennethreitz/requests/commit/cb64d311719e627df0f78c8446d40326899206c3

ejucovy
  • 907
  • 9
  • 10
0

Works here:

In [6]: import requests

In [7]: r = requests.get("http://t.co/Uspy071j")

In [8]: r
Out[8]: <Response [200]>

In [9]: print r.url
http://lesardoises.com/6277/roxino-cest-tout-vert.html?utm_medium=twitter&utm_source=twitterfeed