1

I am trying to make a POST with some data and the cookies received before with a GET. I had a code with PyCurl that works correctly but I need to use Requests. However, the using Request does not work although I am doing the same steps. The working code on PyCurl is:

import time
import urllib
import pycurl

curl = pycurl.Curl()
curl.setopt(pycurl.URL, 'https://www.myURL.com/header?time='+str(int(time.time())))
curl.setopt(pycurl.COOKIEJAR, 'cookie_1.txt')
curl.perform()
curl.close()

curl = pycurl.Curl()
curl.setopt(pycurl.URL, 'https://www.myURL.com/login')
post_data = {'actualURL': 'http://www.myURL.com', 'user': 'myemail@gmail.com', 'pass': 'asdfqwerty'}
curl.setopt(pycurl.COOKIEFILE, 'cookie_1.txt')
curl.setopt(pycurl.POSTFIELDS, urllib.urlencode(post_data))
curl.setopt(pycurl.COOKIEJAR, 'cookie_2.txt')
curl.perform()
curl.close()

I receive in 'cookie_2.txt' file the correct answer with new data on cookies. However, as said before, the following code give me a response of Error 403

import time
import urllib
import requests

r1 = requests.get('https://www.myURL.com/header?time='+str(int(time.time())))
payload = {'actualURL': 'http://www.myURL.com', 'user': 'myemail@gmail.com', 'pass': 'asdfqwerty'}
r2 = requests.post('https://www.myURL.com/login', data=urllib.urlencode(payload), cookies=r1.cookies)

As you can see, I use the same cookies received from previous request as mentioned here. I checked 'r1.cookies' and are set correct. Also, as POST data has special characters I used an encoding from urllib (PyCurl does not work without this).

I firstly thought that the problem on 'Requests' comes from special characters on POST data, but I have tried with encoding, without encoding, as a json string,... but nothing works. 'r2.cookies' is empty

Community
  • 1
  • 1
iblasi
  • 1,269
  • 9
  • 21

1 Answers1

1

If you neeed to handle cookies , i recommend requests.Session() , example : 

ses = requests.Session()  # use this object for all get / post requests #

payload = {'actualURL': 'http://www.myURL.com', 'user': 'myemail@gmail.com', 'pass': 'asdfqwerty'}
r1 = ses.get('https://www.myURL.com/header?time='+str(int(time.time())))
r2 = ses.post('https://www.myURL.com/login', data=payload)

print(ses.cookies.get_dict())

You can find more info here : http://docs.python-requests.org/en/master/user/advanced/

t.m.adam
  • 15,106
  • 3
  • 32
  • 52
  • the cookies are still empty. I tried with different POST-data options but it doesn't work. Just for your information, first it is necessary to make a GET for extract a JSESSION and the with that cookies you can make the POST, so the order of your answer is flipped. – iblasi May 04 '17 at 11:28
  • Usually you need to first send your creds to get authenticated and then continue, that's why i placed the POST first. My code is just an example , i dont know how the actual website handles authentication and cookies , but i guess it could be another HTTP header that is missing ( User-Agent, Referer, etc ) . You 'll need to inspect the network traffick and see what headers you send when you get cookies successfully with a web browser or pycurl – t.m.adam May 04 '17 at 11:58
  • Anyway , i updated the code to match the order you send your requests, try again . If it still prints an empty dict , try with 'User-Agent' in `headers` , and make sure your creds are right . I can't help you anymore unless i have the actual url . – t.m.adam May 04 '17 at 23:11
  • Thank you for updating the code, because it makes me realize that I was looking to `r2.cookies` and not to `ses.cookies`. I have checked and yes it works without `urlencode` as you write it on the example. Thanks! – iblasi May 07 '17 at 07:04