I am trying to scrap this website for fund price history, by providing the start date, end date and click the 'Get Prices' button via POST method.
However the page requests.post() return does not contain the results, as if the "Get Price" button was never pressed. This is the URL I put together through the code:
I read other posts on Stackoverflow about submitting form via POST in Python and I couldn't make it work. Could you please help? Many thanks!
import requests
import datetime
startDate = datetime.datetime(2016,1,1).strftime('%m/%d/%Y')
endDate = datetime.datetime(2016,2,20).strftime('%m/%d/%Y')
serviceurl = 'https://www.nysaves.org/nytpl/fundperform/fundHistory.do?'
payload = {'fundid':1003022, 'startDate':startDate, 'endDate': endDate, 'submit':'Get Prices'}
r = requests.post(serviceurl, params=payload)
#from IPython.core.display import HTML
#HTML(r.content.decode('utf-8'))