I am continuing my quest to build in my database the ability to download files from company's we work with. I use fidler while I login thru a browser & watch what happens while I download a file. Mostly I can build a job to get cookies & then use that to automate download. This website though is complicated & I've determined the one cookie I need using fiddle. I copy the raw text from fiddler to run thru a download & see if it gives me a good file. Now I am trying to get that cookie. When i send my winhttp GET request I get back most of the expected cookies but not the one i want. I get a status 302 - redirection. Now i read that the "location" header tells me to send a second request to the location url. I copy from fiddler. it looks different from my first one but it returns the same cookies. Also my first request, the location header returns a different answer than fiddler. I am not sure how to move forward from here. I've tried adding in the cookies gained in request 1, but i get nothing back in that case. I've turned on & off enable redirects in winhttp.I've tried an XML request(gets almost no cookies). Cany anyone point me in the right direction. What should i Look at in fiddler? my basic request looks like:
Set WinHttpReq = CreateObject("WINHTTP.WinHTTPRequest.5.1")
WinHttpReq.Open "GET", Trim(strURL), False 'post request
WinHttpReq.setRequestHeader "Cache-Control", "no-cache"
WinHttpReq.Option(6) = False 'enable redirects
'WinHttpReq.setRequestHeader "Content-Type", "application/x-www-form-urlencoded"
'WinHttpReq.setRequestHeader "Referer", "https://www.foragentsonly.com/login/" 'in case the server cares
WinHttpReq.setRequestHeader "Accept", "text/html, application/xhtml+xml, image/jxr, */*"
WinHttpReq.setRequestHeader "Accept-Language", "en-US"
WinHttpReq.setRequestHeader "Accept-Encoding", "gzip, deflate"
WinHttpReq.setRequestHeader "Host", "servicing1.foragentsonly.com"
WinHttpReq.setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko"
WinHttpReq.setRequestHeader "Connection", "keep-alive"
WinHttpReq.setRequestHeader "Cookie", strFullCookie
WinHttpReq.Send
strHeaders = WinHttpReq.getAllResponseHeaders()
Thank you in advance for any advice or comments. I can post some info from fiddler as well. -Jim
Edit: fiddler request
GET https://servicing1.foragentsonly.com/xps.web/scripts/DisplayPage.aspx?Page=PolSrv.Common.ExternalEntryPoint&FinalDest=Documents.ArchivePDFList&Location=BunkerEast&SessionType=FAORetrieveDocuments&OTG=qV2pgILPkAQczD-ZtExJiJKEzwpix-4LXL6y4bI8BbDI0UT90&OfferingId=ExecuteUITransactionFAORetrieveDocuments&transNum=&Act=&AccessType=FAO&Brand=&ContentGroup=&Tab=NoTab HTTP/1.1
Accept: text/html, application/xhtml+xml, image/jxr, */*
Accept-Language: en-US
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko
Accept-Encoding: gzip, deflate
Connection: Keep-Alive
Host: servicing1.foragentsonly.com
Cookie: bunch of cookies here
response:
HTTP/1.1 302 Found
Cache-Control: no-cache, no-store
Pragma: no-cache
Content-Type: text/html; charset=utf-8
Expires: -1
Location: https://servicing1.foragentsonly.com/xps.web/scripts/DisplayPage.aspx?Page=Documents.ArchivePDFList&Location=BunkerEast&SessionType=FAORetrieveDocuments&OfferingId=ExecuteUITransactionFAORetrieveDocuments&transNum
Server: Microsoft-IIS/7.0
X-AspNet-Version: 4.0.30319
Set-Cookie:
Set-Cookie:
Set-Cookie:
Set-Cookie: the cookie i want is here
Set-Cookie:
Set-Cookie:
Set-Cookie:
Date: Thu, 13 Feb 2020 15:38:52 GMT
Content-Length: 6070
I removed the actual cookies & modified the displayed URLs. Its worth noting that the cookie i'm looking for is found on a page within the website after we've already logged in. So i'm not sending any password in this request, just cookies found on previous pages. I am getting a batch of http only cookies back on my first request, its the redirect i think that give's me the one i want. Also in my winhttp request i get a location that's different. it looks like:Location: /xps.web/scripts/SessionTimeout/TimeoutFAO.aspx instead of what i posted above.
