0

I'm stuck with a near success on automating a download process. I'm using fiddler & taking the values & hard coding them for now til it works. The website is secure & i get the cookies OK. However when testing the download i am getting a file of 239K but it won't open. Windows tells me it might be corrupted. File size though should be 250K, so I know its not pulling the whole thing. I was going to try & run fiddler to capture my codes traffic, but interestingly it only captures what I do in a browser.

I have tried:

strFullCookie = "a cookie in here"
strDocLink = "https://gateway.frontlineinsurance.com/pc/service/edge/document/gpa/document/pc:somenumberhere?token=withatokenhere&portalRoute=/AgentDocumentError"

Set WinHttpReq2 = CreateObject("WINHTTP.WinHTTPRequest.5.1")

WinHttpReq2.Open "GET", Trim(strDocLink), False
WinHttpReq2.Option(6) = False          
WinHttpReq2.setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko"
WinHttpReq2.setRequestHeader "Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"
WinHttpReq2.setRequestHeader "Accept-Language", "en-US,en;q=0.9"
WinHttpReq2.setRequestHeader "Connection", "keep-alive"
WinHttpReq2.setRequestHeader "Host", "gateway.frontlineinsurance.com"
WinHttpReq2.setRequestHeader "Referer", "https://gateway.frontlineinsurance.com/"
WinHttpReq2.setRequestHeader "Upgrade-Insecure-Requests", "1"
WinHttpReq2.setRequestHeader "Accept-Encoding", "gzip, deflate"
WinHttpReq2.setRequestHeader "Sec-Fetch-User", "?1"
WinHttpReq2.setRequestHeader "Sec-Fetch-Site", "same-origin"
WinHttpReq2.setRequestHeader "Sec-Fetch-Mode", "Navigate"
WinHttpReq2.setRequestHeader "Cache-Control", "no-cache"
WinHttpReq2.setRequestHeader "Content-Type", "application/pdf"
WinHttpReq2.setRequestHeader "Cookie", strFullCookie

WinHttpReq2.Send

'MsgBox WinHttpReq2.responseBody
'MsgBox WinHttpReq2.responseText

Debug.Print WinHttpReq2.Status
strHeaders = WinHttpReq2.getAllResponseHeaders()
Debug.Print strHeaders

Sleep 2000

         If WinHttpReq2.Status = 200 Then
            Set oStream = CreateObject("ADODB.Stream")
            oStream.Open
            oStream.Type = 1
            oStream.Write WinHttpReq2.responseBody
            oStream.SaveToFile "C:\Users\JCarney\Desktop\DownloadedMail\FPITest.pdf", 2    ' 1 = no overwrite, 2 = overwrite
            oStream.Close
        End If

And i have used:

'Set WinHttpReq2 = CreateObject("MSXML2.serverXMLHttp")
'Set WinHttpReq2 = CreateObject("Microsoft.XMLHTTP"

ServerXMLhttp refturned same result as Winhttp. the other was rejected by the server.

Fiddler Request looks like:

GET https://gateway.frontlineinsurance.com/pc/service/edge/document/gpa/document/pc:somenumberhere?token=sometokehere&portalRoute=/AgentDocumentError HTTP/1.1
Accept: text/html, application/xhtml+xml, image/jxr, */*
Referer: https://gateway.frontlineinsurance.com/
Accept-Language: en-US
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko
Accept-Encoding: gzip, deflate
Host: gateway.frontlineinsurance.com
Connection: Keep-Alive
Cookie: _cookies here

the fiddler response looks like:

HTTP/1.1 200 OK
Date: Thu, 20 Feb 2020 18:45:05 GMT
Content-Type: application/pdf
Connection: keep-alive
Set-Cookie: a cookie
Set-Cookie: a cookie
Server: Apache/2.4.6 (Red Hat Enterprise Linux)
Content-Disposition: filename=Declarations Page.pdf
Vary: Accept-Encoding,User-Agent
Set-Cookie: a cookie
Set-Cookie: a cookie
Cache-Control: max-age=0, must-revalidate
Content-Length: 255177

when i print my returned headers from my code i get:

Cache-Control: max-age=0, must-revalidate
Connection: keep-alive
Date: Thu, 20 Feb 2020 18:49:50 GMT
Transfer-Encoding: chunked
Content-Type: application/pdf
Content-Encoding: gzip
Server: Apache/2.4.6 (Red Hat Enterprise Linux)
Set-Cookie: a good cookie
Set-Cookie: a good cookie
Set-Cookie: a good cookie
Set-Cookie:a good cookie
Vary: Accept-Encoding,User-Agent
Content-Disposition: filename=Declarations Page.pdf

I get the expected 4 cookies but my file is short (a pdf). I'm not sure what else to modify. i've tried opening the file as txt or html, its gibberish, but lots of gibberish. Just not a perfect pdf. Thank you in advance for ideas on things I can tweak.

Jim Carney
  • 87
  • 2
  • 10
  • Checked the file content (open in text editor)? It starts with`%PDF` (make not printable chars visible to eg find a BOM)? Or garbage? Then maybe its compressed, limit encoding to deflate. Seems like you solved [how-to-handle-a-302-error](https://stackoverflow.com/questions/60196763/how-to-handle-a-302-error-using-winhttp-vba), if true, share answer! You tried`WGet`, as it tells us to make request easy in docs? – ComputerVersteher Feb 22 '20 at 21:20
  • Thank you for the reply. I'll check in a text editor Monday or Tuesday and post back. On the other post with the 302 error. I have not. I am guessing it is something with the website & that i need to send a request to an earlier page & work my way forward,my guess is that the server is smart & expects the requests in order to authenticate. It is a very request intensive site so I moved on for now. – Jim Carney Feb 23 '20 at 22:22
  • That completely worked! Thank you very much! File comes down right size. In the text editor it was just gibberish. I fixed this line only: WinHttpReq2.setRequestHeader "Accept-Encoding", "gzip, deflate" to WinHttpReq2.setRequestHeader "Accept-Encoding", "deflate" & i was good to go! – Jim Carney Feb 25 '20 at 17:29

0 Answers0