1

I'm trying to maintain a piece of Visual Basic 6 software that was written to help expedite some text formatting for a department at my job. I'm not much of a computer engineer/scientist, but this piece of software falls into my jurisdiction as it was written by the gentleman I was hired to replace.

This tool was made to format a large amount of data in a short amount of time in order to eliminate manual formatting.

It seems to be that the WinHttpRequest that's passed to the URL as specified by the users of the program is mostly working. Yet when the ResponseBody property is called, the information that is expected by the written software is not being grabbed. I am not certain of how the data was hosted on the URL page before, but when the page source of the URL in question is examined via a web browser, the data that is expected to be retrieved is visible but appears to be hot-linked in from a different URL/location. But when the WinHttpReq.ResponseBody is made and dumped into a file locally, the information that is expected and needed is passed over.

It looks like it's passing over the data because the data is referenced in via separate hot-links at the original URL. My question is, what is the easiest way to retrieve the data that is visible when examining the page source, but isn't retrieved in the ResponseBody function call. The ResponseBody call also seems to be jettisoning the hot-linked URL's into the initial HTTP request URL, but I think the information and ID's used to build the hot-links is stored in the data. 

' Create an array to hold the response data.
Dim d() As Byte

' Assemble an HTTP request.
WinHttpReq.Open "GET", ap_feedreq, False

' Set the user name and password.
WinHttpReq.SetCredentials ap_user, ap_pass, _
HTTPREQUEST_SETCREDENTIALS_FOR_SERVER

' Send the HTTP Request.
WinHttpReq.Send

' Display the response headers.
' MsgBox WinHttpReq.GetAllResponseHeaders & "  " & WinHttpReq.ResponseText

If WinHttpReq.status = 200 Then

    webwire1.StatusBar1.Panels(1).Text = "Authentication Complete ..... 
    Parsing XML File"
    webwire1.StatusBar1.Refresh

    docpath1$ = mydocpath + "aptemp.txt"
    Open docpath1$ For Binary As #1
    d() = WinHttpReq.ResponseBody
    Put #1, 1, d()
    Close #1

Here's an image illustrating what I see to hopefully better show what I'm referring to.

Feed Body Comparison Browser vs. Local Data

The parsing function that was written within the software looks for the "nitf version =" flags to know when it's found the statistical data/tables that's to be used to format.

My question is, is there an easier, more elegant solution that will pull down all of the webpage source data at once as visible within a web browser? Or will I have to pull the response body, parse through the body text, build URL's using variables, pass more individual page URL requests, store that text, parse through that data, then load the parsed data back into the original body data?

I want to change the written software as little as possible.

Jones
  • 13
  • 2

1 Answers1

0

Your best bet is to look at output from Fiddler and see what differs at the lowest level. For someone to help you on SO, they will likely need to view the raw HTTP response. From what I can tell, this is not (directly) related to VB6 nor WinHttpRequest but I'm not sure how the browser is deciding to fetch the data -- it may be via JavaScript or something XML/HTTP/feed specific. It could even be that you need to send an Accept header to get different content from the AP server. Or, like you said, you may have to manually pull each URL. If you need to act exactly as a browser would, you could use NWJS, Electron, CEF, PhantomJS (or now headless Chromium), etc. Or there is a WebBrowser control in VB6.

Graeme Wicksted
  • 1,770
  • 1
  • 18
  • 21
  • I dl'd Fiddler. What's weird, running our VB6 app, it pulls down the HTTP req data as defined in the logic, but I'm not seeing the HTTP action appear within Fiddler. It's picking up browser traffic fine & correctly it appears. AP's been slow to assist/give a firm response with what they changed on their end to help trace the issue. They did mention that maybe it's an issue with encoding & the difference between ASCII & UTF-8 encode, but Im unsure. Thanks for your help. – Jones Apr 19 '17 at 17:40
  • I figured it out, and the issue was something stupid. Initially, when I was first made aware of the issue, the VBA debugger wasn't working due to another problem, so I did some of the "debugging" through mental step-through with pen and paper to try and trace what was happening. I got that issue fixed, but I didn't go back and re-examine my initial mental checking. Turns out, the variable used to build the URL for the HTTP request was getting a single space appended into it. That 1 space doesn't totally break the URL, but it prevents the hot-linked data from appearing. – Jones Apr 19 '17 at 18:44