http response text fetching incomplete html

Question

I have a code (given below) in excel vba that fetches web page source html. The code is working fine but the html that it fetches is incomplete. When the line webpageSource = oHttp.ResponseText is executed, the variable webpageSource contains "DOCTYPE html PUBLIC ....... etc etc till the end /html" and that is how it should be. Everything is correct till here. But the next line debug.print webpageSource prints only half the html from "(adsbygoogle = window.adsbygoogle || []).push({}); ...... etc etc till the end /html" Why is that so? I want to find some strings from the returned response text but since it is incomplete, I am unable to do so. Can someone shed some light on it?

Thanks

Sub source()
    Dim oHttp As New WinHttp.WinHttpRequest
    Dim sURL As String
    Dim webpageSource As String
    sURL = "http://www.somewebsite.com"
    oHttp.Open "GET", sURL, False
    oHttp.send
    webpageSource = oHttp.ResponseText
    debug.print webpageSource
End Sub

EDIT: I also tried .WaitForResponse with no help :(

what happens when you, say, place `webpagesource` into a cell? Does it put all the text in there? or maybe even a word document, since it may fit into a cell? — Scott Holtzman, Oct 09 '15 at 16:23
i havent put it into a cell. In the vba editor when i use F8 to execute the code line by line, when I move my mouse over the variable webpagesource, it shows me the content that starts with DOCTYPE so I am assuming that it is capturing full html but when I see the result of debug.print, it gives me only half the source — Sabha, Oct 09 '15 at 16:33
well, this maybe obvious, but are you using the scroll bar in the immediate window to see if the beginning of the text is actually there above what you see? also, if you write the line to cell or word document you can confirm that you are indeed getting everything. — Scott Holtzman, Oct 09 '15 at 16:34
yes, I am using scrollbar and i wish to confirm that it is incomplete. Does excel strip off initial part if the text is long? Can you edit my code to write the contents of webpagesource to a word or text document? — Sabha, Oct 09 '15 at 16:40
you just got that hand delivered to you in the answer below :) — Scott Holtzman, Oct 09 '15 at 16:43

score 4 · Accepted Answer · answered Oct 09 '15 at 16:41

Debug.Print and/or the immediate window have limitations. Nowhere documented however they have.

So try writing the webpageSource to a file:

Sub source()
    Dim oHttp As New WinHttp.WinHttpRequest
    Dim sURL As String
    Dim webpageSource As String
    sURL = "http://www.google.com"
    oHttp.Open "GET", sURL, False
    oHttp.send
    webpageSource = oHttp.ResponseText

    Set FSO = CreateObject("Scripting.FileSystemObject")
    Set oFile = FSO.CreateTextFile("webpageSource.txt")
    oFile.Write webpageSource
    oFile.Close

    Shell "cmd /C start webpageSource.txt"

End Sub

Does the file contain all content?

Yes, it does contain everything. may be the immediate window strips the initial part of it while debug.print. May I request you to have a look at my other thread? http://stackoverflow.com/questions/33030951/fetching-webpage-source-when-the-response-is-different-in-logged-in-and-logged-o thanks — Sabha, Oct 09 '15 at 16:54

http response text fetching incomplete html

1 Answers1

Linked