Excel VBA: Wait for JavaScript execution in Internet Explorer

Question

I am trying to do some web scraping in Excel VBA. Here is the part of the code that I am having trouble with:

IE.Navigate URL
Do
  DoEvents
Loop While IE.ReadyState <> 4 Or IE.Busy = True
Set doc = IE.document

After running this doc contains html that still has unexecuted JavasScript in it. This is the signature of the script that has not been executed:

<SCRIPT type=text/javascript>
        goosSearchPage.Initialize(...)...;
</SCRIPT>

I can wait for execution by doing Application.Wait(Now + TimeValue(x)) but that really is not satisfactory, as the amount of time the script takes to execute is quite variable depending on input.

Is there a way to either wait for the script to finish evaluating or to just evaluate the script directly in the doc object?

Is there some result of the script having run which you can check for? Do page elements get created, or some form values filled? AFAIK there is no "scriptfinishedrunning" event, so you'd have to check for the output or result of the script. Hard to say more without konwing what the script does. — Tim Williams, Oct 23 '12 at 16:39
Thank you Tim, thats a great idea, although i feel kind of stupid for not coming up with it myself. I'll just check whether SearchPage.Initialize is still in the body. — Jonas Raedle, Oct 23 '12 at 20:22

score 0 · Answer 1 · answered Oct 23 '12 at 13:59

I found code that does wait for a page to complete. per the notes here, it requires the Microsoft Internet Controls as a reference in your code.

Code reproduced here, just in case the link dies:

'Following code goes into a sheet or thisworkbook class object module
Option Explicit

'Requires Microsoft Internet Controls Reference Library
Dim WithEvents ie As InternetExplorer
Sub start_here()
  Set ie = New InternetExplorer
  'Here I wanted to show the progress, so setting ie visible
  ie.Visible = True
  'First URL to go, next actions will be executed in
  'Webbrowser event sub procedure - DocumentComplete
  ie.Navigate "www.google.com"
End Sub
Private Sub ie_DocumentComplete(ByVal pDisp As Object, URL As Variant)
  'pDisp is returned explorer object in this event
  'pDisp.Document is HTMLDocument control that you can use
  'Following is a choice to follow,
  'since there is no do-loop, we have to know where we are by using some reference
  'for example I do check the URL and do the actions according to visited URL

  'In this sample, we use google entry page, set search terms, click on search button
  'and navigate to first found URL
  'First condition; after search is made
  'Second condition; search just begins
  If InStr(1, URL, "www.google.com/search?") > 0 Then
    'Open the first returned page
    ie.Navigate pDisp.Document.getelementsbytagname("ol")(0).Children(0).getelementsbytagname("a")(0).href
  ElseIf InStr(1, URL, "www.google.com") > 0 Then
    pDisp.Document.getelementsbyname("q")(0).Value = "VB WebBrowser DocumentComplete Event"
    pDisp.Document.getelementsbyname("btnG")(0).Click
  End If
End Sub

Thank you for the answer Sean. Unfortunately this does not work, since the DocumentComplete Event fires when READYSTATE changes to READYSTATE_COMPLETE (=4), which is exactly what I do in my solution. Source: [link](http://msdn.microsoft.com/en-us/library/aa768329(v=vs.85).aspx) — Jonas Raedle, Oct 23 '12 at 14:38
This solution also only works if the visibility setting of the browser is set to true, which is not acceptable in my case. — Jonas Raedle, Oct 23 '12 at 14:45

score 0 · Answer 2 · answered Oct 18 '13 at 11:38

0

You actually can evaluate the javascript function with the ie window. But you gotta set up a Callback because the function will be evaluated async.

answered Oct 18 '13 at 11:38

Iwan1993

1,669
2
17
25

score 0 · Answer 3 · answered Aug 22 '20 at 02:17

This post is quite old, but I'll answer this also as a reply to myself, now that I've discovered how to do this.

Simply point to a content you expect to be there after the jQuery script has run, trigger the desired event using JavaScript ran through IE automation, and do a Loop to wait until the desired content appears.

'This will trigger the jQuery event.
Doc.parentWindow.execScript "$('#optionbox').trigger('change')"

'This is the code that will make you wait. It's surprisingly efficient
Do While InStrB(Doc.getElementById("optionbox").innerHTML, "<desired html tag>") = 0
    DoEvents
Loop

Excel VBA: Wait for JavaScript execution in Internet Explorer

3 Answers3

Linked