0

I've just started learning VBA a couple of weeks ago so i apologise if my work has any obvious errors. I've written some code that successfully scrapes data from tables on a website, the problem is that it runs very slowly. I am aware that using internetexplorer.application is quite a slow way to scrape websites and that a quicker way is to use MSXML2.XMLHTTP.

So i'm trying to convert my current code to use MSXML2.XMLHTTP instead of internetexplorer.application but i seem to be making a mess of it. I've searched through quite a lot of questions but i seem to be having issues using things like .getElementsByTagName.

Here's my internetexplorer.application code that i want to convert to MSXML2.XMLHTTP

Sub macro6()

Dim internet As InternetExplorer

Dim internetdata As HTMLDocument

Dim internetlink As Object
Dim bbb As Long
Dim internetinnerlink As Object
Dim LR As Long
    LR = Sheets("Sheet1").Range("C" & Rows.Count).End(xlUp).Row
Dim LR2 As Long
    LR2 = Sheets("Sheet1").Range("H" & Rows.Count).End(xlUp).Row + 1
Dim LC As Long
    LC = Sheets("Sheet1").Cells(2, Columns.Count).End(xlToLeft).Column + 1
Dim y As Long


For bbb = 1 To (LR - 1)
Set internet = CreateObject("InternetExplorer.Application")


internet.Visible = False

internet.navigate ("http://data.nowgoal.com/3in1odds/31_" & Sheets("Sheet1").Range("c1").Offset(bbb, 0)) & ".html"

Do While internet.Busy

  DoEvents

Loop

Do Until internet.READYSTATE = READYSTATE_COMPLETE

  DoEvents

Loop

Set internetdata = internet.document
For y = 2 To 4

hg_blue = internetdata.getElementsByTagName("tbody")(y).getElementsByClassName("hg_blue").Length + 1
hg_red = internetdata.getElementsByTagName("tbody")(y).getElementsByClassName("hg_red").Length
hg_green = internetdata.getElementsByTagName("tbody")(y).getElementsByClassName("hg_green").Length

xx = hg_blue + 1
yy = hg_blue + hg_red
zz = hg_blue + hg_red + hg_green

If zz > 1 Then

    Set internetlink = internetdata.getElementsByTagName("tbody")(y).getElementsByTagName("tr")(xx).getElementsByTagName("td")
    Set internetlink2 = internetdata.getElementsByTagName("tbody")(y).getElementsByTagName("tr")(yy).getElementsByTagName("td")
    Set internetlink3 = internetdata.getElementsByTagName("tbody")(y).getElementsByTagName("tr")(zz).getElementsByTagName("td")


    For Each internetinnerlink In internetlink
    Sheets("Sheet1").Cells(LR2, LC) = internetinnerlink.innerText

    LC = LC + 1
    Next internetinnerlink


    For Each internetinnerlink In internetlink2
    Sheets("Sheet1").Cells(LR2, LC) = internetinnerlink.innerText

    LC = LC + 1
    Next internetinnerlink


    For Each internetinnerlink In internetlink3
    Sheets("Sheet1").Cells(LR2, LC) = internetinnerlink.innerText

    LC = LC + 1
    Next internetinnerlink

Else

    Set internetlink = internetdata.getElementsByTagName("tbody")(y).getElementsByTagName("tr")(1).getElementsByTagName("td")
    Set internetlink2 = internetdata.getElementsByTagName("tbody")(y).getElementsByTagName("tr")(1).getElementsByTagName("td")
    Set internetlink3 = internetdata.getElementsByTagName("tbody")(y).getElementsByTagName("tr")(1).getElementsByTagName("td")


    For Each internetinnerlink In internetlink
    Sheets("Sheet1").Cells(LR2, LC) = ""

    LC = LC + 1
    Next internetinnerlink


    For Each internetinnerlink In internetlink2
    Sheets("Sheet1").Cells(LR2, LC) = ""

    LC = LC + 1
    Next internetinnerlink


    For Each internetinnerlink In internetlink3
    Sheets("Sheet1").Cells(LR2, LC) = ""

    LC = LC + 1
    Next internetinnerlink

End If

Next y
LR2 = LR2 + 1
LC = Sheets("Sheet1").Cells(LR2, Columns.Count).End(xlToLeft).Column + 1

internet.Quit
Next bbb


End Sub

The data in column C are numbers 1274444 in C2, 1274445 in C3, 1274446 in C4 etc.

Is it easier just to start from scratch and write the code again or are there simple changes i can make? Thank you for any help.

P McC
  • 1
  • 2
  • 1
    You can have a look at my answer [here](http://stackoverflow.com/questions/43596731/click-js-button-with-vba-when-the-button-has-no-name-id/43597210#43597210) to see the usage of MSXML2.XMLHTTP. You can easily convert your code into MSXML2.XMLHTTP with it. – Tehscript Apr 26 '17 at 16:16
  • Web-scraping with XHR allows to achieve higher performance using no DOM methods, thus it significantly differs from IE automation. Especially for AJAX / DHTML. – omegastripes Apr 26 '17 at 18:18

0 Answers0