VBA+Selenium+Edge Cannot 'find' element

Question

I've tried all kinds of ele.findElementAs--- and cannot "find" a particular (multiply nested) element on a webpage - even though I can visually see the element (and values) when inspecting the webpage.

I've used VBA+Edge+Selenium before and can locate / "find" other elements on this page, but not the one (or similar ones) I need.

url: www.cmegroup.com item: the Price for the December Corn Futures ("ZCZ2")

JSpath: document.querySelector("#main-content > div > div.component.section.cme-homepage-background-gradient-2.pt-5.reverse > div > div:nth-child(8) > div:nth-child(1) > div.component.react.heat-map.loaded > div > div > div:nth-child(1) > div > a.heat-map-card.heat-map-color_1 > div.product-values > div.rate")

snapshot of webpage code above target:

Code from webpage

my code sample:

    Sub FindDecCorn()
  
  Dim Edgdriver As New EdgeDriver
  Edgdriver.Start "edge"
  
  Edgdriver.Get "https://cmegroup.com"
  
  ' *** this one works - finds "main-content" ***
    Dim ch As Selenium.WebElement
    Set ch = Edgdriver.FindElementById("main-content")

    'Set ch = driver.FindElementByLinkText("www.cmegroup.com/etc.clientlibs/cmegroupaem/clientlibs/heat-map.cc2d1dd424fd10c5642e7137587e27a7.css")
    Debug.Print ch.tagname, ch.Attribute("id")
  
  ' *** I've tried all kinds of .FindElement(s)ByXXXX --- all failed ***
  ' *** this one fails to find anything with 'product-code' although there are several ***
    Dim myElements As Selenium.WebElements
      Set myElements = Edgdriver.FindElementsByCss("div[class='product-code']")
     For Each myElement In myElements
        Debug.Print myElement.Attribute("innerHTML")
     Next myElement
  
      
  Edgdriver.Quit
    End Sub

Can you show your code? looking at the site it loads the data your looking for after the page has loaded, so its there when you inspect it but it might not be ready when the script is executing — NickSlash, Aug 07 '22 at 17:43

NickSlash · Accepted Answer · 2022-08-07T20:31:52.987

0

Think I've found your problem, as I commented the elements your looing for are loaded in after the page is loaded. After playing about it looks like they are not loaded until scrolled into view.

Sub main()
Dim Edge As New EdgeDriver

Edge.Start "edge"
Edge.Get "https://cmegroup.com"

' let the page load
Edge.Wait 500
' scroll the page
'Edge.ExecuteScript "window.scrollTo(0, document.body.scrollHeight/4);"
' scroll to bottom (smoothly no jump)
Edge.ExecuteScript "window.scrollTo({top:document.body.scrollHeight, behavior: 'smooth'});"
' wait a little more...
Edge.Wait 500

Dim List As Selenium.WebElements
Dim Item As WebElement
Dim Index As Long: Index = 1

Set List = Edge.FindElementsByClass("product-code")

If List Is Nothing Then
    Debug.Print "couldnt find product-code"
    Exit Sub
End If

Do
    Set Item = List(Index)
    If Item.Text = "ZCZ2" Then
        Exit Do
    Else
        If Index > List.Count Then
            Exit Do
        Else
            Index = Index + 1
        End If
    End If
Loop

If Item Is Nothing Then
    Debug.Print "found product-code but not ZCZ2"
Else
    ' find the parent element then find the desired child element
    Debug.Print Item.FindElementByXPath("../..").FindElementByClass("rate").Text
End If

Edge.Quit

Set Edge = Nothing

End Sub

edited Aug 07 '22 at 20:31

answered Aug 07 '22 at 18:53

NickSlash

4,758
3
21
38

many thanks - I tested it twice (so far) - had to adjust the scroll height to "8" on my browser (when only "4", I got an error at the ```Set Item = List(Index)``` line – arc Aug 07 '22 at 19:03
actually - that was a false hope - I continue to get errors on that line ```Set Item``` when trading out the ZCZ2 symbol for other test cases The error is "Index was outside the bounds of the array" – arc Aug 07 '22 at 19:21
**update**: when I put a line break in the code to test ... then scrolled the browser so the target area was visible - everything worked as you designed. However, when I scrolled all the way to the bottom (so the target area was not visible) - the error happens. Importantly, when the 25% scroll as you created (or 1/3 scroll on my browser) happens to 'make visible' the target... then everything works. THANKS – arc Aug 07 '22 at 19:53
I tried to scroll to the bottom too, but it was more of a jump rather than scroll which I imagine is why it fails. I've updated the code with a smooth scroll to the bottom and it seems to work for me. – NickSlash Aug 07 '22 at 20:34
Nick :: "thumbs up" .... another thought (I have not tried this yet - have to find the code snippet to do this) ... if the page is 'shrunk' to completey fit on one visible browswer page (even though too tiny for human eyes), would that help in a more general case?? – arc Aug 08 '22 at 01:35
found this: ```Edge.ExecuteScript "document.body.style.zoom='10%';"``` and tried it - seemed to work "just fine". Might be a bit of a hack, but worked. – arc Aug 08 '22 at 01:47

VBA+Selenium+Edge Cannot 'find' element

1 Answers1