Webdriver executeScript sporadically not detecting dynamically generated elements

Question

It has taken me long-long time to figure out and still I m struggling with this. I am trying to access dynamically generated elements on a web page using javascript injection using selenium webdriver. For example:

 String hasclass = js.executeScript("return document.getElementById('additional-details').children[0].children[0].children["
                            + k + "].children[0].classList.contains(\"results-execs-name\")").toString();

Now when I execute this script in firefox console, it works ok. But this line throws exception when run in webdriver 5-6 out of 10 times(even thought the element IS present physically).

Why is it happening? And whats the solution? I will surely upvote if any hints/answers are useful.

EDIT:

I have already put in Thread.sleep(500) and even 1000 seconds wait before each occurence of executeScript() in my code. Still its not working.

Here's the(partial) stacktrace:

org.openqa.selenium.WebDriverException: document.getElementById(...).children[0].children[0] is undefined
Command duration or timeout: 169 milliseconds
Build info: version: '2.39.0', revision: 'ff23eac', time: '2013-12-16 16:12:12'
System info: host: 'rahulserver-PC', ip: '121.245.92.68', os.name: 'Windows 7', os.arch: 'amd64', os.version: '6.1', java.version: '1.7.0_17'
Session ID: 747d2095-09f3-48b9-a433-59c5e334d430
Driver info: org.openqa.selenium.firefox.FirefoxDriver
Capabilities [{platform=XP, databaseEnabled=true, cssSelectorsEnabled=true, javascriptEnabled=true, acceptSslCerts=true, handlesAlerts=true, browserName=firefox, webStorageEnabled=true, nativeEvents=false, rotatable=false, locationContextEnabled=true, applicationCacheEnabled=true, takesScreenshot=true, version=31.0}]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
    at org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:193)
    at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:145)
    at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:554)
    at org.openqa.selenium.remote.RemoteWebDriver.executeScript(RemoteWebDriver.java:463)
    at Scraper.main(Scraper.java:62)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)

Take a dump of the HTML when it fails. Check if it's there. I'll also say it looks to me the JS isn't needed at all, you seem to be checking if a particular element (ignoring the fact it's a child of something) has a specific class. Selenium can do this natively. — Arran, Aug 01 '14 at 13:59
Probably late on this one but have you tried a higher sleep duration just to rule out wait timeout? Maybe 5 or even 10 seconds? I assume when you said you tried '1000 seconds wait' you meant 1000 milliseconds? And could you provide a little more info on the nature of the dynamic elements being introduced? Is it like a collapsible tree being populated by AJAX? Is the script execution within a loop? Noticed you had a 'k' variable in there. — shri046, Nov 03 '14 at 17:30

score 1 · Answer 1 · answered Aug 01 '14 at 14:29

Regarding checking for CSS classes: there is no need to execute a Script for that. You can do that in pure Java Code. Here is what works for me:

String cssClass = driver.findElement(By.xpath(xpath)).getAttribute("class");
assertTrue(cssClass.contains("disabledentry"));

score 1 · Answer 2 · answered Aug 02 '14 at 08:24

For dynamically generated list, its best to use xPath to find the element.

For a better solution, you need to say where the element is being dynamically generated? It can be inside a table or a ul/li and so on.

First, find the xPath to the parent element, i.e. the table or the list item.

Then create a dynamic xPath to the position of the element using a loop. A code snippet is as below:

String xPath_1 = ".//li[@class='item drop-shadow tiny-shadow' and position()=";
String xPath_2 = "]//div[@class='item-inner']//a";
String finalxPath = xPath_1 + i + xPath_2;

Create a method called fluentWait to wait for the element to be present rather than using thread sleep. Thread sleep is unreliable and it is the cause for tests to fail. Also fluentWait method will ignore NoSuchElementException(you can add more to the list such as StateStateElementException). Code snippet is as below:

public void fluentWait(final By by)
{
    FluentWait<WebDriver> wait = new FluentWait<WebDriver>(driver)
           .withTimeout(60, TimeUnit.SECONDS)
           .pollingEvery(5, TimeUnit.SECONDS)
           .ignoring(NoSuchElementException.class);
       WebElement foo = wait.until(new Function<WebDriver, WebElement>() {
         public WebElement apply(WebDriver driver) {
           return driver.findElement(by);
         }
       });
}

Then call this method with you dynamically generated xPath as follows:
```
fluentWait(By.xpath(finalxPath ));
```

To learn xPaths you can use the following Tutorial. You can try this out and let me know. Cheers

score 0 · Answer 3 · answered Aug 01 '14 at 13:32

0

With Selenium you always have to be aware, that script execution may take a little moment. If you have a situation that sometimes succeeds and sometimes fails, check if a Thread.sleep(500) helps you. Otherwise JUnit may evaluate assertions, before the script has even finished.

answered Aug 01 '14 at 13:32

Julian

21
4

Ok, sorry I couldn't help you there – Julian Aug 01 '14 at 13:37
If you could specify more detailed error information (e.g. stack trace), that might be a clue – Julian Aug 01 '14 at 13:38
This clearly looks as though the specified element doen't exist. Make sure your script has the expected results and does not fail silently or swallows errors (strict mode on? always use strict mode in JS). Perhaps children[0].children[0] is a little genric. You should use jQuery for DOM manipulation if possible to avoid programming mistakes. – Julian Aug 01 '14 at 13:46
How we turn on strict mode in webdriver? is it configurable? and regarding dom parsing, its correct. I confirmed via firefox console. – rahulserver Aug 01 '14 at 13:49
You turn on strict mode by putting "use strict"; as first statement in your JS script – Julian Aug 01 '14 at 14:25

Webdriver executeScript sporadically not detecting dynamically generated elements

3 Answers3