Disabling javascript with htmlunit through a remote webdriver

Question

When getting a page with selenium from a python script, using htmlunit through a remote webdriver, I get this error:

WebDriverException: Message: u'TypeError: Cannot find function addEventListener in object [object HTMLDocument]. (https://xxx.xxx.com/static/js/jquery-2.0.3.min.js#4)

So to avoid this exception I tried to disable javascript when initializing my webdriver, but I can't manage to do it...

I tried to set the desired_capabilities with webdriver.DesiredCapabilities.HTMLUNIT (vs HTMLUNITWITHJS), but nothing changed. So I tried to define the capabilities manually, but it didn't help.

Here are some examples of what I did, with the results:

In [45]: driver = webdriver.Remote(desired_capabilities=webdriver.DesiredCapabilities.HTMLUNIT)

In [46]: driver.desired_capabilities
Out[46]:
{u'browserName': u'htmlunit',
 u'cssSelectorsEnabled': True,
 u'javascriptEnabled': True,
 u'platform': u'LINUX',
 u'version': None,
 u'webdriver.remote.sessionid': u'3aa1c9c0-9d85-4e22-ad2b-1116950cf86d'}

In [47]: driver = webdriver.Remote(desired_capabilities={'browserName': 'htmlunit', 'javascriptEnabled': False, 'platform': 'ANY', 'version': ''})

In [48]: driver.desired_capabilities
Out[48]:
{u'browserName': u'htmlunit',
 u'cssSelectorsEnabled': True,
 u'javascriptEnabled': True,
 u'platform': u'LINUX',
 u'version': None,
 u'webdriver.remote.sessionid': u'426aef71-2b7c-45c5-9313-c3dbbec07c7f'}

So here I am. If someone has any idea... :)

score 1 · Accepted Answer · edited May 23 '17 at 11:51

Partial answers, in case someone else encounters the same kind of issue:

1/ Having a look at the server logs, it appears that my instructions regarding the javascriptEnabled setting are actually taken into account by the selenium server. The desired_capabilities dictionary displayed by the client is not relevant.

15:21:35.575 INFO - Executing: [new session: Capabilities [{platform=ANY,  javascriptEnabled=false, browserName=htmlunit, version=}]])
15:21:35.576 INFO - Creating a new session for Capabilities [{platform=ANY, javascriptEnabled=false, browserName=htmlunit, version=}]
15:21:35.590 INFO - Done: [new session: Capabilities [{platform=ANY, javascriptEnabled=false, browserName=htmlunit, version=}]]

2/ Reading this discussion, it appears that what I expected from the javascriptEnabled setting isn't what it is meant for. This doesn't disable javascript execution in the browser, but javascript 'injection' from a client script.

Disabling javascript execution doesn't seem possible using HTMLUNIT. So this question can be closed as it isn't relevant.

The only solution is to get HTMLUNIT to work with jquery-2.0.3. I upgraded it to the latest version, but it didn't help...

[Edit] Finally a suitable solution was found here: preventing HTMLUNIT from throwing an exception on a javascript error.

In [14]: driver = webdriver.Remote(desired_capabilities={'browserName': 'htmlunit', 'javascriptEnabled': False, 'platform': 'ANY', 'version': '', 'setThrowExceptionOnScriptError': False})

I can now load my url! :)

Disabling javascript with htmlunit through a remote webdriver

1 Answers1