0

I am on a page in which I need to click on a variable's logo/name/arrow (3 button options). The variable is called HISCO occupation. HTML from URL: https://icem.data-archive.ac.uk/#step2:

<div class="clearfix ice-tab-wrapper" id="workhistoryunit_category" ng-class="{active:tabActive[child.name]}">
:: before
<button class="fa fa-certificate ice-icon" ng-click="toggleTab(child.name, $event)">
:: before
<span class= "sr-only ng-binding"> HISCO OCCUPATION </span>
</button>
< button .....> </button>

My entire code so far:

ETUPfrom selenium.webdriver.support.ui import WebDriverWait
import selenium
from selenium import webdriver as wd
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

chrome_path = r'C:\webdrivers\chromedriver.exe'

webD=wd.Chrome(executable_path=chrome_path)
webD.get('https://icem.data-archive.ac.uk/#step1')


## STEP 1: SELECTING A YEAR, HERE 1851

webD.find_element_by_xpath('/html/body/div/section/section[1]/article[1]/div[1]/div/div[1]/label/input').click()


## STEP 2: SELECTING ENGLAND

WebDriverWait(webD, 20).until(EC.element_to_be_clickable(
    (By.XPATH, '//b[@id = "country_england"]/preceding-sibling::input'))).click()


## STEP 3: MOVE ON TO VARIABLES

webD.find_element_by_xpath('//html/body/div/section/section[1]/article[2]/div/div/button').click()


## STEP 4.1: COUNTIES, OPEN MENU

WebDriverWait(webD, 20).until(EC.element_to_be_clickable(
    (By.XPATH, '//*[@id="county_category"]/button[1]'))).click()


## STEP 4.2 COUNTIES, MORE VARIABLES
WebDriverWait(webD, 20).until(EC.element_to_be_clickable(
    (By.XPATH, '//*[@id="county_category"]/div[2]/div/div[2]/button'))).click()

## VERSION 1, SELECTION 1 COUNTY/ 1 HISCO_OCC
## STEP 4.3 COUNTIES, SELECT VARIABLES

WebDriverWait(webD, 20).until(EC.element_to_be_clickable(
    (By.XPATH, '/html/body/div[3]/div/div/div[3]/div[1]/label/input'))).click()

## STEP 4.3.2 CLICK APPLY

WebDriverWait(webD, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[contains(.,'Apply')]"))).click()


# STEP 5, HISCO OCCUPATION, OPEN MENU

WebDriverWait(webD, 20).until(EC.element_to_be_clickable((By.ID, "workhistoryunit"))).click()

As of now, I have the following (unsuccessful) code trying to click on either of these buttons:

WebDriverWait(webD, 20).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="workhistoryunit_category"]/button[1]'))).click()

or

webD.find_element_by_xpath('//*[@id="workhistoryunit_category"]/button[1]').click()

The action prior to clicking this button is closing a window (applying a selection).

This code is based on the code from a previous analogous click on the same page, just another variable (hence different ID) that works:

WebDriverWait(webD, 20).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="county_category"]/button[1]'))).click()

Any ideas what the problem maybe?

Norayr Sargsyan
  • 1,737
  • 1
  • 12
  • 26
Rye Rye
  • 57
  • 5

3 Answers3

2

Use following XPath to click on the button.

WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.XPATH,"//button[./span[text()='HISCO classified occupation']]"))).click()
KunduK
  • 32,888
  • 5
  • 17
  • 41
  • Thank you for your help! It seems like it shound work... But still no success... Any additional information I could give to be more precise? – Rye Rye Feb 01 '21 at 17:41
  • 1
    @RyeRye : what error are you getting while clicking? – KunduK Feb 01 '21 at 17:44
  • No error! The button is simply not clicking, so I cannot go forward in my automation. – Rye Rye Feb 01 '21 at 17:44
  • 1
    @RyeRye : I got the catch. Application is too fast and selenium unable to sync properly. use `time.sleep(3)` before clicking It should work. I have just tested with code. – KunduK Feb 01 '21 at 17:48
  • So embed time.sleep(3).click()? I'll try! Thank you very much for all the effort! – Rye Rye Feb 01 '21 at 17:51
  • @RyeRye : NO just use this `time.sleep(3)` before that line `WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.XPATH,"//button[./span[text()='HISCO classified occupation']]"))).click()` – KunduK Feb 01 '21 at 17:53
  • I get an error message in the shell: time.sleep(10) NameError: name 'time' is not defined >>> Any idea how to target this – Rye Rye Feb 01 '21 at 18:01
  • @RyeRye : You need to import following library `import time` – KunduK Feb 01 '21 at 18:02
1
# STEP 5, HISCO OCCUPATION, OPEN MENU


WebDriverWait(webD, 20).until(
    EC.presence_of_element_located((By.XPATH, '//span[@class = "ice-allow-download-alert ng-animate ng-hide-remove ng-hide-remove-active"]')))

WebDriverWait(webD, 20).until(
    EC.invisibility_of_element_located((By.XPATH, '//span[@class = "ice-allow-download-alert ng-animate ng-hide-remove ng-hide-remove-active"]')))

WebDriverWait(webD, 20).until(
    EC.presence_of_element_located((By.XPATH, '//b[text()[contains(.,"HISCO classified occupation")]]/..'))).click()

Try this there is small tool tip being displayed for few seconds that is removing the reference , so use this code first to check its presence then its invisibility and then click that button

There is a green tool tip that comes up same time the tab you want to click is loaded , so it resets the element reference and causes stale element exception. You need to wait for that elemnt to disappear first. But as the element is displayed only for few seconds we cannot find its locator . To find its locator open inspect and right click the html element and add break on > subtree modification and keep resuming execution still you get that tool tip

JeffC
  • 22,180
  • 5
  • 32
  • 55
PDHide
  • 18,113
  • 2
  • 31
  • 46
  • Smart approach but still doesn't click... It's a bit frustrating, it seems many of these propositions should work... – Rye Rye Feb 01 '21 at 17:49
  • @RyeRye it work just add a sleep after that line you shuld add that time.sleep(5) , – PDHide Feb 01 '21 at 17:54
  • else you will get stale element – PDHide Feb 01 '21 at 17:54
  • Makes sense! My shell returns this error though: time.sleep(10) NameError: name 'time' is not defined >>> Am I missing something? – Rye Rye Feb 01 '21 at 17:57
  • @PDHide You have used `WebDriverWait` and further still suggestion **sleep**? – undetected Selenium Feb 01 '21 at 18:00
  • @RyeRye you should have import time – PDHide Feb 01 '21 at 18:04
  • @DebanjanB There is a small screen loading that changes the reference of page – PDHide Feb 01 '21 at 18:04
  • @PDHide I can pull out 10+ of your answers where you have successfully used waits to counter the loading behavior :) – undetected Selenium Feb 01 '21 at 18:07
  • 1
    @DebanjanB Updated the answer , there is a tool tip that shows for few seconds that causes the stale element for the target button . It was hectic to use dom break point and find its locators thats why used time.sleep() instead . But anyways added the wait answer – PDHide Feb 01 '21 at 18:09
  • @RyeRye try this and let me know – PDHide Feb 01 '21 at 18:10
  • 1
    @RyeRye there is a green tool tip that comes up same time the tab you want to click is loaded , so it resets the element reference and causes stale element exception. You need to wait for that elemnt to disappear first. But as the element is displayed only for few seconds we cannot find its locator . To find its locator open inspect and right click the html element and add break on > subtree modification and keep resuming execution still you get that tool tip :D uff long story and effort i know – PDHide Feb 01 '21 at 18:19
  • 1
    @PDHide I'm very impressed!! I'm really just starting out so this was way out of reach for me. Thank you very very much for helping out!! – Rye Rye Feb 01 '21 at 18:24
0

To click on the element with text as HISCO OCCUPATION you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:

  • Using XPATH:

    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@class='ice-tab-button ng-binding']/b[@class='ng-binding' and contains(., 'HISCO classified occupation')]"))).click()

  • Note: You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • Thank you for your help, sadly neither solution is successful in clicking the button... – Rye Rye Feb 01 '21 at 17:36
  • 1
    @RyeRye Checkout the updated answer and let me know the results. – undetected Selenium Feb 01 '21 at 17:55
  • Other users have suggester it isn't actually the locator strategy being an issue but the speed and suggested to include a time.sleep(5) right before, I'm experimenting with that. I'll try your suggestion next and will let you know. Thank you! – Rye Rye Feb 01 '21 at 18:00
  • 1
    @RyeRye I understand you wanted to avoid [WebDriverWait](https://stackoverflow.com/questions/59130200/selenium-wait-until-element-is-present-visible-and-interactable/59130336#59130336) as I suggested in your initial question which you have avoided. [time.sleep()](https://stackoverflow.com/questions/52603847/how-to-sleep-webdriver-in-python-for-milliseconds/52607451#52607451) isn't a good practice. Honestly sooner or later you have to implement [WebDriverWait](https://stackoverflow.com/questions/50154940/how-can-i-make-sure-if-some-html-elements-are-loaded-for-selenium-python/50159359#50159359) – undetected Selenium Feb 01 '21 at 18:05
  • 1
    Hmmm I think I'm a bit confused, I was using both sleep and wait... I think you have a point there. In any case, nothing has yet worked, including your suggestion... – Rye Rye Feb 01 '21 at 18:12