Extracting multiple text using partial href information

Question

I'm trying to extract multiple genre from below site. (I already know the URLs) https://www.discogs.com/master/1515454-Zedd-Katy-Perry-365

<div class="profile">
  <h1 id="profile_title" class="hide_mobile has_action)menu">...<h1>
  <div class="head">Genre:<div> ==$0
  <div class="content">
    <a href="/genre/electronic">Electronic</a>
    ", "
    <a href="/genre/pop">Pop</a>

And here's my Python code

genre = None
try:
  genre = driver.find_element_by_xpath("[contains(concat(' ', @class, ' '), ' profile ')]//*[contains(@href, ' /genre/* '").text

How do I extract genres to text? (e.g. Electronic, Pop)

score 0 · Accepted Answer · answered Apr 18 '22 at 12:28

To extract and print the values of Genre i.e. Electronic, Pop, etc within the website you need to induce WebDriverWait for visibility_of_all_elements_located() and you can use either of the following Locator Strategies:

Using XPATH:

driver.get("https://www.discogs.com/master/1515454-Zedd-Katy-Perry-365")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button#onetrust-accept-btn-handler"))).click()
print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//tr/th[@scope='row' and contains(., 'Genre')]//following::td[1]//a")))])

Console Output:
```
['Electronic', 'Pop']
```

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Thanks. The code is currently in 'for loop'. (reading and writing on excel - for many addresses) If URL meets no information, how do I pass and proceed to the next loop - instead of the 'TimeoutException' error message? — Myoung Nam, Apr 18 '22 at 13:38
@MyoungNam Can you please raise a new question for your new requirement? — undetected Selenium, Apr 18 '22 at 13:49

Extracting multiple text using partial href information

1 Answers1