Using Selenium to get information within the tag 'h1' and id

Question

I'm trying to get the information which is: 'Jarrow Formulas, Methyl Folate, 400 mcg, 60 Veggie Caps'

you can check the picture, thank you so much:

I used this code but it doesnt work out:

driver = webdriver.Chrome(chrome_path)
driver.get("https://www.iherb.com/c/Vitamin-B?sr=2")
wait = WebDriverWait(driver, 10)

item_name = list()

#close the pop up
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,"svg[data-ga-event-action='list-close']"))).click()

#store all the links in a list
item_links = [item.get_attribute("href") for item in wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR,".absolute-link-wrapper > a.product-link")))]

for item_link in item_links:
    driver.get(item_link)item_name.append(driver.find_element_by_css_selector('[id="name"]').text) #this code doesnt work

Answer 1

To print the text value you can use either of the following Locator Strategies:

• Using xpath and text attribute:

  print(driver.find_element_by_xpath("//section[@class='column image-fixed']//following::section[2]//div[@id='product-summary-header']//h1[@id='name']").text)
  

• Using xpath and get_attribute():

  print(driver.find_element_by_xpath("//section[@class='column image-fixed']//following::section[2]//div[@id='product-summary-header']//h1[@id='name']").get_attribute("innerHTML"))
  

• Console Output:

  Jarrow Formulas, Methyl Folate, 400 mcg, 60 Veggie Caps
  

---

Ideally you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following Locator Strategies:

• Using xpath and text attribute:

  driver.get('https://ca.iherb.com/pr/Jarrow-Formulas-Methyl-Folate-400-mcg-60-Veggie-Caps/42778')
  print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//section[@class='column image-fixed']//following::section[2]//div[@id='product-summary-header']//h1[@id='name']"))).text)
  

• Using XPATH and get_attribute():

  driver.get('https://ca.iherb.com/pr/Jarrow-Formulas-Methyl-Folate-400-mcg-60-Veggie-Caps/42778')
  print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//section[@class='column image-fixed']//following::section[2]//div[@id='product-summary-header']//h1[@id='name']"))).get_attribute("innerHTML"))
  

• Console Output:

  Jarrow Formulas, Methyl Folate, 400 mcg, 60 Veggie Caps
  

• Note : You have to add the following imports :

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  

You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python

---

References

Link to useful documentation:

• get_attribute() method Gets the given attribute or property of the element.
• text attribute returns The text of the element.
• Difference between text and innerHTML using Selenium