Python Selenium cannot get anything when using .text method

Question

HTML/CSS I'm trying to extract, the highlighted text is what I'm trying to extract

I'm specifically trying to get a verification code out of an email from Mailinator, however, I have tried getting and printing text from anything and nothing appears to be storing when I use the .text function. Additionally, I can only use find_elements(By.XPATH, "example) and not find_elements_by_xpath("example") as an example (same for css_selector, etc). For reference, I'm using the most current version of Python and Visual Studio Code as my environtment. Does anyone know what may be the issue? I've tried a number of sources, but I can't find a similar case to what I'm getting.

I should add that I've also tried scraping and I'm getting the same result, although that was more of a half try of just trying to find all elements that would fit a regex and store them with a loop.

I'm not getting an error message, it just doesn't appear to be storing text, or, at least, it isn't printing it. It prints a \n. It isn't just this code, by the way. I've tried several websites, including Google's homepage, and I can't seem to store and print text from anywhere on the web. For the record, this is the code with the credentials removed (if you see any other errors, please let me know):

from selenium import webdriver
from selenium.webdriver.common.by import By
import time
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
**# from faker import Faker
**# import re****

options = webdriver.ChromeOptions()
options.add_experimental_option('excludeSwitches', ['enable-logging'])
driver = webdriver.Chrome(options=options)
wait = WebDriverWait(driver, 20)
username = '*'
driver.get("https://www.mailinator.com/v4/login.jsp")
wait.until(EC.element_to_be_clickable((By.XPATH, "/html/body/main/div/div/div/section/form/div[2]/input"))).send_keys(
    "*")
wait.until(EC.element_to_be_clickable((By.XPATH, "/html/body/main/div/div/div/section/form/div[3]/input"))).send_keys(
    "*")
wait.until(EC.element_to_be_clickable((By.XPATH, "/html/body/main/div/div/div/section/form/div[4]/a[1]"))).click()
wait.until(EC.element_to_be_clickable((By.XPATH, "//*[contains(@id,'row_"+ username +"')]/td[2]"))).click()
iframe = driver.find_element('tag name', 'iframe')
print('line21 executed')
driver.switch_to.frame(iframe)
print('line23 executed')
email_text = driver.find_element('tag name', 'body').text
print('line25 executed')
print(email_text)
print('line27 executed')

I'm trying to get text stored so I can extract a verification code to put in as part of a registration process while performing the BDD for a project, but I've hit this stonewall for several hours at this point and cannot resolve the issue.

EDIT:

I've updated the code to the following after suggestions and I will show the output at the bottom:

   from selenium import webdriver
    from selenium.webdriver.common.by import By
    import time
    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.support import expected_conditions as EC
    # from faker import Faker
    # import re

    options = webdriver.ChromeOptions()
    options.add_experimental_option('excludeSwitches', ['enable-logging'])
    driver = webdriver.Chrome(options=options)
    wait = WebDriverWait(driver, 20)
    username = '*'
    driver.get("https://www.mailinator.com/v4/login.jsp")
    wait.until(EC.element_to_be_clickable((By.XPATH, "/html/body/main/div/div/div/section/form/div[2]/input"))).send_keys(
        "*")
    wait.until(EC.element_to_be_clickable((By.XPATH, "/html/body/main/div/div/div/section/form/div[3]/input"))).send_keys(
        "*")
    wait.until(EC.element_to_be_clickable((By.XPATH, "/html/body/main/div/div/div/section/form/div[4]/a[1]"))).click()
    wait.until(EC.element_to_be_clickable((By.XPATH, "//*[contains(@id,'row_"+ username +"')]/td[2]"))).click()
    iframe = driver.find_element(By.ID, 'html_msg_body')
    print('line21 executed')
    driver.switch_to.frame(iframe)
    print('line23 executed')
    email_text = wait.until(EC.visibility_of_element_located(('tag name', 'body'))).text
    print('line25 executed')
    print(email_text)
    print('line27 executed')

Output: line21 executed line23 executed line25 executed

line27 executed

EDIT 2:

I've made the code to a public Mailinator. This is the following code to the email:

from selenium import webdriver
from selenium.webdriver.common.by import By
import time
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
# from faker import Faker
# import re

options = webdriver.ChromeOptions()
options.add_experimental_option('excludeSwitches', ['enable-logging'])
driver = webdriver.Chrome(options=options)
wait = WebDriverWait(driver, 20)
driver.get("https://www.mailinator.com/v4/public/inboxes.jsp?to=testingfor042223")
wait.until(EC.element_to_be_clickable((By.XPATH, '//*[@id="row_testingfor042223-1682214679-12524595"]/td[2]'))).click()
# wait.until(EC.element_to_be_clickable((By.XPATH, "/html/body/div[1]/main/div[2]/div[3]/div/div[4]/div/div/table/tbody/tr/td[2]"))).click()
iframe = driver.find_element(By.ID, 'html_msg_body')
print('line21 executed')
driver.switch_to.frame(iframe)
print('line23 executed')
body = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "body")))
print('line25 executed')
print(body.text)
print('line27 executed')

`.text` is assuredly working because everyone is using it so it's got to be something you are doing. Are you waiting for the element to be visible? Are you sure you are using a locator that actually contains text? It's hard to say without a [mcve]. We can't run your code since it's behind a login. — JeffC, Apr 22 '23 at 21:10

JeffC · Accepted Answer · 2023-04-23T02:21:41.217

0

With access to the email itself, I was able to get it working. The problem was that the page loads and then loads again, causing the original BODY to become stale. So, I grabbed the original BODY tag, waited for it to become stale, grabbed it again, and printed the text. I've verified the code below is working.

NOTE: I left in but commented out the lines that log in if you want to use that later, after resetting the URL.

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait

driver = webdriver.Chrome()
driver.maximize_window()
driver.get("https://www.mailinator.com/v4/public/inboxes.jsp?to=testingfor042223")

username = '*'
wait = WebDriverWait(driver, 10)
# wait.until(EC.visibility_of_element_located((By.ID, "many_login_email"))).send_keys("email")
# wait.until(EC.visibility_of_element_located((By.ID, "many_login_password"))).send_keys("password")
# wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[aria-label='Login link']"))).click()
wait.until(EC.element_to_be_clickable((By.XPATH, "//td[contains(text(),'Your verification code')]"))).click()

wait.until(EC.frame_to_be_available_and_switch_to_it((By.ID, "html_msg_body")))
body = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "body")))
wait.until(EC.staleness_of(body))
body = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "body")))
print(body.text)

edited Apr 23 '23 at 02:21

answered Apr 22 '23 at 21:14

JeffC

22,180
5
32
55

That did work. I've gone ahead and added a picture (it's the link at the top of the page) of exactly what the text I'm trying to extract looks like. Is there something incorrect with the way I'm trying to get access the iframe? – Rush Neighbors Apr 22 '23 at 22:00
I've updated my answer with another suggestion – JeffC Apr 22 '23 at 22:04
That still didn't work. This is the output:\n line21 executed\n line23 executed\n line25 executed\n \n line27 executed – Rush Neighbors Apr 22 '23 at 22:13
I've updated my answer again. This is the best I can do without seeing behind the login. Note the commented line at the bottom. – JeffC Apr 22 '23 at 22:31
That actually didn't work either, but it did find the error. Chrome updated itself without informing me to 112 2 days ago and I didn't update my webdriver from 111. – Rush Neighbors Apr 22 '23 at 22:54
If that was the issue, you should be receiving an error that your chromedriver is out of date. You stated in your question, `I'm not getting an error message`. As long as you are using Selenium 4.6+, my code should automatically download and install the current driver for you using [DriverManager](https://www.selenium.dev/blog/2022/introducing-selenium-manager/). – JeffC Apr 22 '23 at 22:55
I just finished updating my driver and retrying. It's still saying it can't find "body" with the CSS Selector which is what it was saying on top of the driver error it returned last after my last run. I'm about to make a public mailinator to try. They're only for anonymous test accounts that get wiped after running. EDIT: I wasn't getting an error. It was running. It just wasn't executing correctly. It compiled and ran, it just wasn't performing as expected, i.e. it wasn't printing anything other than the print commands that the lines were executed. – Rush Neighbors Apr 22 '23 at 23:24
I have updated the code to a public Mailinator so the code can be accessed freely by anyone. – Rush Neighbors Apr 23 '23 at 02:04
@RushNeighbors I've updated my answer with working code. – JeffC Apr 23 '23 at 02:22
1

I did accept the code as it did work and it handles my problem. I'm just going to adjust my approach accordingly. Thank you so much. – Rush Neighbors Apr 23 '23 at 04:21

Python Selenium cannot get anything when using .text method

1 Answers1