Bs4 AttributeError: 'NoneType' object has no attribute 'get_text'

Question

I just started the project of an app that tracks Amazon prices. According to the tutorial video, at this step when I run the code it has to show the product's name and price, but the only thing that comes out is the next error: AttributeError: 'NoneType' object has no attribute 'get_text'. I already tried it with another pages and revised the same topics here but didn't find the answer. Please help ;(

The code itself:

import requests
from bs4 import BeautifulSoup

URL = 'https://www.amazon.es/dp/B07JQRWLXM/ref=nav_signin?pf_rd_r=FJX3CJC8RWFE3NPQJRNP&'

headers = {
"User-Agent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) 
 Chrome/86.0.4240.75 Safari/537.36'}

 page = requests.get(URL, headers=headers)

 soup = BeautifulSoup(page.content, 'html.parser')

 title = soup.find(id="productTitle").get_text()
 price = soup.find(id = "priceblock_ourprice").get_text()
 converted_price = float(price[0:3])

 print(converted_price)
 print(title.strip())

This is because `soup.find()` call does not find a match. BeautifulSoup [find() documentation](https://www.crummy.com/software/BeautifulSoup/bs4/doc/#find) contains the following: "If find() can’t find anything, it returns None:" — Attila Viniczai, Oct 19 '20 at 22:10
Yeah I'm positive, on Amazon every product's title has this id when you open a product's page. — Valeria Ku, Oct 19 '20 at 22:13
@AttilaViniczai so the problem is invalid id then? Because I can't find another — Valeria Ku, Oct 19 '20 at 22:15
The cause might be a [captcha](https://en.wikipedia.org/wiki/CAPTCHA) prompt. After trying the link posted in the example a few times I have encountered a captcha. — Attila Viniczai, Oct 19 '20 at 22:25

score 0 · Accepted Answer · answered Oct 19 '20 at 22:20

Your script is almost correct, just use lxml or html5lib parser instead of html.parser:

import requests
from bs4 import BeautifulSoup

URL = 'https://www.amazon.es/dp/B07JQRWLXM/ref=nav_signin?pf_rd_r=FJX3CJC8RWFE3NPQJRNP&'

headers = {"User-Agent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.75 Safari/537.36'}

page = requests.get(URL, headers=headers)

soup = BeautifulSoup(page.content, 'lxml')   # <-- change to 'lxml' or 'html5lib'

title = soup.find(id="productTitle").get_text()
price = soup.find(id = "priceblock_ourprice").get_text()
converted_price = float(price.split()[0].replace(',', '.'))   # <-- change the conversion method


print(converted_price)
print(title.strip())

Prints:

69.99
Tablet Fire 7, pantalla de 7'', 16 GB (Negro) - Incluye ofertas especiales

Bs4 AttributeError: 'NoneType' object has no attribute 'get_text'

1 Answers1