0

I am busy doing a tutorial in python, and I am busy doing some Web Scraping. When the instructor runs the program he does not get any errors, however, when I run the program I receive an attribute error. The tutorial was made late last year, so the format of the website has changed a bit. however, I have managed to work around that. I am web scraping forecast.weather.gov The weird thing is that I ran a similar line before that and it worked perfectly, so I am really confused! Here is the code:

import requests
from bs4 import BeautifulSoup

page = requests.get('https://forecast.weather.gov/MapClick.php?lat=34.05349000000007&lon=-118.24531999999999#.X8qd1NgzZPY')
soup = BeautifulSoup(page.content, 'html.parser')
week = soup.find(id="seven-day-forecast-body")
# print(week)

items = week.find_all(class_='forecast-tombstone')
#print(items)

print(items[1].find(class_='period-name').get_text())
print(items[1].find(class_='short-desc').get_text())
print(items[1].find(class_='temp').get_text())

period_names = [item.find(class_='period-name').get_text() for item in items]
short_descriptions = [item.find(class_='short-desc').get_text() for item in items]
temperature = [item.find(class_='temp').get_text() for item in items]
print(period_names)
print(short_descriptions)
print(temperature)

And this is the output:

Overnight
Partly Cloudy
Low: 49 °F
Traceback (most recent call last):
  File "C:/Users/Bongi/PycharmProjects/Qazicourse/Web Scraping.py", line 18, in <module>
    temperature = [item.find(class_='temp').get_text() for item in items]
  File "C:/Users/Bongi/PycharmProjects/Qazicourse/Web Scraping.py", line 18, in <listcomp>
    temperature = [item.find(class_='temp').get_text() for item in items]
AttributeError: 'NoneType' object has no attribute 'get_text'

Process finished with exit code 1

Notice how there is only an error with this line,

temperature = [item.find(class_='temp').get_text() for item in items]

and not this one,print(items[1].find(class_='temp').get_text())

I am a complete beginner with coding(not just python), so I am very sorry if I made this too long. Any suggestions would be much appreciated!

Victoria T
  • 1
  • Your code works fine for me. Maybe you have been requesting too often and weather.gov uses some kind of scraping protection, resulting in a different response. – Maurice Meyer Dec 06 '20 at 11:50
  • The reason you get this is that `item.find(class_='temp')` is `None`. Either handle errors with `try: ... except: ...` or check if it is `None` before accesing it. See [ask-forgiveness-not-permission-explain](https://stackoverflow.com/questions/12265451/ask-forgiveness-not-permission-explain) – Patrick Artner Dec 06 '20 at 11:57
  • Hi, the code works fine now :). It was just the layout of the website, the changed it 2-3 days after I posted this. – Victoria T Dec 11 '20 at 18:00

0 Answers0