0

I am having a hard time troubleshooting this error. Ideally, I would like it to just continue on error. I think that it is finding an empty box with that class... but I am not sure.

LINE 137 is count = int(soup.find("span", {"class": "totalcount"}).text)

LINE 127 is err = soup.find("div",{"class":"alert alert-sm alert-warning"}).text

Error:

Traceback (most recent call last):
  File "/home/domain/public_html/cron2021/craig.py", line 127, in main
    err = soup.find("div",{"class":"alert alert-sm alert-warning"}).text
AttributeError: 'NoneType' object has no attribute 'text'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/domain/public_html/cron2021/craig.py", line 170, in <module>
    main()
  File "/home/domain/public_html/cron2021/craig.py", line 137, in main
    count = int(soup.find("span", {"class": "totalcount"}).text)
AttributeError: 'NoneType' object has no attribute 'text'

Code:

            try:
                url = "%s%s&s=%s"%(uri,sub_url,num)
                full = {'https' : valid[i][2]}
                resp = requests.get(url,proxies=full)
                try:
                    print("[+]FULL PRINT: %s"%resp)
                    soup = BeautifulSoup(resp.text,"html.parser")
                    err = soup.find("div",{"class":"alert alert-sm alert-warning"}).text
                    if "Nothing found" in str(err):
                        print("[-]No Results Found For: %s"%url)
                        i=i+1
                        break
                    else:
                        raise ValueError
                except:
                    print("[+]Results Found For: %s"%url)
                    soup = BeautifulSoup(resp.text,"html.parser")
                    count = int(soup.find("span", {"class": "totalcount"}).text)
                    done = int(soup.find("span", {"class": "rangeTo"}).text)
                    tmp = soup.find("ul", {"class": "rows"})
                    els = tmp.findChildren("li")
                    j=0
                    for el in els:
                        if j == done:
                            break
                        data = product(el,res_id)
                        imgs = ','.join(data[4])
                        #[results_id,ad_id,ad_url,ad_title,images,price,date]
                        new_ids.append(data[1])
                        city = citi.replace("'","")
                        subcity = data[9].replace("'","")
                        curse.execute("INSERT IGNORE INTO mms_results (results_id, ad_id, ad_url, ad_title, ad_images, ad_price, ad_date, city, state, country, sub_location, source, scraped, active) VALUES ('%s','%s','%s','%s','%s','%s','%s','%s','%s','%s','%s','%s','%s','%s')"%(res_id,data[1],data[2],data[3],imgs,data[5],data[6],city,state,country,subcity,data[7],data[8],"1"))
                        mydb.commit()
                        j=j+1
                    if count != done:
                        num=num+120
                        continue
                    else:
                        i=i+1
                        break
            except requests.exceptions.ProxyError:
                i=i+1
                continue
            except requests.exceptions.ConnectTimeout:
                i=i+1
                continue

Actual page it is looking at: https://annapolis.craigslist.org/search/cta?auto_make_model=chevrolet%20nova&max_auto_year=1974&min_auto_year=1962&purveyor=owner

I appreciate any help.

Matt
  • 3
  • 1
  • 3
  • `try...finally`? – depperm Mar 10 '22 at 17:07
  • 2
    Store `soup.find` return values separately. Check `is None` before using `.text` on them – OneCricketeer Mar 10 '22 at 17:09
  • Does this answer your question? [Why do I get AttributeError: 'NoneType' object has no attribute 'something'?](https://stackoverflow.com/questions/8949252/why-do-i-get-attributeerror-nonetype-object-has-no-attribute-something) – Ulrich Eckhardt Mar 10 '22 at 17:35
  • Hello, thank you for your help. I tried this: ```if(soup.find("div",{"class":"alert alert-sm alert-warning"}) is not None): err = soup.find("div",{"class":"alert alert-sm alert-warning"}).text``` and am getting ```IndentationError: unindent does not match any outer indentation level``` – Matt Mar 10 '22 at 17:42

0 Answers0