0

I am trying to find and print the links that occur only inside specific divs on a page.

import requests
from bs4 import BeautifulSoup

r = requests.get('https://targeturl.com/')
soup = BeautifulSoup(r.text, 'html.parser')

for div in soup.find_all('div'):
    if 'target-class' in div.get('class'):
        # find the link in the div
        for link in div.find_all('a'):
            print(link.get('href'))

When I try to run the above, I get TypeError: argument of type 'NoneType' is not iterable

Is this because some of the target-class divs do not contain a link? And thus the loop can't iterate over it?

reallymemorable
  • 882
  • 1
  • 11
  • 28
  • 1
    This error mean that you are trying to loop over something that can't be looped through - in this case you are trying to loop through something that returns `None`. Based on that, either `for div in soup.find_all('div')` or `for link in div.find_all('a')` is returning `None` because there were no matches found – Nicholas Hansen-Feruch Nov 19 '22 at 21:40
  • @reallymemorable Please try to make your questions more replicable. Using dummy names such as target-class and targeturl means we can only guess why you are getting a variable valued None as there are 3 possible failure points in your code. – ferreiradev Nov 19 '22 at 21:44
  • 1
    `find_all` shouldn't return `None`, it should always return a list, even if it's an empty list. So that leaves the other `in`, which ought to be the one on the line number that you got an exception for: `div.get('class')` – Dan Getz Nov 19 '22 at 21:54
  • `if 'target-class' in div.get('class'):` I'm going to guess that the error happens here. There is at least one `
    ` element that has no `class` attribute, and therefore this statement becomes `if 'target-class' in None` which causes the error.
     – John Gordon Nov 19 '22 at 22:00

0 Answers0