-1

Within the following code, I can't get past the first if statement and proceed to the else statement, even when I purposely make it false:

while not scraped:
    print("sleep....")
    time.sleep(1)
    try:
        res = requests.get(url).content
        soup = BeautifulSoup(res, 'lxml')
        links = soup.find_all("span", {"id":"reasonabledoubt"})
        dip = soup.find_all("div")
        print("searching divs")
        if 'keyword' in str(dip) == True:
            print(url)
            print("LINK SCRAPED")
            print(url + " link scraped")
            scraped = True
        else:
            for word in links:
                 print("testing for loop")
                 #rest of code

So basically if the keyword isn't found in str(dip), I need the else clause to execute.

Jonas
  • 121,568
  • 97
  • 310
  • 388
ColeWorld
  • 279
  • 1
  • 3
  • 15
  • You could reduce this example to one or two lines of code. See how to create a [mcve]. – Peter Wood Jan 14 '17 at 23:21
  • @PeterWood , I was not sure if including the while loop and `try` were relevant to the issue I'm facing, the answer suggested hasn't worked for me... – ColeWorld Jan 14 '17 at 23:27
  • The answer should solve it unless you have another issue; I'm assuming you're actually looking for the string `'keyword'` rather than a variable called `keyword` – Chris_Rands Jan 14 '17 at 23:41
  • if you get rid of everything except the `if` and `else`, and hard code a value for `dip` you'll be able to test your assumptions. Try it in the interactive console as well, it's a great place to experiment and to clarify your understanding. – Peter Wood Jan 15 '17 at 00:10

1 Answers1

2

You need:

if 'keyword' in str(dip):

Your old syntax:

if 'keyword' in str(dip) == True:

Is equivalent to:

if 'keyword' in str(dip) and str(dip) == True

This is Python's chaining behaviour.

Chris_Rands
  • 38,994
  • 14
  • 83
  • 119