0

I was wondering why when my scrape_quotes() function is ran it is throwing an error saying UnboundLocalError: local variable 'url' referenced before assignment(line 11). Why is it doing that when I have assigned it before I referenced it?

Either way, why is that an error and there is no error when I access BASE_URL? I'm very much confused...

import requests
from bs4 import BeautifulSoup
from time import sleep
from random import choice

BASE_URL = "http://quotes.toscrape.com"
url = "/page/1/"

def scrape_quotes():
    all_quotes = []
    while url:
        response = requests.get(f"{BASE_URL}{url}")
        print(f"Now Scraping {BASE_URL}{url}")
        soup = BeautifulSoup(response.text, "html.parser")
        quotes = soup.select(".quote")

        for quote in quotes:
            all_quotes.append({
                "quote" : quote.select(".text")[0].get_text(),
                "author" : quote.select(".author")[0].get_text(),
                "bio-link" : quote.select("a")[0].attrs['href']
                })
        next_btn = soup.select(".next")
        url = next_btn[0].select("a")[0].attrs['href'] if next_btn else None
        sleep(2)
    return all_quotes
AMC
  • 2,642
  • 7
  • 13
  • 35
loneliness
  • 33
  • 1
  • 6
  • 1
    `BASE_URL` and `url` should be parameters of the `scrape_quotes` function, not global variables. – Jacques Gaudin Apr 07 '20 at 17:14
  • Please provide the entire error message. – AMC Apr 07 '20 at 19:46
  • Does this answer your question? [How to change a variable after it is already defined?](https://stackoverflow.com/questions/41369408/how-to-change-a-variable-after-it-is-already-defined) – AMC Apr 07 '20 at 19:47

1 Answers1

1

Because it is defined as a global variable. To modify it you need to use the global keyword, i.e.

    global url
    # scrape
    url = next_url

However, I would recommend assigning it inside the function instead. If you are using it outside the function as well, then pass it as a parameter to the function.

Badgy
  • 819
  • 4
  • 14