I am trying to code a simple script to parse a website for links from anchor tags that have a certain role attribute. I looked through the bs4 documentation as well as any other source on why this should not be the case. All the links I am looking for has the same role attribute. I tried to write to a csv as well, the csv was also empty.
import requests
from bs4 import BeautifulSoup
import csv
url = 'https://opensea.io/rankings/trending?category=collectibles'
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')
links = []
# Defining the function to scrape the data
def scrape_data(url):
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')
for row in soup.find_all('a', attrs={'role' : 'row'}):
print(row.get('href'))
scrape_data(url)
