0

I have been trying to web scrape from this website: https://octane.gg/events/e83e-rlcs-x-championship-europe/stats/players

I want to get specific ratings but i get nothing when i execute this code:

from bs4 import BeautifulSoup
import requests

result = requests.get("https://octane.gg/events/e83e-rlcs-x-championship-europe/stats/players")
src = result.content
soup = BeautifulSoup(src, 'lxml')

match = soup.find('div', class_='css-gm45eu')
print(match)

output: None

How can I scrape what is in that class?

ish
  • 3
  • 1
  • 1
    it is returning None because this page is dynamically loaded via javascript. More about it in this SO [question](https://stackoverflow.com/questions/55709463/how-to-scrape-dynamic-content-from-a-website) – Eduardo Coltri Aug 24 '21 at 15:13

1 Answers1

1

Try to use Selenium:

from selenium import webdriver

driver = webdriver.Chrome('PATH_TO --> chromedriver.exe')
driver.get("https://octane.gg/events/e83e-rlcs-x-championship-europe/stats/players")
ratings = driver.find_elements_by_xpath('//div[@class="css-gm45eu"]')
ratings_list = []
for p in range(len(ratings)):
    ratings_list.append(ratings[p].text)
print(ratings_list)

Output:

['1.189', '1.109', '1.098', '1.031', '1.028', '1.005', '0.990', '0.981', '0.967', '0.936', '0.904', '0.846', '0.841', '0.840', '0.836', '0.809', '0.759', '0.726']

Download chromedriver.exe: https://chromedriver.chromium.org/downloads

if you don't want a chrome window to open while running, use this code:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.add_argument('--headless')
driver = webdriver.Chrome('PATH_TO --> chromedriver.exe', options=options)
driver.get("https://octane.gg/events/e83e-rlcs-x-championship-europe/stats/players")
ratings = driver.find_elements_by_xpath('//div[@class="css-gm45eu"]')
ratings_list = []
for p in range(len(ratings)):
    ratings_list.append(ratings[p].text)
print(ratings_list)
dimelu
  • 68
  • 3