0

I wrote about 10 scrapper using undetected chromedriver. They were working all fine until i wanted to create a next one. I started working on it and suddenly i started geting timeouts. When i removed headless, i saw that the cloudflare waiting room that i passed with no struggle kept me in an infinate loop. After that it seems like all of my previous code also broke. Here is a part of the code :

def GetTotalPages(self):
    options = ChromeOptions()
    options.add_argument('--head')
    # options.add_argument('--disable-blink-features=AutomationControlled')
    options.add_argument("start-maximized")

    with Chrome(options=options) as driver:
       
        driver.get("https://mangasehri.com/manga/?m_orderby=alphabet")

        WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CLASS_NAME, "pages")))
        Lastpage = driver.find_element(By.CLASS_NAME, "pages").text
        Lastpage = int(Lastpage.split(" ")[1])
        print(Lastpage,"total pages")

    return Lastpage

I have tried switching to gecko driver, adding options, nothing seems to be working. Is there an update that has been relased for cloudflare thet blocks it ? Or just my ip got blocked ? if so, how can i get around this issue.

PlatinMavi
  • 53
  • 7

1 Answers1

0

You can use SeleniumBase's UC Mode as an alternative.

First pip install -U seleniumbase, and then run the following script with python: (The script bypasses a site protected by Cloudflare)

from seleniumbase import Driver
import time

driver = Driver(uc=True)
driver.get("https://nowsecure.nl/#relax")
time.sleep(6)
driver.quit()
Michael Mintz
  • 9,007
  • 6
  • 31
  • 48
  • Still not able to acces the site i gave in the question. But it fixed some of the scrappers i wrote previously. I was able to acces : https://webtoon-tr.com/ (one of the websites i wrote a scrapper previously) but the link in the question: https://mangasehri.com/ still cant pass cloudflare. – PlatinMavi Jul 13 '23 at 20:40