0

How to run multiple Selenium sessions from one session ID in Python I am getting an issue while running multiple set of scraping task using same session id/user then the first task closes and new gets started.

We are using a static proxy while scraping linkedin after login. Methods we have used so far: 1 - using add cookies (refer code below)

    cookie = driver.get_cookies()

    try:
        for cookie in cookies:
            driver.add_cookie(cookie)
     except Exception as error:
        pass

2 - using assign previous session id

    session_url = driver.command_executor._url  
    session_id = driver.session_id

    new_driver = webdriver.Remote(command_executor=session_url, desired_capabilities={})
    new_driver.session_id = session_id
    new_driver.get("https://www.linkedin.com/")

So we have to scrape/automate multiple set of task simultaneously.above two method works very fine without proxies.

we have tried above two method mentioned.

Akshay Sharma
  • 1
  • You need a separate sessionID for each thread. That ID points to one browser's session. Each thread will need it's own driver/browser pair and it's own sessionID for the driver to communicate with the browser's dev protocol. The way you are doing this you'll have two drivers, but they are both connected to one browser. (You'll have an orphaned browser...) – pcalkins Apr 24 '23 at 23:09

0 Answers0