0

I tried setting up Scrapy Playwright.

Without proxy, cookie applied correctly. But when I use proxy (brightdata), then the cookie is not applied. Did I miss anything?

class ScrapyTest(scrapy.Spider):
    name = 'scrapy test'

    def start_requests(self):

        cookies = {
            'cookieconsent_dismissed': 'yes'
        }

        url = 'https://example.com'

        yield scrapy.Request(url, cookies=cookies, meta={"playwright": True}, callback=self.parse)

settings.py

PLAYWRIGHT_LAUNCH_OPTIONS = {
    "proxy": {
        'server': 'http://zproxy.lum-superproxy.io:22225',
        'username': 'lum-customer-user',
        'password': 'password'
    }
}

COOKIES_ENABLED = True
Jayavinoth
  • 544
  • 1
  • 9
  • 24

0 Answers0