Playwright NotImplementedError

Question

I am running scrapy playwright on pycharm windows but i'm getting a Not ImplementedError.

Here's the error;

Traceback (most recent call last): File "c:\users\brainbox\appdata\local\programs\python\python38-32\lib\site-packages\playwright\_impl\_connection.py", line 166, in run await self._transport.run() File "c:\users\brainbox\appdata\local\programs\python\python38-32\lib\site-packages\playwright\_impl\_transport.py", line 60, in run proc = await asyncio.create_subprocess_exec( File "c:\users\brainbox\appdata\local\programs\python\python38-32\lib\asyncio\subprocess.py", line 236, in create_subprocess_exec transport, protocol = await loop.subprocess_exec( File "c:\users\brainbox\appdata\local\programs\python\python38-32\lib\asyncio\base_events.py", line 1630, in subprocess_exec transport = await self._make_subprocess_transport( File "c:\users\brainbox\appdata\local\programs\python\python38-32\lib\asyncio\base_events.py", line 491, in _make_subprocess_transport raise NotImplementedError NotImplementedError 2023-05-13 09:47:41 [asyncio] ERROR: Task exception was never retrieved future: <Task finished name='Task-4' coro=<Connection.run() done, defined at c:\users\brainbox\appdata\local\programs\python\python38-32\lib\site-packages\playwright\_impl\_connection.py:16 3> exception=NotImplementedError()> Traceback (most recent call last): File "c:\users\brainbox\appdata\local\programs\python\python38-32\lib\site-packages\playwright\_impl\_connection.py", line 166, in run await self._transport.run() File "c:\users\brainbox\appdata\local\programs\python\python38-32\lib\site-packages\playwright\_impl\_transport.py", line 60, in run proc = await asyncio.create_subprocess_exec( File "c:\users\brainbox\appdata\local\programs\python\python38-32\lib\asyncio\subprocess.py", line 236, in create_subprocess_exec transport, protocol = await loop.subprocess_exec( File "c:\users\brainbox\appdata\local\programs\python\python38-32\lib\asyncio\base_events.py", line 1630, in subprocess_exec transport = await self._make_subprocess_transport( File "c:\users\brainbox\appdata\local\programs\python\python38-32\lib\asyncio\base_events.py", line 491, in _make_subprocess_transport raise NotImplementedError NotImplementedError 2023-05-13 09:48:41

import scrapy
from quotes_js_scraper.items import QuoteItem


class QuotesSpider(scrapy.Spider):
    name = 'quotes'

    def start_requests(self):
        url = "https://quotes.toscrape.com/js/"
        yield scrapy.Request(url, meta={'playwright': True})

    def parse(self, response):

            yield{
                response.text
            }

Here's the settings;


# Scrapy settings for quotes_js_scraper project


BOT_NAME = 'quotes_js_scraper'

SPIDER_MODULES = ['quotes_js_scraper.spiders']
NEWSPIDER_MODULE = 'quotes_js_scraper.spiders'
# Obey robots.txt rules
ROBOTSTXT_OBEY = True

DOWNLOAD_HANDLERS = {
    "http": "scrapy_playwright.handler.ScrapyPlaywrightDownloadHandler",
    "https": "scrapy_playwright.handler.ScrapyPlaywrightDownloadHandler",
}

TWISTED_REACTOR = "twisted.internet.asyncioreactor.AsyncioSelectorReactor"
PLAQYWRIGHT_LAUNCH_OPTIONS = {
    "headless": False
}

Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. — Community, May 19 '23 at 23:46

score 0 · Answer 1 · answered Aug 01 '23 at 13:42

0

I ran into the same issue, reading the docs on https://github.com/scrapy-plugins/scrapy-playwright#known-issues, it's apparently not possible to run scrapy-playwright on windows.

answered Aug 01 '23 at 13:42

Gabriel Barata

1

Playwright NotImplementedError

Here's the error;

1 Answers1