i would like to run scrapy along with another asyncio script in the same file, but am unable to.
(using the asyncio reactor in the settings:
# TWISTED_REACTOR = "twisted.internet.asyncioreactor.AsyncioSelectorReactor")
asyncio.run(playwright_script())
process = CrawlerProcess(get_project_settings())
process.crawl(TestSpider)
process.start()
this raises the error:
RuntimeError: There is no current event loop in thread 'MainThread'.
i'm guessing because the previous run command blocks the event loop?
i have tried to add the scrapy commands into a coroutine like so:
async def run_scrapy():
process = CrawlerProcess(get_project_settings())
process.crawl(TestSpider)
process.start()
asyncio.run(run_scrapy_script())
but receive the error:
RuntimeError: This event loop is already running
How do i properly configure the scrapy CrawlerProcess to run on my asyncio loop?
