Scrape text with playwright is slow on part of the code (using page.locator)

Question

I have made a script in playwright, to scrape some text. This is the part i have problems, here i scrape a series name, using locator:

# Series
global nfo_series
try:
  nfo_series = page.locator("#Series > span:nth-child(1)")
  nfo_series = nfo_series.inner_text()
  logger_scraper.info('nfo_series is now:%s', nfo_series)
except:
  logger_scraper.info('This book as no series.')

and it works, when the series are not present is skipped, but it takes 30 seconds to the code move on, the log is below, what is the problem, can i improve it?

19:28:31 nfo_author is now:Thomas Fincham
19:29:01 This book as no series.

i think its because of the try statement, but why? as i have the same code, for per example, author and it is fast. i know it is not good to use playwright, best are other modules.

score 0 · Accepted Answer · answered Dec 19 '21 at 09:42

Playwright's default timer is 30 seconds. So in the try part, it tries for 30 seconds to get the series, but there is none, so after 30 seconds only, it goes to the exception.

You can set a custom timout when calling inner_text().

See: https://playwright.dev/python/docs/api/class-locator#locator-inner-text

Scrape text with playwright is slow on part of the code (using page.locator)

1 Answers1