0

I am making a code with Selenium in Python and collecting data on a website, and I have noticed that in Chrome's Devtools specifically in the "Network" section I have a file named "search", which gives me all the information that I need neat, this file is updated every 10 seconds

Is there a way to get the data from that file every time it appears again, that is, every time the file is updated it generates another file with the same name, I would like to be able to have access to that information whenever it generates a new one

Picture of what I need

I was looking a bit in the Selenium documentation but I did not find much, and looking in some posts, I have only seen that they have taken data but from Performance, but I do not have any data there

If you could guide me a little or provide a documentation that talks more about that and how this information could be obtained, I would appreciate it.

Val K.
  • 3
  • 2
xAnThOnYx
  • 1

1 Answers1

0

I did solve this issue with Selenium-Wire package https://pypi.org/project/selenium-wire/.

  1. It has a method .wait_for_request to wait certain request before moving on
  2. To fetch data you can use something like that:
def get_request_counter_by_query(driver: webdriver, host: str, query: str) -> int:
    for request in driver.requests:
            if request.host == host and query in request.querystring:
                   
                ...do something

Also Selenium 4 will support CDP command which you can use to do same))

Val K.
  • 3
  • 2