How to access requesting URL in network tab listed using any programming language?

Question

Suppose there is a website which calls ajax requests everytime however I can't find(or see) any related URL in page source. But I can see every request or ajax calls in network tab using developer tools.

Please refer screenshot I attached to find URL in network tab. Let's say Google analytics link. How can I search the same URL using javascript or any programming language instead of opening network tab or any developer tool?

I want to see that request URL using programming language. I would prefer javascript if there is. Is it possible or any API available to find the usage of requested links?

I would appreciate if it is possible to get it done using Python, Javascript, Java or PHP.

HELP WOULD BE APPRECIATED!

You can use `PerfomanceObserver`, see https://stackoverflow.com/questions/45406906/detect-ajax-requests-from-raw-html/ — guest271314, Nov 10 '17 at 06:41
I have gone through your link. but it was not really helpful. Can you please give an example? I was not able to see all the links. — Baba ramdev, Nov 10 '17 at 06:44
What do you mean by "not really helpful"? Have you tried the code at the Answer at linked Question? — guest271314, Nov 10 '17 at 06:49
Can you describe "not working"? At which browser did you try code? Can you reproduce the issue at stacksnippets? — guest271314, Nov 10 '17 at 06:51
I tried it on chrome using browser console but nothing is being after writing that snippet in console — Baba ramdev, Nov 10 '17 at 06:54
@Babaramdev, will you please try out mine snippet same way ? — Vivek Doshi, Nov 10 '17 at 06:59
One option is to use the QT framework and use [WebView](http://doc.qt.io/qt-5/qml-qtwebview-webview.html) as well as [QNetworkAccessManager] (http://doc.qt.io/qt-5/qnetworkaccessmanager.html) to build your own 'network montor'. QT is available for multiple languages (Python, C++, Lua, Haskell, ...) — Maurice Meyer, Nov 10 '17 at 08:34

score 0 · Answer 1 · answered Jan 12 '23 at 12:45

You can use a browser automation tool like Selenium.

Try this python code.

from selenium import webdriver

driver = webdriver.Chrome()
driver.get("yourwebsiteurl here")
driver.implicitly_wait(10)
ajax_urls = driver.execute_script("return performance.getEntries().map(entry => entry.name)")

for url in ajax_urls:
    print(url)

How to access requesting URL in network tab listed using any programming language?

1 Answers1