10

I have a webdriver using selenium that opens a browser for me, points it to an ip Address, does a bunch of stuff and closes.

I want to know all of the urls accessed during this time. That is, any ads that are loaded, any css calls that were made out to any url and so on.

Here is the code im using

from selenium import webdriver

browser = webdriver.Firefox(profile) # Get local session of firefox
browser.get(url) # Open a url and wait for it to finish
Johnny
  • 14,397
  • 15
  • 77
  • 118
Cripto
  • 3,581
  • 7
  • 41
  • 65
  • I think this is a dupe of http://stackoverflow.com/questions/3712278/selenium-rc-how-do-you-use-capturenetworktraffic-in-python (although I'm not certain that it gives you all the information that you want). Also possibly relevant: http://blog.qaevangelist.com/?p=317 – Mark Amery Jul 17 '13 at 21:52
  • Its close. But I need the web driver to capture the traffic. :( – Cripto Jul 17 '13 at 23:37
  • 1
    @MarkAmery hmm, it looks like the post you linked (stackoverflow.com/questions/3712278) uses Selenium v1. Cripto and I are curious how to monitor network traffic in Selenium v2. (proxy server would work, but it'd be awesome to find a solution built into Selenium v2) – solvingPuzzles Jul 14 '14 at 00:23

2 Answers2

1

I did it by loading the firefox plugins Firebug and Netexport. The first is a tool that allows you to see all the exchange of information, the second allows to write all of it in a file (.har extension). So basically selenium has to load the plugins, the website and wait the time you want, and when it closes, you get a file with the result.

chuse
  • 373
  • 3
  • 18
1

Its not a python solution.. But you can add fiddler plug in to Firefox. We needed to do exact same thing about a year ago. We used selenium to open browser and all UI stuff and in background Fiddler captured all traffic (http and https) .. This also list all JS CSS src and you can debug later with inspector see what request is sent and what response is received

PxP
  • 31
  • 3