I have made a script to scrap a website for my company. Everything was more or less fine, until I get the information that selenium wasn't made to collect trafic and performance data. That was what made me know about browsermob-proxy. I tried to use it :
from browsermobproxy import Server
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
dict={'port':8090}
opt={'captureHeaders':True}
server=Server("C:\\[...]\\browsermob-proxy-2.1.4\\bin\\browsermob-proxy.bat",options=dict)
server.start()
cap = DesiredCapabilities().INTERNETEXPLORER
cap['platform'] = "Win7"
cap['version'] = "11"
cap['browserName'] = "internet explorer"
cap['ignoreProtectedModeSettings'] = True
cap['IntroduceInstabilityByIgnoringProtectedModeSettings'] = True
cap['nativeEvents'] = True
cap['ignoreZoomSetting'] = True
cap['requireWindowFocus'] = True
cap['INTRODUCE_FLAKINESS_BY_IGNORING_SECURITY_DOMAINS'] = True
cap['enablePersistentHover'] = True
cap['ie.usePerProcessProxy'] = True
cap.setdefault("nativeEvents", False)
proxy.add_to_webdriver_capabilities(cap)
proxy=server.create_proxy()
url="http://localhost/"
driver_path="C:\\[...]\\IEDriverServer.exe"
delay=5
browser=webdriver.Ie(capabilities=cap, executable_path=driver_path)
proxy.new_har("focus", options=opt)
browser.implicitly_wait(delay)
browser.maximize_window()
browser.get(url)
print(proxy.har)
The value of my variable cap is :
{'browserName': 'internet explorer', 'version': '11', 'platform': 'Win7', 'ignoreProtectedModeSettings': True, 'IntroduceInstabilityByIgnoringProtectedModeSettings': True, 'nativeEvents': True, 'ignoreZoomSetting': True, 'requireWindowFocus': True, 'INTRODUCE_FLAKINESS_BY_IGNORING_SECURITY_DOMAINS': True, 'enablePersistentHover': True, 'ie.usePerProcessProxy': True, 'proxy': {'proxyType': 'MANUAL', 'httpProxy': 'localhost:8103', 'sslProxy': 'localhost:8103'}}
But what I get is this :
{'log': {'version': '1.2', 'creator': {'name': 'BrowserMob Proxy', 'version': '2.1.4', 'comment': ''}, 'pages': [{'id': 'focus', 'startedDateTime': '2018-06-15T09:58:16.657+02:00', 'title': 'focus', 'pageTimings': {'comment': ''}, 'comment': ''}], 'entries': [], 'comment': ''}}
There is nothing to see... Can anybody tell me what I'm doing wrong?
Unfortunately, I'm forced to work with IE (11 actually).
I'm using :
- Python 3.6.5
- Selenium 3.12.0
- browsermob-proxy 0.8.0
- BMP 2.1.4
