I want to analyze the usage of RAM when executing this single line of code
webdriver.Chrome().get("url")
I want to get results of memory usage like what Task Manager on Chrome shows
I tried several methods, but I couldn't even get similar results.
First, I used the following code before and after calling driver.get("get") to compare the memory usage info by using the PID of Webdrive (Parent) and its children PIDs.
p = psutil.Process(driver.service.process.pid)
print('webdriver pid -{} START INFO: {}'.format(str(p), p.memory_info().rss))
pc = p.children(recursive=True)
for id in pc:
print('children pid -{} START INFO: {}'.format(str(id), psutil.Process(id.pid).memory_info().rss))
Here is the first result that first loads the page with a new URL from a driver that is just initiated.
Before driver.get("url")
webdriver pid -psutil.Process(pid=20742, name='chromedriver', status='sleeping', started='03:42:14') START INFO: 6975488 Bytes
children pid -psutil.Process(pid=20748, name='chrome', status='sleeping', started='03:42:14') START INFO: 88371200
children pid -psutil.Process(pid=20753, name='cat', status='sleeping', started='03:42:14') START INFO: 401408
children pid -psutil.Process(pid=20754, name='cat', status='sleeping', started='03:42:14') START INFO: 405504
children pid -psutil.Process(pid=20756, name='chrome', status='sleeping', started='03:42:14') START INFO: 35823616
children pid -psutil.Process(pid=20757, name='chrome-sandbox', status='sleeping', started='03:42:14') START INFO: 708608
children pid -psutil.Process(pid=20836, name='chrome', status='sleeping', started='03:42:21') START INFO: 54599680
children pid -psutil.Process(pid=20758, name='chrome', status='sleeping', started='03:42:14') START INFO: 35983360
children pid -psutil.Process(pid=20763, name='chrome-sandbox', status='sleeping', started='03:42:14') START INFO: 708608
children pid -psutil.Process(pid=20766, name='chrome', status='sleeping', started='03:42:14') START INFO: 11218944
children pid -psutil.Process(pid=20846, name='chrome', status='sleeping', started='03:42:21') START INFO: 15519744
children pid -psutil.Process(pid=20869, name='chrome', status='sleeping', started='03:42:21') START INFO: 23572480
children pid -psutil.Process(pid=20875, name='chrome', status='sleeping', started='03:42:21') START INFO: 39026688
children pid -psutil.Process(pid=20764, name='nacl_helper', status='sleeping', started='03:42:14') START INFO: 1679360
children pid -psutil.Process(pid=20832, name='chrome', status='sleeping', started='03:42:21') START INFO: 86355968
children pid -psutil.Process(pid=20945, name='chrome', status='sleeping', started='03:42:29') START INFO: 27017216
webdriver -20742 used memory: 6928.0 KB
Children -20748-20945 used memory: 657304.0 KB
Totoal used memory: 664232.0 KB
After driver.get("url")
webdriver pid -psutil.Process(pid=20742, name='chromedriver', status='sleeping', started='03:42:14') START INFO: 7094272
children pid -psutil.Process(pid=20748, name='chrome', status='sleeping', started='03:42:14') START INFO: 92909568
children pid -psutil.Process(pid=20753, name='cat', status='sleeping', started='03:42:14') START INFO: 401408
children pid -psutil.Process(pid=20754, name='cat', status='sleeping', started='03:42:14') START INFO: 405504
children pid -psutil.Process(pid=20756, name='chrome', status='sleeping', started='03:42:14') START INFO: 35823616
children pid -psutil.Process(pid=20757, name='chrome-sandbox', status='sleeping', started='03:42:14') START INFO: 708608
children pid -psutil.Process(pid=20836, name='chrome', status='sleeping', started='03:42:21') START INFO: 57266176
children pid -psutil.Process(pid=20758, name='chrome', status='sleeping', started='03:42:14') START INFO: 35983360
children pid -psutil.Process(pid=20763, name='chrome-sandbox', status='sleeping', started='03:42:14') START INFO: 708608
children pid -psutil.Process(pid=20766, name='chrome', status='sleeping', started='03:42:14') START INFO: 11218944
children pid -psutil.Process(pid=20846, name='chrome', status='sleeping', started='03:42:21') START INFO: 16039936
children pid -psutil.Process(pid=20869, name='chrome', status='sleeping', started='03:42:21') START INFO: 64634880
children pid -psutil.Process(pid=20966, name='chrome', status='sleeping', started='03:42:31') START INFO: 22982656
children pid -psutil.Process(pid=20764, name='nacl_helper', status='sleeping', started='03:42:14') START INFO: 1679360
children pid -psutil.Process(pid=20832, name='chrome', status='sleeping', started='03:42:21') START INFO: 96940032
children pid -psutil.Process(pid=20945, name='chrome', status='sleeping', started='03:42:29') START INFO: 27017216 Bytes
webdriver -20742 used memory: 6928.0 KB
Children -20748-20945 used memory: 453828.0 KB
Total used memory: 460756.0 KB
As we can see here PID 20875 before calling driver.get("url") is terminated when calling after executing driver.get("url"), and new PID 20966 after calling driver.get("url") is created. But this will not happen when redirecting from an URL page to another URL page. It occurs only when the driver is newly defined and is directed to an URL.
Here is what Chrome Task Manager1 shows for the first try.
| Task | Memory footprint | Process ID |
|---|---|---|
| Browser | 24,648 K | 20748 |
| GPU Process | 59,220 K | 20832 |
| Utility: Network Service | 20,860 k | 20836 |
| Utility: Storage Service | 10,752 k | 20846 |
| Spare Renderer | 13,024 k | 20966 |
| Tab: Google | 25,400 k | 20869 |
Ex. PID 20869 64634880/1024 = 63,120 != 25,400 KB or (64634880 - 23572480) /1024 = 40,100 != 25,400 KB Results here do not match.
Here is the result that we redirect the same driver to the same URL.
Before driver.get("url")
webdriver pid -psutil.Process(pid=32326, name='chromedriver', status='sleeping', started='04:10:11') START INFO: 7118848 Bytes
children pid -psutil.Process(pid=32332, name='chrome', status='running', started='04:10:11') START INFO: 91762688
children pid -psutil.Process(pid=32337, name='cat', status='sleeping', started='04:10:11') START INFO: 405504
children pid -psutil.Process(pid=32338, name='cat', status='sleeping', started='04:10:11') START INFO: 401408
children pid -psutil.Process(pid=32340, name='chrome', status='sleeping', started='04:10:11') START INFO: 35823616
children pid -psutil.Process(pid=32341, name='chrome-sandbox', status='sleeping', started='04:10:11') START INFO: 708608
children pid -psutil.Process(pid=32427, name='chrome', status='sleeping', started='04:10:18') START INFO: 57339904
children pid -psutil.Process(pid=32625, name='xdg-settings', status='sleeping', started='04:10:36') START INFO: 1740800
children pid -psutil.Process(pid=32629, name='xprop', status='sleeping', started='04:10:36') START INFO: 1150976
children pid -psutil.Process(pid=32630, name='grep', status='sleeping', started='04:10:36') START INFO: 991232
children pid -psutil.Process(pid=32342, name='chrome', status='sleeping', started='04:10:11') START INFO: 35983360
children pid -psutil.Process(pid=32347, name='chrome-sandbox', status='sleeping', started='04:10:11') START INFO: 708608
children pid -psutil.Process(pid=32350, name='chrome', status='sleeping', started='04:10:11') START INFO: 11218944
children pid -psutil.Process(pid=32436, name='chrome', status='sleeping', started='04:10:18') START INFO: 15757312
children pid -psutil.Process(pid=32453, name='chrome', status='running', started='04:10:18') START INFO: 66072576
children pid -psutil.Process(pid=32544, name='chrome', status='sleeping', started='04:10:27') START INFO: 23273472
children pid -psutil.Process(pid=32348, name='nacl_helper', status='sleeping', started='04:10:11') START INFO: 1679360
children pid -psutil.Process(pid=32423, name='chrome', status='sleeping', started='04:10:18') START INFO: 92196864
children pid -psutil.Process(pid=32535, name='chrome', status='sleeping', started='04:10:26') START INFO: 27021312
webdriver -17562 used memory: 6948.0 KB
Children -17570-17766 on the list used memory: 680864.0 KB
Totoal Parent + Children used memory: 687812.0 KB
After driver.get("url")
webdriver pid -psutil.Process(pid=32326, name='chromedriver', status='sleeping', started='04:10:11') START INFO: 7114752 Bytes
children pid -psutil.Process(pid=32332, name='chrome', status='sleeping', started='04:10:11') START INFO: 92536832
children pid -psutil.Process(pid=32337, name='cat', status='sleeping', started='04:10:11') START INFO: 405504
children pid -psutil.Process(pid=32338, name='cat', status='sleeping', started='04:10:11') START INFO: 401408
children pid -psutil.Process(pid=32340, name='chrome', status='sleeping', started='04:10:11') START INFO: 35823616
children pid -psutil.Process(pid=32341, name='chrome-sandbox', status='sleeping', started='04:10:11') START INFO: 708608
children pid -psutil.Process(pid=32427, name='chrome', status='sleeping', started='04:10:18') START INFO: 57380864
children pid -psutil.Process(pid=32625, name='xdg-settings', status='sleeping', started='04:10:36') START INFO: 1740800
children pid -psutil.Process(pid=32629, name='xprop', status='sleeping', started='04:10:36') START INFO: 1150976
children pid -psutil.Process(pid=32630, name='grep', status='sleeping', started='04:10:36') START INFO: 991232
children pid -psutil.Process(pid=32342, name='chrome', status='sleeping', started='04:10:11') START INFO: 35983360
children pid -psutil.Process(pid=32347, name='chrome-sandbox', status='sleeping', started='04:10:11') START INFO: 708608
children pid -psutil.Process(pid=32350, name='chrome', status='sleeping', started='04:10:11') START INFO: 11218944
children pid -psutil.Process(pid=32436, name='chrome', status='sleeping', started='04:10:18') START INFO: 15757312
children pid -psutil.Process(pid=32453, name='chrome', status='running', started='04:10:18') START INFO: 78528512
children pid -psutil.Process(pid=32544, name='chrome', status='sleeping', started='04:10:27') START INFO: 23273472
children pid -psutil.Process(pid=32348, name='nacl_helper', status='sleeping', started='04:10:11') START INFO: 1679360
children pid -psutil.Process(pid=32423, name='chrome', status='sleeping', started='04:10:18') START INFO: 102768640
children pid -psutil.Process(pid=32535, name='chrome', status='sleeping', started='04:10:26') START INFO: 27021312
webdriver -17562 used memory: 6948.0 KB
Children -17570-17766 on the list used memory: 476640.0 KB
Total parent + children used memory: 483588.0 KB
Here is what Chrome Task Manager2 shows for the second try.
| Task | Memory footprint | Process ID |
|---|---|---|
| Browser | 25,596 K | 32332 |
| GPU Process | 59,604 K | 32423 |
| Utility: Network Service | 20,872 k | 32427 |
| Utility: Storage Service | 10,760 k | 32436 |
| Spare Renderer | 13,068 k | 32544 |
| Tab: Google | 28,456 k | 32453 |
However, no matter how I add or subtract between two result sets or within a single set, I still cannot get a value close to any value provided by Chrome Task Manager.
I also tried to read the file in /proc/meminfo before and after calling driver.get("get"), and then subtract two values to get the memory value used by Chrome.
#total - available in MB
print(int(open('/proc/meminfo').readlines()[0].split(' ')[-2])-int(open('/proc/meminfo').readlines()[2].split(' ')[-2]))
I got 626376 and 657304 for the first one and 659256 and 680864 for the second one. Results (657304-626376) = 30,928 KB and (680864-659256) = 21,608 KB are same as subtraction results (psutil.virtual_memory().total - psutil.virtual_memory().available) / 1024 called before and after driver.get ( "URL"). But they are all smaller then the sum of values that Chrome Task Manger gives.
Could anyone please help me to get memory usage values (Browser + Spare Renderer + Tab:url) of that single line of code (directing to new URL driver.get("url")) using any methods or packages? Referencing the values Chrome Task Manager 1 and 2 provides.
