selenium and web automation test: how to run large volumn stress tests

Question

I am using selenium and firefox webdriver to test my website, it works well. The only problem is with the computing resource restrictions, I can only run 10 browsers simultaneously in one physical machine, which is not enough for our testing suite.

The big resource bottleneck is at firefox side, it will consume a lot of RAM and CPU when it's running. I am wondering if there is any technique to reduce the RAM and CPU usage so that in one machine I can run 100 firefox browsers in the same time. That will boost my efficiency a lot.

Any ideas?

score 3 · Answer 1 · answered Jul 24 '12 at 14:50

3

Assuming you want to test server load and are not relying on AJAX, you can use Apache Jmeter to bombard the server with random requests according to parameters you specify.

Because it's a headless browser that just requests some HTTP content and then throws it away, it can easily scale to 100 instances on a standard desktop.

answered Jul 24 '12 at 14:50

Matt Gibson

14,616
7
47
79

OK, still possible, but a bit more effort - you'll have to ask it to request AJAX urls in the same way that your page normally does, adding POST data or whatever to the request. – Matt Gibson Jul 24 '12 at 14:53
Again, this will stress the server, but won't tell you much about the real-world load time for the user which will involve the parsing overhead too. – Matt Gibson Jul 24 '12 at 14:54
it's not what I want, actually the ajax is very complex and we don't want to mess with it, thanks ;) – Bin Chen Jul 24 '12 at 14:54
Fair enough. In that case, have you looked at Selenium grid? SauceLabs allow you to run a lot of parallell selenium tests in the cloud but can be expensive. http://saucelabs.com/ – Matt Gibson Jul 24 '12 at 14:58

score 3 · Accepted Answer · answered Jul 24 '12 at 15:59

Selenium is not designed for performance testing, at all.

http://selenium-grid.seleniumhq.org/faq.html#would_you_recommend_using_selenium_grid_for_performanceload_testing

Selenium Grid can go so much to help you by ensuring the tests are done in parallel, but this is not what Selenium was created for, and the bottleneck of browser performance & RAM usage will be a problem with Selenium.

A better solution would be to use an application devoted to performance testing. I've used Redgate's solution as well as the performance testing solution integrated into Visual Studio 2010:

http://www.red-gate.com/products/dotnet-development/ants-performance-profiler/

selenium and web automation test: how to run large volumn stress tests

2 Answers2