Parallelizing Cucumber BDD test on multiple machines using Gems or Jenkins (or both)

Question

I’m running a project using BDD with Cucumber (with page_object gem), Watir and Jenkins. Right now we’re looking the best way to parallelize test to reduce testing time on multiple virtual machines, with different navigators and so on.

I think there is two approaches:

to use a Ruby tool like parallel_test, Hydra, TestJour, TestBot...
or organize everything through Jenkins, using paths, tags, etc.
to use both in some way

In order to put effort on the right way… do you consider Jenkins is a good option to make parallel test on multiple machines or should I give a chance to another tool? Suggestions are welcome :)

we had some good discussions on this at the Test Automation Bazaar during the open-space portion. This might also be a good question (since it's a more general discussion of potential approaches etc) to ask in both the Watir and Cucumber groups on googlegroups — Chuck van der Linden, May 30 '12 at 06:26

score 8 · Accepted Answer · answered May 29 '12 at 16:04

Jenkins is only a part of the solution. You can use parallel_tests to run multiple cukes concurrently (as long as your Jenkins server is not running on Windows) but you still need more.

You will need to setup a grid of servers to handle the watir tests. I strongly suggest you look at Selenium Grid 2 (http://code.google.com/p/selenium/wiki/Grid2). Setup nodes on your remote VMs and start the hub on your Jenkins server. Then parallel_tests can run the tests in parallel having them connect to the local hub which will in turn send them out to the remote servers for execution.

One note - In order for this to work well you need to have a good test data management strategy in place. Each tests needs to be able to run independently. In addition, they should each be responsible for setting up the necessary data for the test to run and then clean up that data after the test executes.

Cheezy - how to configure it so that the same test(s) run on multiple nodes simultaneously? Currently I have the setup you described above but the nodes run different tests, I want the nodes to run the exact same tests (to test different OS/broswer combos) — Farooq, Jul 17 '14 at 20:51
Check out Zalenium as an alternative to regular selenium grid if you want a live preview of your tests and the ability to record and playback videos of the runs. http://opensource.zalando.com/zalenium/ . Using parallel cucumber works well as well, but you may want to use the `--group-by scenarios` option to get better efficiency — David West, Jul 30 '18 at 12:32

score 1 · Answer 2 · answered May 29 '12 at 14:50

1

I use Jenkins to split jobs to various slave machines. When not requiring parallel tests, this also allows testing of different environments by different users (testers testing, developers troubleshooting, etc). Very easy to set up!

answered May 29 '12 at 14:50

adam reed

2,024
18
24

score 1 · Answer 3 · answered Apr 18 '13 at 21:52

You can parallelize by distributing your tests into different tags, then running a different tag set on each test machine.

require 'rubygems'
  require 'cucumber'

  require 'cucumber/rake/task'

  Cucumber::Rake::Task.new( :features) do |t|
    t.cucumber_opts = "features --tags @testset_1 "
end

Parallelizing Cucumber BDD test on multiple machines using Gems or Jenkins (or both)

3 Answers3