Puppet, test roles in control-repo with beaker-rspec

Question

I want to test my control repo (having profiles and roles in it) with rspec-puppet and beaker-rspec. I am still having some trouble with beaker-rspec:

• I would like to test the profiles individually, for example profile::postgresql and see if the database has been installed.
• Then I would also like to test my roles individually, e.g. role::fileserver or role::webserver.

I have set-up beaker-rspec and it works, but it takes a considerable amount of time because it will apply all the profiles one after another. I see the following problems in this:

• If too many profiles get applied to the Vagrant VM, it will get polluted (leftover packages, files etc. from another run, so that the test is not representative anymore)
• Some profiles do not have anything in common or might even contradict them (e.g. nginx vs. apache; they will never be in one role, but are both defined in the control-repo).

What I rather would do is:

• beaker-rspec should check which roles have been defined and then spin up an individual virtual machine for each of the roles.
• These should be done either serially or (better) in parallel.
• How can I reduce the time for each run, especially if having a rather large Puppet setup with multiple roles and different OS versions etc. Maybe select the type of tests (“only role tests” or “only profile tests”).

It would also be ok if beaker-rspec tested every profile in a new VM (but then I would like to switch of the tests for other usage).

How can I accomplish this? Thanks :)

Answer 1

A few different responses here:

beaker-rspec should check which roles have been defined and then spin up an individual virtual machine for each of the roles. These should be done either serially or (better) in parallel.

Beaker-rspec doesnt have the ability to use a completely new SUT (system under test) per test suite. The only way to do this right now is create a new _spec.rb file per test, then run each test in a new process eg. bundle exec rspec spec/acceptance/profile_test_one.rb && bundle exec rspec spec/acceptance/profile_test_two.rb

For running in serial or parallel, it's probably best to do that within your CI platform. So for Travis (or Jenkins with the Matrix plugin) for example, you'd create a test matrix with different tests per profile, so they can be run in parallel. This requires significantly powerful testing machine, especially if you're using chunky VMs.

How can I reduce the time for each run, especially if having a rather large Puppet setup with multiple roles and different OS versions etc. Maybe select the type of tests (“only role tests” or “only profile tests”).

When you say reduce time, which part to you mean? How long the Puppet run takes, or how long to get the machine to a testable state?

Using Docker instances significantly speeds up the machine provisioning state.

To speed up the Puppet run, it depends on what's slow. If it's computation, make the SUT bigger, give it more cores or RAM. For network dependant issues, you can use caching. For example on Mac, you can use Squid to cache all the required RPM/APT packages

I usually don't do acceptance testing for roles. Roles are basically just meta-profiles: they should not have logic and are just a collection of profiles.

The only benefit of testing the role would be detection of conflicts when using multiple profiles, but conflicts and dependency checks can be better detected using rspec-puppet rather than an acceptance test.

So for example, a role like:

class role::foo_blog {
  include profile::base
  include profile::nginx
  include profile::php
  include profile::mysql
}

It would be easier to just test each of the four profiles with acceptance tests, and write an rspec-puppet test to make sure the different profiles don't have dependency issues or conflicts.