How to test modified chef recipe on single node?

Question

I have a chef server managing a handful of nodes. The server has recipes and data bags with various secrets. I'd like to make changes to one of the recipes and test it on a single node only: I do not want other nodes to update and use the modified recipe.

Solutions I have explored:

Test Kitchen, etc.

Tests are nice, but there are some details I cannot test without the data bags and other things unique to the chef server.

rsync code into /var/chef/cache/cookbooks

And run chef-client --skip-cookbook-sync

This fails because my cookbook source code lacks a metadata.json and various other things.

bump the cookbook version, upload, edit node's run list

This is difficult to coordinate concurrent work among multiple developers (particularly if testing lasts days or weeks). There are also several environments outside my control, and I can't really assume they're all locked to a particular version of the cookbook I wish to modify.

rename cookbook, upload to server, apply alternate role to node

This is the solution I'm using for now. It's tedious because changing the cookbook name changes the names of any resources it defines, so I've got to edit the cookbook quite a bit. Maintaining the alternate role set to use the renamed cookbook is also an annoyance.

score 2 · Accepted Answer · answered Mar 24 '16 at 17:22

I've handled that by setting the cookbook version to something like 0.0.1 and locking just the target environment to that version -- this avoids causing everyone with an unlocked environment to adopt the new cookbook.

If you need it to be truly a specific node, you could create a dummy cookbook that 'depends' on your specific 0.0.1 cookbook and assign that recipe as an additional runlist entry to a node. As long as the environment's pin allows for older cookbooks (ie <= x.y.z instead of = x.y.z) then it'll force that one node to use the special cookbook.

Also, look at test-kitchen + chef-zero to test server features inside the kitchen.

Uploading a cookbook at a low version such as 0.0.1 is probably the beast short-term solution. The long term solution is to refactor the cookbooks into more testable pieces and get tests in place as suggested by @StevenKing. — Patrick, Apr 06 '16 at 18:19

score 1 · Answer 2 · answered Mar 25 '16 at 22:47

If your tests are really lasting for weeks, you might think about applying successful ideas from the software engineering world, like Feature Toggles. This way, you could switch between the old/new code using such toggles implemented through role attributes or tags in Chef.

As we are, however, we also flip the environment of particular nodes from time to time and restrict production to version latest - 1, I currently experiment with the following that eases the use of multiple environments with the same settings (no guarantee that it really works well, I haven't tried that in production, yet):

# environments/production.rb

name "production"

env_cookbook_versions = {
  "mysql"               => "<= 6.0.18",
  "extra-constrained"   => "= 1.1.2"
}

env_default_attributes = {}

eval File.read("production_settings.rb")

This file (that I upload through knife environment from file) contains some specific constraints, that don't hold for another environment, like pre-production, which i.e. has no version pin for the extra-constrained cookbook.

Finally, the production_settings.rb (the parts common between all of these environments (for me, I'm only speaking of production and pre-production, not many more)) can look like this:

# environments/production_common.rb

env_cookbook_versions["foo"] = "= 0.0.1"
cookbook_versions(env_cookbook_versions)

env_default_attributes.merge!({
  "rabbitmq" => {
    "server" => "mq.example.org"
  },
  "base" => {
    "flags" => {
      "production" => true
    }
  }
})
default_attributes(env_default_attributes)

As you know, you then have to be careful, when searching for other nodes in the same environment. Further, I'm not entirely sure, if all of this practically works good. You have to ensure that every environment depending in production_settings.rb is uploaded, once this is changed. But maybe this gives you some ideas.

Btw. regarding testability without Chef-Server: I seldomly encounter things that I can't test with test-kitchen. You know that you can also add data bags and also other fake nodes to your cookbook to test code that uses such search queries?

Interesting idea about feature flags. Will keep that one in mind. — Patrick, Apr 06 '16 at 18:19

How to test modified chef recipe on single node?

2 Answers2