0

We have recently started implementing a proper CI-based build pipeline for continuous deployment scenario and decided to use Chef as our CM tool. We also use Vagrant for creating isolated local dev environments for our devs, since it allows a simple 'vagrant up' and the env just runs. Vagrant is supposed to support Chef Zero, which it does. However, that requires holding predownloaded cookbooks in a 'cookbooks' dir, meaning managing dependencies and their versions would be a huge pain without a "packet manager".

What I would like to do is to use either a Policyfile or a Berksfile to manage the cookbook dependencies. Unfortunatelly, this does not work (or I am doing something wrong). First of all, I could not find a way to implement the Policyfile using Vagrant, even though the feature was released 2 years ago. The only thing that popped up was this. The Berkshelf way was more promising, because there is a 'vagrant-berkshelf' plugin for Vagrant, which does exactly what I want...but with a few hiccups. One of the issues is the dependency on ChefDK. The main one, however, is that it's been sort of deprecated in favour of the TestKitchen.

So, TestKitchen, there's been a lot of fuss about this neat tool. It supports both Policyfiles and Berkshelf, it has a Vagrant provider that runs the VM using Vagrant. So what is wrong? To my understanding Chef's TestKitchen was intended for Cookbook developers, because it allows automated local Cookbook testing, etc. But what about other devs? For example if a dev only works on the project (app) itself, and not the infrastructure, why would the dev even need ChefDK? All the dev wants to do is download the source code for the app (web app in our case), which includes Vagrantfile in the root dir, then simply type 'vagrant up' (because VirtualBox and Vagrant are already installed on the machine) and start working!

Yes, it is possible to use the TestKitchen as a Vagrant wrapper (this guy says it's fine). Instead of the 'vagrant up', the dev would type 'kitchen converge' and then start working. But that implies the dev already installed ChefDK, alongside the VirtualBox and Vagrant.

Is there any [good and modern] way of not using the TestKitchen as a development environment provider? Or is it not considered a bad practice? To me, it seems wierd that the dev has to install ChefDK if he wants to spin the dev env on his laptop. How do you have your dev environment set up?

simon
  • 550
  • 1
  • 5
  • 14
  • Why would it be less good to install chefdk than vagrant and a bunch of plugins to support chef as vagrant provisionner ? This question is probably highly opinion based and so not a good fit for StackOverflow. – Tensibai Jan 12 '17 at 15:58
  • Tensibai well..all I can now is just hope it won't get downvoted =) Thanks for your input though! – simon Jan 12 '17 at 16:03
  • Well, the question is clear for me, would probably be useful on another media (maybe on the mailing list https://discourse.chef.io ). so there's no reason to downvote it, I did vote to close it as opinion based, but I don't think it worth a downvote. – Tensibai Jan 12 '17 at 16:06
  • I think you do not need the whole chefDK, just install test-kitchen as rubygem on its own. – Draco Ater Jan 13 '17 at 19:56

2 Answers2

1

We use Jankins to update cookbooks on the chef-server. Also, we use Berkshelf to control cookbook dependencies.

Vagrant can provision machine using chef-client. It connects to the chef-server gets the latest cookbooks/environments/roles and provisions the machine.

It works, but it takes a lot of time to provision vagrant-machine. So it works for now but we are looking for some better options.

chef-zero can be used for debugging (but it is better to use test kitchen) and I think it is not suitable for the workflow you want.

Also, you can check these vagrant plugins, they're really helpful: vagrant-omnibus, vagrant-cachier, vagrant-vbguest, vagrant-hostsupdater, vagrant-notify

Niv-Mizzet
  • 361
  • 3
  • 6
  • Thanks, this is good. However, if the cookbooks are stored on the Chef Server and local Vagrant instances are provisioned via chef-client that pulls latest cookbooks from the server, the enrvironment is now tied to the network connection. So, say if I'd like to work on the proj on the airplane during the flight, the workflow breaks, because I don't have the cookbooks... – simon Jan 18 '17 at 10:24
  • @lime You need the Internet to provision the machine. After that, you can work offline: just do `vagrant up` in the airplane. Even if you provision machine with `chef-zero` you might need internet connections for some provisioning needs like: install software, clone repositories, download dump of the DB, etc. – Niv-Mizzet Jan 18 '17 at 10:30
  • One person has suggested putting Vagrantfile, along with other stuff (basically, a 'chef-repo') into a separete repo and use CI to update cookbooks there, as well as on the chef server. This way, when cloning the repo it would be possible to provision Vagrant VM from local cookbooks directory. One question remains though, at which stage should the CI update the cookbooks? Every time they are _stable_? Or even when the build is failing it should still update the above repo with new versions? Create a issue-based branch on this chef-repo and keep that specific branch updated? – simon Jan 18 '17 at 10:32
  • But what happens if the work on the airplane requires changing the cookbook and reprovisioning the machine again? Just modify the VM manually, finish the feature and when back in the office fix the cookbook accordingly? – simon Jan 18 '17 at 10:33
  • You should push only working and tested cookbooks to the chef-server otherwise it can cause problems to all users who work with them. For example, you sync your master branch with chef-server then you can run some auto-tests before merging you code to master: cookstyle, foodcritic, unit tests, etc. – Niv-Mizzet Jan 18 '17 at 10:37
  • Our cookbooks contain all needed configurations so they include installations of different software, which needs the Internet connection. So I'm sure I can not run them without a fast connection. If you need some urgent change in the environment you can stop chef-client and do that directly in the provisioned machine. After that you can describe it in the cookbooks and provision machine again – Niv-Mizzet Jan 18 '17 at 10:39
  • Ok, makes sense, but what happens if I need a cookbook of a specific version to develop my feature? Vagrant only has these 'chef.add_recipe', which, to my understanding, are not version specific and just take the latest? – simon Jan 18 '17 at 10:52
  • I've never had such case. When we came to a stable environment it almost does not change during the feature development. Vagrant and chef-server do not have any versioning features. It should be done separately. You can provision one more machine to test your cookbook. But I don't think it is needed to be versioned. – Niv-Mizzet Jan 18 '17 at 12:01
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/133451/discussion-between-niv-mizzet-and-lime). – Niv-Mizzet Jan 18 '17 at 13:19
0

The policyfile workflow was built expecting the use of Chef Server and chef-client. As such you would generally use it via the chef_client provisioner in Vagrant. This is probably not going to be the workflow you want though. It would be possible to build something which uses the policy export system and some fancy file trickery to make a Berks-like Vagrant plugin but you would have to do that yourself.

Most of the Chef world has stopped using Vagrant for setting up development environments in favor of cloud systems as making something "like production" in a VBox VM is generally not a possible as you might think.

coderanger
  • 52,400
  • 4
  • 52
  • 75