0

So, I've successfully used Vagrant and Chef to install an MSI onto a Windows guest VM. Woot! Then, I wrote a few tests with ServerSpec that check that after the MSI was installed, files got put into their proper places and things like that.

My question is: What sort of goals should I have when it comes to acceptance testing an install of a package like that? Should I be checking for specific files? But what if there are lots of files? Or is checking for each file too low-level? Too close to the implementation details? What level should I be at when writing acceptance tests for a successful package install?

NickRamirez
  • 340
  • 4
  • 10

2 Answers2

2

I use test driven development, so I generally have one test per resource in my recipe. I write the tests first. So if my basic design is:

  1. I need to install xxx
  2. I need to fix the config file for xxx
  3. I need to start the xxx service

Then I'd write three tests in my serverspec.

  1. check with package to ensure the install happend
  2. check with file to ensure the config file is present, with the correct mode, owner, group, and content
  3. check with service that the service is running.

Then I'd write my recipe using the package, file or template, and service resources.

Tejay Cardon
  • 4,193
  • 2
  • 16
  • 31
0

Personally, I never subscribed to this level of testing. Windows Installer has been around 15+ years and is very predictable. It's already a declarative language and if you start writing tests to validate the result you basically are maintaining two different sets of data. Maybe I'm wrong, decide for yourself.

Personally I'd write tests to verify that 1 entry is in Add Remove Programs and that it's the version you expect. You could then maybe go a little farther to validate the uninstall is "clean". You may have to list exceptions for known user data. The only other test I can think of is that install new and install old, upgrade new get you to the same file set.

Then I'd write tests to verify the application functionality itself. If the application works, the install is good. If it fails, root cause, fix app or installer and rinse and repeat.

Christopher Painter
  • 54,556
  • 6
  • 63
  • 100
  • One benefit that I can think of making tests is making it repeatabe, making it possible to test that the install still works after the application changes. And that it works on various OS versions. Good idea with testing uninstall and upgrade. – NickRamirez Feb 11 '15 at 02:43