5

I'm trying to get Puppet to upgrade our Varnish 3 servers to Varnish 4, a major update which requires an updated config file or it won't start. This is on Ubuntu 12.04.

The Varnish module is essentially built with these classes and dependencies:

Class['varnish::repo']
-> Class['varnish::install']
-> Class['varnish::tools']
-> Class['varnish::config']
~> Class['varnish::service']

I've updated the Apt-repo URL in varnish::repo, set ensure=>latest in varnish::install and provided an updated config file to varnish::config. So far so good.

When Puppet runs these dependencies, the run fails at the varnish::install stage because Apt tries to restart the Varnish daemon immediately after upgrading it, not giving Puppet the chance to replace the config file. The failure in the varnish::install class leads to a broken dependency chain and causes the remaining classes to fail as well. The result is a broken Varnish installation that needs manual recovery.

How do you deal with this?

I thought about using policy-rc.d, which essentially tells Apt not to perform automatic stops and starts of services. I tried creating the file before the upgrade and removing it afterwards.

file {'/usr/sbin/policy-rc.d':
    ensure  => $ensure,
    content => "#!/bin/sh\nexit 101",
    owner   => 'root',
    group   => 'root',
    mode    => '0755',
}

Of course creating and removing is a problem because Puppet sees this as a duplicate resource.

Why do I want to remove the policy again, after just installing it, you ask? Because we use unattended-upgrades to perform minor security-upgrades and I want to allow automatic service restarts in those cases, just not in this case. Furthermore, policy-rc.d affects all services, not just Varnish.

Maybe I'm thinking about this wrongly, but can I somehow tell Puppet or Apt to wait with the restart until the config file is replaced as well?

Martijn Heemels
  • 7,728
  • 7
  • 40
  • 64
  • 2
    This is one of the most annoying misfeatures of Ubuntu (and Debian). My own solution is to get rid of the offending distribution and use something more sane. – Michael Hampton Dec 11 '14 at 17:27
  • Does it restart the service if it's stopped/disabled prior to install? In a major upgrade like this, you may need to remove Varnish 3, then install and configure Varnish 4 rather than an in-place upgrade. That way, no offending Varnish 3 configs exist. – Aaron Copley Dec 11 '14 at 17:36
  • 1
    @AaronCopley Purging Varnish 3 before installing 4 would probably work, but is hard to do in Puppet in a single run, since both package versions have the same name 'varnish' and would conflict. Usually in Puppet you describe the desired state, instead of the steps to get there. That doesn't work in this case. – Martijn Heemels Dec 11 '14 at 17:49
  • I see your point. Your other option is to roll your own .deb package that doesn't try to start the service? – Aaron Copley Dec 11 '14 at 17:59
  • Did you solve this in the end, somehow? – gxx Sep 04 '16 at 13:01

1 Answers1

2

Why not replace the config file before installing the update? If it loads on restart, it won't use the "wrong" config file till after the update restarts it . . .

jmp242
  • 688
  • 3
  • 15
  • I think that will fail on nodes that do not yet have Varnish installed. In that case the config directories don't exist yet, so Puppet cannot manage files inside them. – Martijn Heemels Dec 11 '14 at 17:46
  • So add an onlyif that checks for if Varnish is already installed? – jmp242 Dec 12 '14 at 12:46
  • 1
    Manage the config directory as well, to be on the safe side. Also, build the manifest in a fashion that only applies these workarounds if an `upgrade => true` flag of any sort (class parameter?) is given. Once your whole infrastructure is safely on the `varnish 4` side, you can restore your manifest to a sane state without this kind of shenanigans. – Felix Frank Dec 15 '14 at 16:49