Quickly install multiple puppet modules from directory

Question

Is there a way to install multiple puppet modules from a directory in one command?
I have a directory containing multiple puppet modules in tar.gz form downloaded from the forge, and I need to install all the modules. However puppet module install only takes an argument for one module. I initially used a bash loop to call puppet module install once for each module in the directory. However, I quickly discovered that it takes 1.6 to 2 seconds on my system to install each module due to the slow puppet start time (it appears to be due to ruby being slow to load all the gems). As I have added more modules to install, the time to install them all has become a problem.

Running puppet with no arguments or just puppet help takes 1.6 to 2 seconds; it also takes that long to run puppet module install when the module is already installed. This shows that the majority of the time to install the module is just start up time of puppet and not the time it takes to install the module. When I strace the puppet call there are thousands of stat and lstat calls resulting in ENOENT no such file or directory that I believe, based on my research, is ruby loading the gems some of which use shared libraries.

The hosts on which I am installing the modules are on an isolated network and do not have access to the forge, so installing directly from the forge is not an option.

I briefly looked into using bolt, but I could not find in the documentation a way to install modules from a local directory.

have you tried just untaring them to the module path `sudo puppet config print modulepath` — balder, Jan 14 '21 at 09:39
I am hoping to avoid manually discovering where to extract the modules since the puppet tool currently manages that logic. I am also hoping avoid having to manually extract them; installing a module it isn't as straight forward as just extracting the tar because the top directory in the tar is not named the same as when it is installed. For example, the directory in the puppetlabs stdlib module tar is `puppetlabs-apache-6.5.0` but needs to be named just `stdlib`. — bdrx, Jan 14 '21 at 15:25

score 0 · Answer 1 · answered Jan 19 '21 at 15:28

I wrote a simple ruby wrapper script around puppet module install as a work around for now. This is not an ideal solution since the script catches SystemExit and calls an "internal" test method and because I have to maintain this extra script. I was hoping to call a puppet provided command that provides the behavior of installing multiple modules, but this wrapper script should get me by for now.

#!/opt/puppetlabs/puppet/bin/ruby

begin
  require 'puppet/util/command_line'
  ARGV.each { |arg|
    begin
      Puppet::Util::Commandline.new('puppet', ['module', 'install', arg]).execute
    rescue SystemExit => e
      if e.success?
        $stderr.puts e.message
      else
        raise
      end
    end
    Puppet.settings.send(:clear_everything_for_tests) # needed after each puppet Commandline call to clear the global variables in order to call Commandline execute again
  }
rescue LoadError => e
  $stderr.puts e.message
  exit(1)
end

Quickly install multiple puppet modules from directory

1 Answers1