7

I'm working on building a docker image to be able to run all of our Perl applications. The applications require hundreds of CPAN modules to be installed. The full build of the docker image takes about an hour to complete.

After doing the initial image, I'm not sure how best to handle ongoing updates.

  1. We could keep a single Dockerfile in git, and then modify this as required, and push new builds up to dockerhub. However if the person doing the build doesn't have all of the intermediate images, then adding a single CPAN module could be an extremely tedious process, and it might take an hour before they even know if the new module installs correctly. Also it would be downloading every CPAN module again, which seems a bit risky, as there might be a breaking change in the new module.

  2. Alternatively, the person doing the build could pull the latest docker-hub image, and then install the cpan module interactively, commit the build and push the new image to dockerhub. However then we only have our dockerhub images, but not master Dockerfile.

  3. Or another option would be to create a Dockerfile for each new build, which references the previous dockerhub image. This seems overly complicated though.

Option 1) seems wrong. I'm fairly sure we don't want to be rebuilding the entire image from the base OS just to install one additional module. However being dependent on images without Dockerfiles seems risky as well.

Sinan Ünür
  • 116,958
  • 15
  • 196
  • 339
user1751825
  • 4,029
  • 1
  • 28
  • 58
  • Option 3 or some variant would seem the most "Docker-appropriate" solution. If you follow an organized tagging strategy, users "upgrading" to the most recent version would be able to utilize the previous image layers and only need to build the most recent. You may want to create a "base" version with a different name to avoid having to do a full build in case you ever need to a full re-build. – ldg Jul 30 '16 at 19:19
  • I found this (old) question because I'm investigating what strategy to use for a similar problem. You've probably solved this already, but I just wanted to chime in with a few points: 1. Sounds like you are using the OS's perl and package manager. Most OS's will only update perl module packages if there are security implications, so for updates I'd probably indeed rebuild the base image. Same for new additional packages (once you want them in actual production at least). Also consider locking what versions you want in your application, maybe with the help of Carton. No solution, just tips :-) – Gilimanjaro Jul 27 '18 at 07:35

1 Answers1

5

You could use the standard module installer for your underlying OS on your docker image.

For example, if its RedHat then use yum and only use CPAN when they are not available 

FROM centos:centos7
  RUN  yum -y install cpanm gcc perl perl-App-cpanminus perl-Config-Tiny &&  yum clean all
  RUN cpanm install Some::Module; rm -fr root/.cpanm; exit 0

taken from here and modified

I would try to have a base image which the actual applications use

I would also avoid doing things interactively (e.g. script a dockerfile) as you want to be able to repeat the build when upstream dependencies change, which docker hub does for you.

EDIT You can convert perl modules into your own packages using dh-make-perl

You can load these into your own Ubuntu repo using reprepro or a paid solution of Artifactory

These can then be installed using apt-get when you use your repo as a source from within a dockerfile.

When I have tried a similar thing before There are a few problems

  • Your apps don't work with the latest version of modules
  • There are far more dependencies than you expected
  • Some modules wont package

Benefits are

  • You keep the build tools (gcc, etc) off the app servers
  • You know much more about your dependencies
KeepCalmAndCarryOn
  • 8,817
  • 2
  • 32
  • 47
  • Yes, I've been using official packages when I can find them. I'm most familiar with Ubuntu, so have been using apt-get. I have been wondering though if it might be safer to use an official perl docker base, with a specific perl version that is known to work with our applications. The trouble with this is that it seems to ignore the apt-get packaged libraries. – user1751825 Jul 31 '16 at 00:24
  • Unfortunately there are quite a few modules we need which don't seen to be in the official os packages. – user1751825 Jul 31 '16 at 00:28
  • Another option I've seen suggested is using something like local::lib, and then installing all required modules on a build instance running the same os and perl, and then manually packaging and deploying the whole lib folder. Would this offer any benefits over creating individual packages from modules? – user1751825 Jul 31 '16 at 09:29
  • You could use a docker volume where all the modules are installed to which would be shared across all images on a host. I think it would work but you would be bound to that OS and version – KeepCalmAndCarryOn Jul 31 '16 at 09:58