0

I am creating an image in wsl with scripting starting from a base Amazon Linux2 import. The following is some of the scripting. The {{THIRDPARTYCACHE}} label is replaced with a path to persistent storage, persistent in that it survives through distribution unregistration (image destroy) :

    sudo yum -y --setopt=cachedir={{THIRDPARTYCACHE}}/yum/cache install \
dejavu-sans-fonts.noarch \
freetds \
mailx \
moreutils \
nmon \
perl-Archive-Zip \
perl-Class-Accessor \
perl-DateTime \
perl-DateTime-Format-Strptime \
perl-DBD-MySQL \
perl-DBI \
perl-Digest-SHA \
perl-GD \
perl-JSON \
perl-libwww-perl \
perl-Log-Log4perl \
perl-LWP-Protocol-https \
perl-parent \
perl-XML-Parser \
pigz \
rng-tools \
ruby \
smem \
sysstat \
tree \
nmap-ncat \
git \
git-lfs \
xorg-x11-server-Xvfb \
wget \
openssl11 \
"perl(Data::Dump)" \
libXtst \
which \
perl-Coro \
perl-AnyEvent-AIO \
perl-PadWalker \
perl-App-cpanminus \
net-tools \
nss-tools 

sudo yum -y --setopt=cachedir={{THIRDPARTYCACHE}}/yum/cache group install "development tools"
sudo cpanm --no-test Perl::LanguageServer

The last two lines take a significant amount of time and successfully achieve the desired result of the installing the perl module Perl::LanguageServer. The final message from cpanm is: Successfully installed Perl-LanguageServer-v2.5.0 48 distributions installed

I'm looking for a way that I could cache the Perl::LanguageServer (along with all it's dependencies) allowing the ability to potentially skip the development tools group install and the compile/build of the module and all it's dependencies.

brian d foy
  • 129,424
  • 31
  • 207
  • 592
mikedoy
  • 117
  • 1
  • 6
  • Linux distros make RPMs from the installed modules. – ikegami Mar 30 '23 at 17:46
  • Everything to install is in the `blib` dir created by `make`/`Build make`. If you get the whole installation work directory including `blib`, all you'd have to do is `make install`/`Build install`. – ikegami Mar 30 '23 at 17:50
  • Thank you for responding. I'm not sure I understand. An rpm would be great, how do I make this after running cpanm? – mikedoy Mar 30 '23 at 17:52
  • 1
    New questions aren't for the comments. Did you even do any research on it? (And the original question you asked is off-topic.) – ikegami Mar 30 '23 at 17:54
  • Thanks for the response, Yes I did research and didn't find anything, this is why I asked the question. blib dir where? under ~./cpanm? I see nothing when running find ~/.cpanm -name blib – mikedoy Mar 30 '23 at 17:57
  • Sorry I was incorrect user when searching. Yes I see the blib directories for each of the dependent packages. Thanks – mikedoy Mar 30 '23 at 18:04
  • 1
    If you didn't find anything about creating RPMs, you didn't look enough. – ikegami Mar 30 '23 at 18:23

1 Answers1

1

@ikegami's suggestion of making packages of the complete Perl module installation is a good one. That also lets you stabilize the installation since the major CPAN tools prefer to upgrade whenever possible;they install the latest version when asked and not told otherwise. If you can make packages, do that. In some separate directory, install a perl for your project, install all the modules in it, and wrap up that directory into your package.

Emil Perhinschi reminded my on Twitter that DhMakePerl exists. The Debian people have very good tools for making Perl packages. I know you say you are on Amazon Linux (RedHat or Fedora based I think), but other people might find those tools handy. I don't know much about packaging.

But, aside from deploying a package, you can speed up the installation with cpan or cpanm by not testing the modules. You'll want to know if they work, but you only need to do that once:

 % cpan -T ...
 % cpanm --notest ...

Using cpanm over cpan means you can skip downloading very large data file, but cpanm means more network connections.

If you want to host all of the modules yourself (say, in an MiniCPAN, you can cut down on the time to download modules.

And last, there's a balance between the number of third-party modules you want to use and the ease of your deployment. Different people have different tolerances for that. Consider if you really need some of the modules you use and if that causes too much pain in the sysadmin side. Using one module might come with tens of other distributions as dependencies (and this is especially true with Test::* modules). MetaCPAN shows dependencies. For example, if you want just the current year, do you really need DateTime?

brian d foy
  • 129,424
  • 31
  • 207
  • 592
  • I misunderstood @ikegami's response. Didn't consider packaging the entire Perl configuration as a module. Wondering if perl -e "print \"@INC\"" output is complete. Installing multiple modules using yum, but only one module (Perl::LanguageServer) doesn't have a distribution other than CPAN. – mikedoy Mar 30 '23 at 19:03
  • Re "*the entire Perl configuration*", I just meant the directory in which you run `perl Makefile.PL` or `perl Build.PL` and the rest of the process. You really only need the `blib` subdirectory, but having the whole dir makes you can just run `make install` or `Build install` on the other machine – ikegami Mar 30 '23 at 20:31
  • In that case, get someone to make an rpm for the missing module so you don't have to do the extra workaround. But, you can make rpms for particular modules, which is what @ikegami suggested. I suggested a different, more expansive approach, which sounds like you might not need. But, for what it's worth, this sort of context is useful in the question. – brian d foy Mar 30 '23 at 21:16
  • I've updated the question description in an attempt to make it clearer. I hope that helps. – mikedoy Mar 31 '23 at 11:57
  • Instead of *not* testing, just use HARNESS_OPTIONS to set the tests to run in parallel. I tend to use HARNESS_OPTIONS=j16 most of the time. Occasionally you will bump into a distribution that is not parallel safe, but when you encounter that just install it in series mode. HARNESS_OPTIONS works with anything that uses Test::Harness which is basically everything. – demerphq Apr 07 '23 at 12:04
  • The problem with testing is the installation of all of the modules you need to support testing. If people limited themselves to Test::More, that wouldn't be a big deal, but I often find it's a heavier burden to install a long chain of black magic test modules than to just skip it. – brian d foy Apr 08 '23 at 02:47