Installing additional files at install time with ExtUtils::MakeMaker/Dist::Zilla (dzil)

Question

tl;dr I want to ship a package.json with my Perl library, run yarn install (or npm install during the installation) and install the downloaded JavaScript dependencies with the Perl modules.

I have the following dist.ini:

name = Foobar
version = 1.2.3
license = Perl_5
copyright_holder = Yours Truly
copyright_year   = 2018

[@Filter]
-bundle = @Basic
-remove = GatherDir
[Git::GatherDir]

[Web::FileHeader]
header_filename = EMM-include.pm
file_match = ^Makefile\.PL$

The file EMM-include.pm contains a MY::postamble method:

package MY;

use strict;

use Cwd qw(abs_path);
use File::Spec;

sub postamble {
    my ($self) = @_;

    my $here = Cwd::abs_path();
    my $libdir = File::Spec->catdir($here, 'lib', 'Foobar');
    chdir $libdir or die;
    0 == system 'yarn', 'install' or die;
    chdir $here or die;

    return '';
}

The plug-in [Web::FileHeader] takes the file and patches it to the beginning of Makefile.PL.

Then there is a lib/Foobar/package.json:

{
  "name": "foobar",
  "version": "1.2.3",
  "main": "index.js",
  "dependencies": {
    "ajv": "^6.5.4"
  }
}

The MY::postamble section from EMM-include.pm invokes yarn install (replace it with npm install if you don't have yarn) and populate the directory lib/Foobar/node_modules with ajv and its dependencies.

Finally, there must be a module lib/Foobar.pm:

package Foobar;
# ABSTRACT: Just a test.
1;

That almost works as intended: The distribution can be created with dzil build. In the distribution directory, perl Makefile.PL invokes yarn install, the directory lib/Foobar/node_modules gets populated but the files in there are not installed with make install.

If I run perl Makefile.PL a second time, everything works, the JavaScript dependencies make it into blib/ and make install would install the JavaScript modules alongside the Perl modules.

Shipping the JavaScript dependencies with the distribution is not an option. They are already too many and they may have conflicting licenses (I am using GPLv3 here). Downloading the deps at runtime, after the installation will mostly fail because of missing privileges.

True, this has not that much to do with Dist::Zilla, it's rather a problem with ExtUtils::MakeMaker. But I am actually using Dist::Zilla here.

In case it matters, the real distribution is https://github.com/gflohr/qgoda and the last commit at the time of this writing is https://github.com/gflohr/qgoda/commit/3f34a3dfec8da665061432c3a8f7bd8eef28b95e.

My next try tomorrow would be to find all files that were installed by `yarn/npm` and then add them to the `Makefile` variable `TO_INST_PM`. Am I on the right track? Or is there a simpler approach? — Guido Flohr, Oct 31 '18 at 22:03

Grinnz · Accepted Answer · 2018-11-01T02:46:25.503

First, instead of using [Web::FileHeader] to alter your Makefile.PL, replace [MakeMaker] (used by @Basic) with [MakeMaker::Awesome], which allows you to modify Makefile.PL directly, and correctly enables dynamic_config since your distribution needs it. Also, don't give your include-file a .pm extension since it's not a perl module, and exclude it from being gathered into the resulting distribution so it doesn't accidentally get installed.

[@Filter]
-bundle = @Basic
-remove = GatherDir
-remove = MakeMaker
[Git::GatherDir]
exclude_filename = EMM-include
[MakeMaker::Awesome]
header_file = EMM-include

I strongly suggest using my @Starter bundle instead of the outdated @Basic, but if not at least add [MetaJSON] so you have modern metadata.

[@Starter::Git]
revision = 3
installer = MakeMaker::Awesome
Git::GatherDir.exclude_filename[] = EMM-include
MakeMaker::Awesome.header_file = EMM-include

Regarding what needs to be done at install time. First I will caution that requiring an internet connection to install is not something you can always rely on, nor is having yarn available of course. But the Alien series of modules for installing external libraries does this sort of thing often. Since you don't need to compile this code you probably don't need the whole Alien::Build/Alien::Base setup, but it may turn out to be an easier way to solve your problem than Makefile hacking described below. Basically you would first release an Alien distribution which installs your javascript library if necessary, and then this distribution could depend on that to load the library. If you decide to pursue this direction, check out Alien::Build, and the IRC channel #native on irc.perl.org.

The postamble section for ExtUtils::MakeMaker is not for running arbitrary code; it's for adding custom rules to the Makefile it generates; this is the way you need to influence the make process. I know very little about Makefiles so I can't help you further here, all I can suggest is to read all of the EUMM docs and note that the postamble is a function from MM_Any which you override to add your text, among other options from MM_Any and MM_Unix. You may be able to find people to help you in this direction on the IRC channel #toolchain on irc.perl.org.

I am aware that the installation requires an internet connection but that's like it is, nothing I can do about it if I need external files. — Guido Flohr, Nov 01 '18 at 06:03
I have done a quick check, your suggestions will work. You are, of course, right about postamble, I will move the code outside of the method. My example was just meant to reproduce the problem. I already have a `META.json` and `EMM-include.pm` is already excluded from the installation. Alien::Build doesn't seem to fit my use case but that's another issue. Thanks for an awesome answer! — Guido Flohr, Nov 01 '18 at 07:30

Installing additional files at install time with ExtUtils::MakeMaker/Dist::Zilla (dzil)

1 Answers1

Linked