Webpack DllPlugin with gulp: Cannot find module '... vendor-manifest.json'

Question

I have a rather large React application built with webpack 2. The application is embedded into a Drupal site as a SPA within the existing site. The Drupal site has a complex gulp build setup and I can't replicate it with webpack, so I decided to keep it.

I have split my React application into multiple parts using the DllPlugin / DllReferencePlugin combo which is shipped out of the box in webpack 2. This works great, and I get a nice vendor-bundle when building with webpack.

The problem is when I try to run my webpack configuration in gulp, I get an error. I might be doing it wrong, as I have not been able to find much documentation on this approach, but nevertheless, it's not working for me.

It looks like it's trying to include the the manifest file from my vendor-bundle before creating it.

Whenever I run one of my defined gulp tasks, like gulp react-vendor I get an error, saying that it cannot resolve the vendor-manifest.json file.

If I on other hand run webpack --config=webpack.dll.js in my terminal, webpack compiles just fine and with no errors.

I have included what I think is the relevant files. Any help on this is appreciated.

webpack.config.js

// Use node.js built-in path module to avoid path issues across platforms.
const path = require('path');
const webpack = require('webpack');
// Set environment variable.
const production = process.env.NODE_ENV === "production";

const appSource = path.join(__dirname, 'react/src/');
const buildPath = path.join(__dirname, 'react/build/');

const ReactConfig = {
  entry: [
    './react/src/index.jsx'
  ],

  output: {
    path: buildPath,
    publicPath: buildPath,
    filename: 'app.js'
  },

  module: {
    rules: [
      {
        exclude: /(node_modules)/,
        use: {
          loader: "babel-loader?cacheDirectory=true",
          options: {
            presets: ["react", "es2015", "stage-0"]
          },
        },
      },
    ],
  },

  resolve: {
    modules: [
      path.join(__dirname, 'node_modules'),
      './react/src/'
    ],
    extensions: ['.js', '.jsx', '.es6'],
  },

  context: __dirname,
  devServer: {
    historyApiFallback: true,
    contentBase: appSource
  },
  // TODO: Split plugins based on prod and dev builds.
  plugins: [

    new webpack.DllReferencePlugin({
      context: path.join(__dirname, "react", "src"),
      manifest: require(path.join(__dirname, "react", "vendors", "vendor-manifest.json"))
    }),

    new webpack.optimize.CommonsChunkPlugin({
      name: 'manifest',
      filename: 'webpack-loader.js'
    }),
  ]
};
// Add environment specific configuration.
if (production) {
  ReactConfig.plugins.push(
    new webpack.optimize.UglifyJsPlugin()
  );
}

module.exports = [ReactConfig];

webpack.dll.js

const path = require("path");
const webpack = require("webpack");
const production = process.env.NODE_ENV === "production";

const DllConfig = {
  entry: {
    vendor: [path.join(__dirname, "react", "vendors", "vendors.js")]
  },
  output: {
    path: path.join(__dirname, "react", "vendors"),
    filename: "dll.[name].js",
    library: "[name]"
  },
  plugins: [
    new webpack.DllPlugin({
      path: path.join(__dirname, "react", "vendors", "[name]-manifest.json"),
      name: "[name]",
      context: path.resolve(__dirname, "react", "src")
    }),
    // Resolve warning message related to the 'fetch' node_module.
    new webpack.IgnorePlugin(/\/iconv-loader$/),
  ],
  resolve: {
    modules: [
      path.join(__dirname, 'node_modules'),
    ],
    extensions: ['.js', '.jsx', '.es6'],
  },
  // Added to resolve a dependency issue in this build #https://github.com/hapijs/joi/issues/665
  node: {
    net: 'empty',
    tls: 'empty',
    dns: 'empty'
  }
};

if (production) {
  DllConfig.plugins.push(
    new webpack.optimize.UglifyJsPlugin()
  );
}

module.exports = [DllConfig];

vendors.js (to determine what to add to the Dll)

require("react");
require("react-bootstrap");
require("react-dom");
require("react-redux");
require("react-router-dom");
require("redux");
require("redux-form");
require("redux-promise");
require("redux-thunk");
require("classnames");
require("whatwg-fetch");
require("fetch");
require("prop-types");
require("url");
require("validator");

gulpfile.js

'use strict';

const gulp = require('gulp');
const webpack = require ('webpack');
const reactConfig = require('./webpack.config.js');
const vendorConfig = require('./webpack.dll.js');

// React webpack source build.
gulp.task('react-src', function (callback) {
  webpack(reactConfig, function (err, stats) {
    callback();
  })
});

// React webpack vendor build.
gulp.task('react-vendor', function (callback) {
  webpack(vendorConfig, function (err, stats) {
    callback();
  })
});

// Full webpack react build.
gulp.task('react-full', ['react-vendor', 'react-src']);

NOTE: If I build my vendor-bundle with the terminal with webpack --config=webpack.dll.js first and it creates the vendor-manifest.json file, I can then subsequently successfully run my gulp tasks with no issues.

This is not very helpful though, as this still will not allow me to use webpack with gulp, as I intend to clean the build before new builds run.

Answer 1

I ended up using the solution mentioned in the end of my question. I build my DLL file first and then I can successfully run my gulp webpack tasks.

One change that can make it easier to debug the issue, is to use the Gulp utility module (gulp-util) to show any webpack errors that might show up during build of webpack, using gulp.

My final gulp setup ended up looking like this:

gulpfile.js

'use strict';

const gulp = require('gulp');
const gutil = require('gulp-util');
const webpack = require('webpack');
const reactConfig = require('./webpack.config.js');
const vendorConfig = require('./webpack.dll.js');

// React webpack source build.
gulp.task('react', function (callback) {
  webpack(reactConfig, function (err, stats) {
    if (err) {
      throw new gutil.PluginError('webpack', err);
    }
    else {
      gutil.log('[webpack]', stats.toString());
    }
    callback();
  });
});

// React webpack vendor build.
gulp.task('react-vendor', function (callback) {
  webpack(vendorConfig, function (err, stats) {
    if (err) {
      throw new gutil.PluginError('webpack', err);
    }
    else {
      gutil.log('[webpack]', stats.toString());
    }
    callback();
  });
});

// React: Rebuilds both source and vendor in the right order.
gulp.task('react-full', ['react-vendor'], function () {
  gulp.start('react');
});

I hope this might help someone in a similar situation.

Answer 2

Whenever I run one of my defined gulp tasks, like gulp react-vendor I get an error, saying that it cannot resolve the vendor-manifest.json file.

Your gulpfile.js contains this:

const reactConfig = require('./webpack.config.js');
const vendorConfig = require('./webpack.dll.js');

And webpack.config.js contains this:

new webpack.DllReferencePlugin({
  context: path.join(__dirname, "react", "src"),
  manifest: require(path.join(__dirname, "react", "vendors", "vendor-manifest.json"))
}),

The require() calls are currently all executed immediately. Whenever you run Gulp, it will evaluate both Webpack configuration files. As currently configured, Node runs the code in webpack.config.js at startup, and from there it sees the require() used in your creation of the DllReferencePlugin, so it will also try to read manifest.json at that time and turn it into an object...which is before it has been built.

You can solve this in one of two ways:

1. The DllReferencePlugin's manifest option supports either an object (which is what you are currently providing), or else a string containing the path of the manifest file. In other words, it should work if you remove the require() from around your path.join(...) call.
2. Alternatively, you can also defer the loading of the Webpack config files. Moving your const reactConfig = require('./webpack.config.js'); from the top of the file directly into the gulp task function should be sufficient, assuming that this function is not invoked until after the manifest has been built.