Webpack4 thread-loader bring little optimization for babel-loader. Is something wrong, or why?

Question

As mentioned in Webpack docs and many information that introducing how to improve webpack build speed, thread-loader can obviously improve the build speed of babel-loader.

But when I try it in my project, it seems like that thread-loader does not bring any optimization for babel-loader.

I use speed-measure-webpack-plugin to measure build speed, and there are the code and result of my test.

my device: Macbook Pro (15-inch, 2018, 6-core, 12-thread)

version information:

{
  "webpack": "^4.41.2",
  "thread-loader": "^3.0.1",
  "babel-loader": "^8.2.2",
  "webpack-preprocessor-loader": "^1.1.3",
}

Before using thread-loader for babel-loader, code of webpack.config.js:

// thread-loader rule:

{
  test: /\.(jsx?)$/,
  exclude: [
    /node_modules/,
  ],
  use: [
    {
      loader: 'babel-loader',
      options: {
        babelrc: true,
        cacheDirectory: true,
        rootMode: 'upward',
      }
    }, {
      loader: 'webpack-preprocessor-loader',
      options: {
        params: env
      }
    }]
}

Time cost:

 SMP  ⏱  Loaders
thread-loader, and
babel-loader, and
webpack-preprocessor-loader took 15.71 secs
  module count = 315

After:

// thread-loader warmup:
const threadLoader = require('thread-loader');
const threadLoaderOptions = {
  poolTimeout: 2000,
  name: 'thread-loader-for-babel'
}

threadLoader.warmup(threadLoaderOptions, [
  'webpack-preprocessor-loader',
  'babel-loader'
]);

// thread-loader rule:
{
  test: /\.(jsx?)$/,
  exclude: [
    /node_modules/,
  ],
  use: [
    {
      loader: 'thread-loader',
      options: threadLoaderOptions
    },
    {
      loader: 'babel-loader',
      options: {
        babelrc: true,
        cacheDirectory: true,
        rootMode: 'upward',
      }
    }, {
      loader: 'webpack-preprocessor-loader',
      options: {
        params: env
      }
    }]
}

Time cost:

 SMP  ⏱  Loaders
thread-loader, and
babel-loader, and
webpack-preprocessor-loader took 14.64 secs
  module count = 313

15.71s VS 14.64s, this effect is not obvious.

thread-loader change babel-loader work from serial execution to parallel execution, theoretically in my device with 6 cores and 12 threads, even if you factor in the worker startup time and other factors, there will be at least several times improvement, but it doesn't.

Is the thread-loader used correctly? (It should be correct, I have tried to modify the source code of thread-loader and log to ensure that multiple workers are running) If it is used correctly, what is the reason for the effect not obvious?

I have a guess. For modules that hit the same loader, webpack 4 has built-in multi-thread processing, that is, webpack has already done thread-loader work. So thread-loader has little effect. But this guess needs to go deep into the webpack 4 source code to verify.

(I will continue to study, but it will take some time. If there are anyone that have been studied, please share it)

thanks in advance. :)

score 0 · Answer 1 · answered Jul 13 '21 at 07:11

Possible reasons as to why it's not as fast, is these reasons (found on the documentation):

Loaders running in a worker pool are limited. Examples:

Loaders cannot emit files.
Loaders cannot use custom loader API (i. e. by plugins).
Loaders cannot access the webpack options.

Webpack4 thread-loader bring little optimization for babel-loader. Is something wrong, or why?

1 Answers1