__webpack_require__ Module not found on the deployment server

Question

I am trying to use webpack to bundle my application through teamcity and deploy it on a remote server. I was able to achieve it as well, but on the final step of running the application using 'node ./dist/main' I am getting the below error.

PS C:\engage-analytics> node .\dist\main.js
node:internal/modules/cjs/loader:928
throw err;
^

Error: Cannot find module '@nestjs/core'
Require stack:
- C:\engage-analytics\dist\main.js
 at Function.Module._resolveFilename (node:internal/modules/cjs/loader:925:15)
 at Function.Module._load (node:internal/modules/cjs/loader:769:27)
 at Module.require (node:internal/modules/cjs/loader:997:19)
 at require (node:internal/modules/cjs/helpers:92:18)
 at Object.@nestjs/core (C:\engage-analytics\dist\main.js:181:18)
 at __webpack_require__ (C:\engage-analytics\dist\main.js:229:32)
 at fn (C:\engage-analytics\dist\main.js:331:21)
 at eval (webpack://nestjs-intro/./src/main.ts?:3:16)
 at Object../src/main.ts (C:\engage-analytics\dist\main.js:52:1)
 at __webpack_require__ (C:\engage-analytics\dist\main.js:229:32) {
code: 'MODULE_NOT_FOUND',
requireStack: [ 'C:\\engage-analytics\\dist\\main.js' ]
}

PS C:\engage-analytics> webpack -v
webpack 5.18.0
webpack-cli 4.4.0

PS C:\engage-analytics> node -v
v15.5.1

When I run the same command on the teamcity work directory I am able to start the application. After build I have simply copied the contents of dist to remote server. Here is the package.json, I am executing the highlighted scripts through npm on teamcity server.

package.json

{
name: "nestjs-intro",
version: "0.0.1",
description: "<p align="center"> <a href="http://nestjs.com/" target="blank"><img src="https://nestjs.com/img/logo_text.svg" width="320" alt="Nest Logo" /></a> </p>",
author: "",
private: true,
license: "UNLICENSED",
scripts: {
prebuild: "rimraf dist",
**build**: "npm install && npx webpack",
format: "prettier --write "src/**/*.ts" "test/**/*.ts"",
start: "nest start",
start:dev: "nest start --watch",
start:debug: "nest start --debug --watch",
**start:prod**: "node dist/main",
lint: "eslint "{src,apps,libs,test}/**/*.ts" --fix",
test: "jest",
test:watch: "jest --watch",
test:cov: "jest --coverage",
test:debug: "node --inspect-brk -r tsconfig-paths/register -r ts-node/register node_modules/.bin/jest --runInBand",
test:e2e: "jest --config ./test/jest-e2e.json"
},
dependencies: {
@nestjs/common: "^7.6.5",
@nestjs/core: "^7.5.1",
@nestjs/mongoose: "^7.2.2",
mongoose: "^5.11.3",
reflect-metadata: "^0.1.13",
@nestjs/platform-express: "^7.6.5",
rxjs: "^6.6.3"
},
devDependencies: {
@nestjs/cli: "^7.5.1",
@nestjs/schematics: "^7.1.3",
@nestjs/testing: "^7.5.1",
@types/express: "^4.17.8",
@types/jest: "^26.0.15",
@types/node: "^14.14.6",
@types/supertest: "^2.0.10",
@typescript-eslint/eslint-plugin: "^4.6.1",
@typescript-eslint/parser: "^4.6.1",
eslint: "^7.12.1",
eslint-config-prettier: "7.1.0",
eslint-plugin-prettier: "^3.1.4",
jest: "^26.6.3",
prettier: "^2.2.1",
rimraf: "^3.0.2",
ts-jest: "^26.4.4",
ts-loader: "^8.0.14",
ts-node: "^9.1.1",
tsconfig-paths: "^3.9.0",
tsconfig-paths-webpack-plugin: "^3.3.0",
typescript: "^4.0.5",
webpack: "^5.15.0",
webpack-cli: "^4.4.0"
},
jest: {
moduleFileExtensions: [
"js",
"json",
"ts"
],
rootDir: "src",
testRegex: ".*\.spec\.ts$",
transform: {
^.+\.(t|j)s$: "ts-jest"
},
collectCoverageFrom: [
"**/*.(t|j)s"
],
coverageDirectory: "../coverage",
testEnvironment: "node"
},
main: "webpack.config.js",
}

webpack-config

const webpack = require('webpack');
const path = require('path');
const nodeExternals = require('webpack-node-externals');
// const config = require('config');
const isProduction = typeof process.env.NODE_ENV !== 'undefined' && process.env.NODE_ENV === 'production';
const mode = isProduction ? 'production' : 'development';

module.exports = {
    entry: [
      'webpack/hot/poll?100',
      './src/main.ts',
    ],
    optimization: {
        minimize: false,
    },
    target: 'node',
    mode,
    externals: [
      nodeExternals({
          allowlist: ['webpack/hot/poll?100'],
      }),
    ],
    module: {
        rules: [
            {
                test: /\.tsx?$/,
                loader: 'ts-loader',
                options: {
                  transpileOnly: true
                },
                exclude: /node_modules/,
              }
        ]
    },
    resolve: {
        extensions: ['.tsx', '.ts', '.js'],
    },
    plugins: [
        new webpack.HotModuleReplacementPlugin(),
        // new webpack.WatchIgnorePlugin([/\.js$/, /\.d\.ts$/]),        
        new webpack.DefinePlugin(
            { 
                // CONFIG: JSON.stringify(config) 
            }
        ),
    ],
    output: {
        path: path.join(__dirname, 'dist'),
        filename: 'main.js',
    },
};

Do you have your webpack config available? What are you specifically moving to the server you're trying to run on? Just the webpacked file? — Jay McDoniel, Jan 29 '21 at 06:42
I have added the webpack-config. I am only copying the bundled main.js from dist to the on-premise windows server. When I run the same on the teamcity server the app starts successfully, its only with the remote server that I am facing the issue. — Nutan, Jan 29 '21 at 07:38
You have `nodeExternals` added, meaning node_modules are excluded from the final webpack file. You still need to run the production dependencies installed. — Jay McDoniel, Jan 29 '21 at 07:45
my understanding is, even if we externalize node_modules, the final file generated will have all the dependencies injected in it and doesnt need node_modules to run. Do you mean I would still need to copy node_modules? — Nutan, Jan 29 '21 at 09:21

score 1 · Accepted Answer · answered Jan 29 '21 at 16:51

1

When using nodeExternals webpack skips bundling the node_modules into the final bundle, so you till need to run your production dependency install command on your production server.

This is done due to low level dependencies of some node packages, like pg needing C++ bindings or bcrypt using node-pre-gyp and python. Bundling these dependencies into your code isn't compatible, which is why they need to be left external. There's a lot of discussion that goes more into depth on this here

answered Jan 29 '21 at 16:51

Jay McDoniel

57,339
7
135
147

is there a way to identify which dependencies I need to add to nodeExternal and which I can bundle? I did come across the link you mentioned earlier, but reading it in depth now, dont understand if I should be using webpack at all. 1) If I generate a single bundled file and just copy the output js with node_modules, that doesnt work. 2) If I add nodeExternal like its mentioned in the link, it generates a d.ts file for each in the dist folder and I still need to copy node_module to the server for it to run successfully. – Nutan Feb 01 '21 at 11:13
3) or like you suggested, copy dist contents (main.js and d.ts files), package.json to server and do a npm install – Nutan Feb 01 '21 at 11:27
WebPack s very much a utility for the frontend that _can_ be used on the backend with some caveats. This being on of the major ones. Usually it's easiest just to let it bundle the `main.js` and move it with the package.json – Jay McDoniel Feb 01 '21 at 17:15

__webpack_require__ Module not found on the deployment server

1 Answers1