5

Question

What is wrong with my Dockerfile or bitbucket-pipelines.yml? Why are modules missing from the bitbucket pipelines environment?

Error

When I try to npm run build my Vue2 project with webpack using Bitbucket Pipelines, I get errors regarding missing modules.

From Logs

npm run build
> people-is@1.0.0 build /opt/atlassian/pipelines/agent/build
> node build/build.js
module.js:549
    throw err;
    ^
Error: Cannot find module 'cli-spinners'

Files

Here are the files for configuration.

Dockerfile - builds cportwine/people-is

FROM node:8.10.0

RUN npm install
RUN npm install -g firebase-tools

CMD [ "npm", "run", "build" ]

bitbucket-pipelines.yml

image:
  name: cportwine/people-is
pipelines:
  default:
    - step:
        script:
          - npm run build

package.json

{
  "name": "people-is",
  "version": "1.0.0",
  "description": "A Vue.js project",
  "author": "cportwine",
  "private": true,
  "scripts": {
    "dev": "node build/dev-server.js",
    "start": "node build/dev-server.js",
    "build": "node build/build.js",
    "deploy": "firebase deploy --token $FIREBASE_TOKEN"
  },
  "dependencies": {
    "rxjs": "^5.5.8",
    "uuid": "^3.2.1",
    "vue": "^2.5.16",
    "vue-json-excel": "^0.1.9",
    "vue-router": "^2.8.1",
    "vue-rx": "^5.0.0",
    "vuefire": "^1.4.5",
    "vuetify": "^0.15.2"
  },
  "devDependencies": {
    "autoprefixer": "^7.2.6",
    "babel-core": "^6.22.1",
    "babel-loader": "^7.1.4",
    "babel-plugin-transform-runtime": "^6.22.0",
    "babel-preset-env": "^1.6.1",
    "babel-preset-stage-2": "^6.22.0",
    "babel-register": "^6.22.0",
    "chalk": "^2.3.2",
    "connect-history-api-fallback": "^1.5.0",
    "copy-webpack-plugin": "^4.5.1",
    "css-loader": "^0.28.11",
    "cssnano": "^3.10.0",
    "eslint": "^4.19.1",
    "eslint-config-standard": "^11.0.0",
    "eslint-friendly-formatter": "^3.0.0",
    "eslint-loader": "^1.9.0",
    "eslint-plugin-html": "^4.0.2",
    "eslint-plugin-promise": "^3.7.0",
    "eslint-plugin-standard": "^3.0.1",
    "eventsource-polyfill": "^0.9.6",
    "express": "^4.16.3",
    "extract-text-webpack-plugin": "^2.0.0",
    "file-loader": "^0.11.1",
    "firebase": "^4.12.0",
    "firebase-tools": "^3.17.7",
    "friendly-errors-webpack-plugin": "^1.1.3",
    "html-webpack-plugin": "^2.28.0",
    "http-proxy-middleware": "^0.17.3",
    "opn": "^5.3.0",
    "optimize-css-assets-webpack-plugin": "^2.0.0",
    "ora": "^1.4.0",
    "rimraf": "^2.6.0",
    "semver": "^5.5.0",
    "shelljs": "^0.7.6",
    "url-loader": "^0.5.8",
    "vue-loader": "^13.7.1",
    "vue-style-loader": "^3.1.2",
    "vue-template-compiler": "^2.5.16",
    "vuex": "^2.5.0",
    "webpack": "^2.6.1",
    "webpack-bundle-analyzer": "^2.11.1",
    "webpack-dev-middleware": "^1.12.2",
    "webpack-hot-middleware": "^2.21.2",
    "webpack-merge": "^4.1.2"
  },
  "engines": {
    "node": ">=8.10.0",
    "npm": ">= 5.6.0"
  },
  "browserslist": [
    "> 1%",
    "last 2 versions",
    "not ie <= 8"
  ],
  "main": "index.js",
  "repository": {
    "type": "git",
    "url": "git+https://chaddportwine@bitbucket.org/jahnelgroup/people-is.git"
  },
  "keywords": [],
  "license": "ISC",
  "homepage": "https://bitbucket.org/jahnelgroup/people-is#readme"
}

What I see

When I ls the node_modules folder in both environments, they do not match. Modules are missing from bitbucket pipelines.

local folder

people-is/node_modules

...
chalk
char-spinner
chardet
check-types
chokidar
chownr
cipher-base
circular-json
cjson
clap
class-utils
clean-css
cli-boxes
cli-cursor
cli-spinners
cli-table
cli-table2
cli-width
cliui
...

bitbucket folder

/opt/atlassian/pipelines/agent/build/node_modules

Woah, missing modules!

...
chalk
cli-cursor
co
...

What I have tried

I added a command to the bitbucket-pipelines.yml to npm install before I build.

bitbucket-pipelines.yml

image:
  name: cportwine/people-is
pipelines:
  default:
    - step:
        script:
          - npm install
          - npm run build

This adds some additional modules (like cli-spinners from the error) to /opt/atlassian/pipelines/agent/build/node_modules.

bitbucket folder

/opt/atlassian/pipelines/agent/build/node_modules

...
chalk
char-spinner
chardet
check-types
chokidar
chownr
cipher-base
circular-json
cjson
clap
class-utils
clean-css
cli-boxes
cli-cursor
cli-spinners
cli-table
cli-table2
cli-width
cliui
clone
clone-response
co
...

However, the build command still fails, due to a different missing module.

Error

> people-is@1.0.0 build /opt/atlassian/pipelines/agent/build
> node build/build.js
module.js:549
    throw err;
    ^
Error: Cannot find module './_safeGet'
Chadd
  • 636
  • 7
  • 30
  • Have you checked if you have same versions of node and npm on both environments? – BeeBee8 Mar 29 '18 at 18:58
  • @BhushanBabar yes, I see: Local - `node v8.10.0` `npm 5.6.0`; Pipelines - `node v8.10.0` `npm 5.6.0`. To get the versions I added `- node --version` and `- npm --version` to the `bitbucket-pipelines.yml` file. Is there a better way to confirm the versions in Bitbucket Pipelines? – Chadd Mar 29 '18 at 19:54
  • No that's Ok, have tried shifting your `dev dependencies` to `dependencies` in package.json? – BeeBee8 Mar 30 '18 at 02:41
  • @BhushanBabar, I did try putting **all** the package.json dependencies in `devDependencies`, which is appropriate for this VueJS project, however, it had no effect on the bitbucket pipelines build. I found a different solution to get the project building. The project now builds with dependencies in both `dependencies` and in `devDependencies`. Although, I've decided to move them all to `devDependencies`. – Chadd Apr 03 '18 at 15:55
  • I have found that in the `bitbucket-pipeline.yml` file, I must `rm -r node_modules` and then `npm install -g npm` and `npm install`. This installs all the required dependencies within `node_modules`. **I'm not sure why I need to remove and install anything. I thought the Docker Image sorted this out so Bitbucket Pipelines didn't need to.** – Chadd Apr 03 '18 at 16:02
  • Nice. You also have option to cache the `node_modules` in `bitbucket pipeline`, I don't know what is the default if don't specifically mention it. – BeeBee8 Apr 03 '18 at 16:49

2 Answers2

2

Solutions

I can now build the app, but I don't know why!

1 - Simplify the Dockerfile

I removed all the npm commands. Maybe the npm install commands were redundant? There was no advantage using the Docker Image to pre-install npm packages.

2 - Remove Node_Modules before install

Using the bitbucket-pipelines.yml, remove the node_modules folder, and then perform npm install -g npm and npm install and npm install -g firebase-tools.

File Changes

bitbucket-pipelines.yml (added lines)

image:
  name: cportwine/people-is
pipelines:
  default:
    - step:
        script:
          - rm -r node_modules              <---- remove
          - npm install -g npm              <---- install
          - npm install                     <---- install
          - npm install -g firebase-tools   <---- install
          - npm run build

Dockerfile (lines removed)

FROM node:8.10.0

                                            <---- remove

CMD [ "npm", "run", "build" ]

Answer ?

I'm not sure why moving all the npm install stuff into the bitbucket.pipelines.yml solved my issue building the project. I thought Docker would enable me to define my environment, e.g., install a version of node/npm and firebase. And pipelines would "run" that.

If someone could explain what I am missing, I would accept that answer.

Chadd
  • 636
  • 7
  • 30
1

Answer

I received support from the Atlassian Team

  1. Leave npm install -g firebase in the docker image.
  2. Move npm install from the docker image to the bitbucket-pipelines.yml file.

Reason

The node_modules folder was listed in .gitignore

tl;dr

My mistake - I forgot about .gitignore and how that affects the node_modules folder in source control, e.g., Bitbucket Pipelines.

I was looking at my local node_modules folder and building locally which worked.

However

The node_modules in source control, by design, is not in-sync with my local folder because it's included in the .gitignore file.

So

It was necessary for me to rm node_modules and npm install using the bitbucket-pipelines.yml. Now, BitPipes finds the modules I have installed locally.

This is sort of the point of maintaining the package.json, but I got confused.

Chadd
  • 636
  • 7
  • 30