How to install Node Packages for an Elastic Beanstalk Python 3.7 Project Running on 64bit Amazon Linux 2?

Question

I am using a Django package named django-pipeline to compress js and css files.

When I deploy my project, I run Django's collectstatic command, from .ebextensions folder:

01_django.config:

container_commands:
    ...
    07_collectstatic:
        command: "source /var/app/venv/*/bin/activate && python3 manage.py collectstatic --noinput"

The problem is, django-pipeline uses compressor libraries that require Node.

I have copied two libraries (cssmin and terser) from my node_modules directory locally to my static directory.

static
|_vendor
|___|.bin
|_____|cssmin
|_____|cssmin.cmd
|_____|terser
|_____|terser.cmd
|___|cssmin
|_____|...
|___|terser
|_____|...

Secondly, I have the following setin on the Pipeline to tell the pipeline where the binaries exist:

PIPELINE_CONFIG.update({
    'CSSMIN_BINARY': os.path.join(BASE_DIR, "static", "vendor", ".bin", "cssmin"),
    'TERSER_BINARY': os.path.join(BASE_DIR, "static", "vendor", ".bin", "terser"),
})

The Error

2021-09-14 05:00:58,513 P4080 [INFO] ============================================================
2021-09-14 05:00:58,513 P4080 [INFO] Command 07_collectstatic
2021-09-14 05:00:59,502 P4080 [INFO] -----------------------Command Output-----------------------
2021-09-14 05:00:59,503 P4080 [INFO]    Traceback (most recent call last):
2021-09-14 05:00:59,503 P4080 [INFO]      File "manage.py", line 21, in <module>
2021-09-14 05:00:59,503 P4080 [INFO]        main()
2021-09-14 05:00:59,503 P4080 [INFO]      File "manage.py", line 17, in main
2021-09-14 05:00:59,503 P4080 [INFO]        execute_from_command_line(sys.argv)
2021-09-14 05:00:59,503 P4080 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
2021-09-14 05:00:59,503 P4080 [INFO]        utility.execute()
2021-09-14 05:00:59,503 P4080 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/django/core/management/__init__.py", line 413, in execute
2021-09-14 05:00:59,503 P4080 [INFO]        self.fetch_command(subcommand).run_from_argv(self.argv)
2021-09-14 05:00:59,503 P4080 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/django/core/management/base.py", line 354, in run_from_argv
2021-09-14 05:00:59,503 P4080 [INFO]        self.execute(*args, **cmd_options)
2021-09-14 05:00:59,503 P4080 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/django/core/management/base.py", line 398, in execute
2021-09-14 05:00:59,503 P4080 [INFO]        output = self.handle(*args, **options)
2021-09-14 05:00:59,503 P4080 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 187, in handle
2021-09-14 05:00:59,503 P4080 [INFO]        collected = self.collect()
2021-09-14 05:00:59,503 P4080 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 128, in collect
2021-09-14 05:00:59,503 P4080 [INFO]        for original_path, processed_path, processed in processor:
2021-09-14 05:00:59,504 P4080 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pipeline/storage.py", line 30, in post_process
2021-09-14 05:00:59,504 P4080 [INFO]        packager.pack_stylesheets(package)
2021-09-14 05:00:59,504 P4080 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pipeline/packager.py", line 98, in pack_stylesheets
2021-09-14 05:00:59,504 P4080 [INFO]        variant=package.variant, **kwargs)
2021-09-14 05:00:59,504 P4080 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pipeline/packager.py", line 116, in pack
2021-09-14 05:00:59,504 P4080 [INFO]        content = compress(paths, **kwargs)
2021-09-14 05:00:59,504 P4080 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pipeline/compressors/__init__.py", line 75, in compress_css
2021-09-14 05:00:59,504 P4080 [INFO]        css = getattr(compressor(verbose=self.verbose), 'compress_css')(css)
2021-09-14 05:00:59,504 P4080 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pipeline/compressors/cssmin.py", line 8, in compress_css
2021-09-14 05:00:59,504 P4080 [INFO]        return self.execute_command(command, css)
2021-09-14 05:00:59,504 P4080 [INFO]      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pipeline/compressors/__init__.py", line 250, in execute_command
2021-09-14 05:00:59,504 P4080 [INFO]        raise CompressorError(stderr)
2021-09-14 05:00:59,504 P4080 [INFO]    pipeline.exceptions.CompressorError: b'/var/app/staging/static/vendor/.bin/cssmin: line 12: node: command not found\n'
2021-09-14 05:00:59,504 P4080 [INFO] ------------------------------------------------------------
2021-09-14 05:00:59,504 P4080 [ERROR] Exited with error code 1

My Question

How do I install node within a Python 3.7 Elastic Beanstalk environment so this collectstatic command will work?

Edit #1

I tried:

commands:
  05_install node:
    command: |
      curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.34.0/install.sh | bash
      . ~/.nvm/nvm.sh
      nvm install node

It returns the same error:

pipeline.exceptions.CompressorError: b'/var/app/staging/static/vendor/.bin/cssmin: line 12: node: command not found\n'

This is what the cssmin file where the error is coming from looks like:

#!/bin/sh
basedir=$(dirname "$(echo "$0" | sed -e 's,\\,/,g')")

case `uname` in
    *CYGWIN*) basedir=`cygpath -w "$basedir"`;;
esac

if [ -x "$basedir/node" ]; then
  "$basedir/node"  "$basedir/../cssmin/bin/cssmin" "$@"
  ret=$?
else 
  node  "$basedir/../cssmin/bin/cssmin" "$@" <----------- line 12
  ret=$?
fi
exit $ret

I have tried prebuild hooks:

#!/bin/bash

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.34.0/install.sh | bash
source /root/.nvm/nvm.sh
nvm install node
export NVM_DIR="/root/.nvm"
node -e "console.log('Running Node.js ' + process.version)"

The result of prebuild hooks, in eb-hooks.log from Elastic Beanstalk:

2021/09/14 15:13:59.896274 [INFO] => nvm is already installed in /root/.nvm, trying to update the script

=> nvm source string already in /root/.bashrc
=> bash_completion source string already in /root/.bashrc
=> You currently have modules installed globally with `npm`. These will no
=> longer be linked to the active version of Node when you install a new node
=> with `nvm`; and they may (depending on how you construct your `$PATH`)
=> override the binaries of modules installed with `nvm`:

/root/.nvm/versions/node/v16.9.1/lib
+-- corepack@0.9.0
=> If you wish to uninstall them at a later point (or re-install them under your
=> `nvm` Nodes), you can remove them from the system Node as follows:

     $ nvm use system
     $ npm uninstall -g a_module

=> Close and reopen your terminal to start using nvm or run the following to use it now:

export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"  # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"  # This loads nvm bash_completion
Now using node v16.9.1 (npm v7.21.1)
Running Node.js v16.9.1

The output above shows that node installs, but my collectstatic container command still fails with the inability to find node.

After it fails, I can SSH into the instance and see the environment variables for locating nvm exist:

declare -x NVM_BIN="/root/.nvm/versions/node/v16.9.1/bin"
declare -x NVM_CD_FLAGS=""
declare -x NVM_DIR="/root/.nvm"

I got an interesting output when I ran which node in container_commands:

2021-09-14 16:40:46,512 P8087 [INFO]    which: no node in (/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin)
2021-09-14 16:40:46,512 P8087 [INFO] ------------------------------------------------------------
2021-09-14 16:40:46,512 P8087 [ERROR] Exited with error code 1

It seems like node is not on the system path in that moment?

Marcin · Accepted Answer · 2021-09-15T03:38:11.633

1

You can use commands and do the following:

UPDATE:

commands:
    05_install node:
        command: |
          curl -sL https://rpm.nodesource.com/setup_16.x | sudo bash -
          yum install -y nodejs

edited Sep 15 '21 at 03:38

answered Sep 14 '21 at 05:50

Marcin

215,873
14
235
294

I failed to mention it but I had tried that as well as predeploy hook .sh files. I provided more context under "Edit #1" of my question. – Jarad Sep 14 '21 at 16:41
@Jarad Did you inspect EB logs or maybe did you try to ssh into the EB instance and pareform the node installation yourself? Just to check if it works? Because the steps should work. – Marcin Sep 15 '21 at 01:39
Yes, I've spent a lot of time in SSH. I agree the steps should work too. The only way I've got it to work is if I SSH and manually install on an instance, then deploy. I want to avoid anything manual for auto-scaling reasons in the future. I'll keep at it. – Jarad Sep 15 '21 at 03:05
@Jarad Does the `commands` I provided install `node`? If you ssh, can you verify it is installed or not? – Marcin Sep 15 '21 at 03:10
@Jarad I verified now, and the `node` is successfuly installed by the `commands` `.ebextention` file I provided. But it is installed in `/root/.nvm/versions/node/v16.9.1/bin/`. Thus your scripts can't find it. – Marcin Sep 15 '21 at 03:24
@Jarad The answer updated. Now it is installed in global scope. – Marcin Sep 15 '21 at 03:37
1

Thanks for that alternative node install approach! You are exactly right; it kept installing in the `/root` location and wasn't in the path so Elastic Beanstalk didn't see it. – Jarad Sep 15 '21 at 22:37

user1716731 · Answer 2 · 2022-09-19T09:05:27.103

This answer is with respect to the installtion of node. This is what that worked for me after too many trial and errors with .ebextensions

# install node

commands:
    01_fetch_sh:
        test: '[ ! -f /usr/bin/node ] && echo "node not installed"'
        command: "curl -sL https://rpm.nodesource.com/setup_16.x | sudo bash -"
    02_install latest node:
        test: '[ ! -f /usr/bin/node ] && echo "node not installed"'
        command: "sudo yum install -y nodejs"

Note: As this is YAML format, pls. make sure not to start with any tab, but rather provide spaces for the whitespaces that you see

As we don't want to install node, every time a new app version is deployed, it is better to do a check and then install. That way, if new instances get created due to auto scaling etc, then node will be installed. In the existing instances, when we deploy a new version of the app, node will not be installed.

How to install Node Packages for an Elastic Beanstalk Python 3.7 Project Running on 64bit Amazon Linux 2?

The Error

My Question

Edit #1

2 Answers2