AWS elastic beanstalk not working with Librosa python

Question

My application works properly on the local machine. However, as I uploaded the application to elastic beanstalk, the import of librosa library broke the application. How to solve the issue?

import os
import pandas as pd
import librosa
import numpy as np
from sklearn.preprocessing import StandardScaler
# split training and test data
from sklearn.model_selection import train_test_split
from flask import Flask
from flask import request
from flask import Response

# EB looks for an 'application' callable by default.
application = Flask(__name__)


@application.route('/', methods=['Get'])
def homepage():
    return "Hello World"

# run the app.
if __name__ == "__main__":
    # Setting debug to True enables debug output. This line should be
    # removed before deploying a production app.
    application.debug = True
    application.run()

I am using Python 3.8 on Amazon linux 2. The packages are installed using requirements.txt :

flask==1.1.2
flask_cors==3.0.10
imageio==2.9.0
librosa==0.8.1
moviepy==1.0.3
numpy==1.19.0
pandas==1.2.3
scikit-learn==0.24.1
tensorflow==2.2.0
werkzeug==1.0.1

The log is given below:

2022/02/11 20:22:31.051794 [ERROR] An error occurred during execution of command [app-deploy] - [InstallDependency]. Stop running the command. Error: fail to install dependencies with requirements.txt file with error Command /bin/sh -c /var/app/venv/staging-LQM1lest/bin/pip install -r requirements.txt failed with error exit status 2. Stderr:ERROR: Exception:
Traceback (most recent call last):
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 164, in exc_logging_wrapper
    status = run_func(*args)
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
    return func(self, options, args)
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 339, in run
    reqs, check_supported_wheels=not options.target_dir
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 93, in resolve
    collected.requirements, max_rounds=try_to_avoid_resolution_too_deep
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_vendor/resolvelib/resolvers.py", line 482, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_vendor/resolvelib/resolvers.py", line 349, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria
    if not criterion.candidates:
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_vendor/resolvelib/structs.py", line 151, in __bool__
    return bool(self._sequence)
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
    return any(self)
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
    return (c for c in iterator if id(c) not in self._incompatible_ids)
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
    candidate = func()
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link
    version=version,
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 287, in __init__
    version=version,
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__
    self.dist = self._prepare()
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 225, in _prepare
    dist = self._prepare_distribution()
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 292, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 482, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 528, in _prepare_linked_requirement
    link, req.source_dir, self._download, self.download_dir, hashes
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 217, in unpack_url
    hashes=hashes,
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 94, in get_http_url
    from_path, content_type = download(link, temp_dir.path)
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_internal/network/download.py", line 145, in __call__
    for chunk in chunks:
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_internal/cli/progress_bars.py", line 144, in iter
    for x in it:
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_internal/network/utils.py", line 87, in response_chunks
    decode_content=False,
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_vendor/urllib3/response.py", line 576, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_vendor/urllib3/response.py", line 519, in read
    data = self._fp.read(amt) if not fp_closed else b""
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_vendor/cachecontrol/filewrapper.py", line 65, in read
    self._close()
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_vendor/cachecontrol/filewrapper.py", line 52, in _close
    self.__callback(self.__buf.getvalue())
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_vendor/cachecontrol/controller.py", line 309, in cache_response
    cache_url, self.serializer.dumps(request, response, body=body)
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_vendor/cachecontrol/serialize.py", line 72, in dumps
    return b",".join([b"cc=4", msgpack.dumps(data, use_bin_type=True)])
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_vendor/msgpack/__init__.py", line 35, in packb
    return Packer(**kwargs).pack(o)
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_vendor/msgpack/fallback.py", line 960, in pack
    self._pack(obj)
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_vendor/msgpack/fallback.py", line 944, in _pack
    len(obj), dict_iteritems(obj), nest_limit - 1
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_vendor/msgpack/fallback.py", line 1045, in _pack_map_pairs
    self._pack(v, nest_limit - 1)
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_vendor/msgpack/fallback.py", line 944, in _pack
    len(obj), dict_iteritems(obj), nest_limit - 1
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_vendor/msgpack/fallback.py", line 1045, in _pack_map_pairs
    self._pack(v, nest_limit - 1)
  File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/pip/_vendor/msgpack/fallback.py", line 889, in _pack
    return self._buffer.write(obj)

Can anyone please confirm if the application works on EB?

The application works fine if i don't import librosa library. Here is the error message: [ERROR] An error occurred during execution of command [app-deploy] - [InstallDependency]. Stop running the command. Error: fail to install dependencies with requirements.txt file with error Command /bin/sh -c /var/app/venv/staging-LQM1lest/bin/pip install -r requirements.txt failed with error exit status 2. Stderr:ERROR: Exception: Traceback (most recent call last): — Md. Rayhan, Feb 12 '22 at 04:56
Sadly your question lacks details and is unclear. You haven't even provided exact version of EB platform that you use, nor even how do you setup librosa in your EB. — Marcin, Feb 12 '22 at 05:02
Thank you for your suggestion. I edited my question with the details. Please help — Md. Rayhan, Feb 12 '22 at 05:07
Did you ssh to the EB instance and tried to manually install the librosa to verify that it even is supported in EB? — Marcin, Feb 12 '22 at 05:11
No. I am new to EB so could you please tell me how to ssh to EB and manually install a python library? — Md. Rayhan, Feb 12 '22 at 05:18
You wrote "I am using Python 3.8", yet the log says you are installing everything using `python3.7`. Sadly your question is not clear. — Marcin, Feb 12 '22 at 23:04

score 0 · Accepted Answer · answered Feb 12 '22 at 23:21

0

The issue is probably with tensorflow==2.2.0. This is a very heavy library and you can't install it on t2.micro. You need at least t2.medium (not in free tier) which has more RAM to successfully install tensorflow==2.2.0 on EB.

answered Feb 12 '22 at 23:21

Marcin

215,873
14
235
294

AWS elastic beanstalk not working with Librosa python

1 Answers1