Invalid requirements.txt on deploying django app to aws beanstalk

Question

I have been deploying my app without any issues to the beanstalk environment until my latest commit.

Now I get this

Time    Type    Details
2013-09-01 10:19:12 UTC+0800    ERROR   Failed to deploy application.
2013-09-01 10:19:11 UTC+0800    ERROR   Responses from [i-50f40d06] were received, but the commands failed.
2013-09-01 10:19:11 UTC+0800    ERROR   Your requirements.txt is invalid. Snapshot your logs for details.
2013-09-01 10:19:11 UTC+0800    ERROR   [Instance: i-50f40d06 Module: AWSEBAutoScalingGroup ConfigSet: Hook-PreAppDeploy] Failed on instance with return code: 1 Output: Error occurred during build: Command hooks failed .

The snapshot logs say this:

2013-09-01 02:19:08,852 [INFO] (9941 MainThread) [directoryHooksExecutor.py-29] [root directoryHooksExecutor info] Output from script: New python executable in /opt/python/run/venv/bin/python2.6
Not overwriting existing python script /opt/python/run/venv/bin/python (you must use /opt/python/run/venv/bin/python2.6)
Installing distribute..................................................................................................................................................................................................done.
Installing pip................done.
2013-09-01 02:19:08,564 ERROR    Error installing dependencies: Command '/opt/python/run/venv/bin/pip install --use-mirrors -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status -9
Traceback (most recent call last):
  File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 31, in main
    install_dependencies()
  File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 25, in install_dependencies
    shell=True)
  File "/usr/lib/python2.6/subprocess.py", line 502, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '/opt/python/run/venv/bin/pip install --use-mirrors -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status -9

2013-09-01 02:19:08,853 [ERROR] (9941 MainThread) [directoryHooksExecutor.py-34] [root directoryHooksExecutor error] Script failed with returncode 1
2

My requirements.txt is the same as before. I tried removing everything in the requirements.txt as well but I get the same error again which makes me think the problem was before requirements.txt was run.

I don't know the order in which things are deployed. ANyway, here is my config file:

packages:
  yum:
    libjpeg-devel: '6b'
    zlib-devel: []
    freetype-devel: []


container_commands:
  01_install_mysqldb:
    command: "pip install distribute==0.6.28; pip install mysql-python;"
  02_syncdb:
    command: "python manage.py syncdb --noinput"
    leader_only: true
  03_createadmin:
    command: "python scripts/createadmin.py"
    leader_only: true
  04_collectstatic:
    command: "python manage.py collectstatic --noinput"
  05_migrate_wizards:
    command: "python manage.py migrate wizards --noinput"
    leader_only: true
  06_migrate_facebook:
    command: "python manage.py migrate facebook --noinput"
    leader_only: true
  07_migrate_socialaccount:
    command: "python manage.py migrate socialaccount 0011 --noinput"
    leader_only: true
  08_migrate_missions:
    command: "python manage.py migrate missions --noinput"
    leader_only: true
  09_migrate_mailchimp:
    command: "python manage.py migrate mailchimp --noinput"
    leader_only: true
  10_migrate_actstream:
    command: "python manage.py migrate actstream --noinput"
    leader_only: true

option_settings:
  - namespace: aws:elasticbeanstalk:container:python
    option_name: WSGIPath
    value: ideatory/wsgi.py
  - namespace: aws:elasticbeanstalk:container:python:staticfiles
    option_name: /static/
    value: static/
  - option_name: DJANGO_SETTINGS_MODULE
    value: ideatory.settings

As @jvannistelrooy below said, the only thing that finally worked for me was rebuilding the environment. — vaisaghvt, Jan 30 '14 at 03:31

JacobF · Accepted Answer · 2014-04-18T08:09:52.443

17

I had the exact same problem. The only thing that worked for me was (automatically) rebuilding my entire environment. You can do that in the AWS console. Go to your environment and click Actions > Rebuild Environment. This will take some time, afterwards it will automatically relaunch your app without this error.

Update:

The problem keeps returning so now and then. I figured out that something is going wrong when pip compiles psycopg, which is needed for postgreSQL support. Haven't found a real solution yet.

edited Apr 18 '14 at 08:09

answered Jan 07 '14 at 13:48

JacobF

2,305
3
24
36

2

did you find a solution for psycopg ? I am facing a same problem and am trying to figure it out. Thanks ! – Sandeep Jul 10 '14 at 21:04
@user2608802 No, unfortunately I stopped using Elastic Beanstalk and changed my AWS setup. – JacobF Jul 16 '14 at 09:24
4

This blog from realpython.com seems to have a solution: https://realpython.com/blog/python/deploying-a-django-app-to-aws-elastic-beanstalk/ – Ceasar Apr 07 '15 at 17:35
Did anyone find a solution for this problem. I encountered the exact same error after my last commit. The interesting point is that I have not changed any line of my code. The only difference is that due to changing my computer, I had to install EB again on my machine. – 1man Dec 30 '15 at 14:21
in order to build psycopg your environment needs postgresql93-devel (or 94, depending on your platform version) – Eduard Gamonal Mar 16 '16 at 09:50
I have postgresql 10.7 running on my local ubuntu machine. Should i add postgresql==10.7 in requirements.txt before psycopg2? – Aseem Mar 23 '19 at 18:36
@Sandeep Gyz have you solve this error i am still going through this error and it sucks. – Rahul Verma Feb 24 '21 at 05:49

score 13 · Answer 2 · answered Nov 20 '14 at 23:32

13

I was able to make this work by adding postgresql93-devel and postgresql93 to my .ebextensions

as described in Customizing the Software on EC2 Instances Running Linux

This is working for 64bit Amazon Linux 2014.09 v1.0.9 running Python 2.7

in .ebextensions/01_pre_deps.config:

packages:
  yum:
    gcc-c++: []
    make: []
    openssl-devel: []
    git: []
    python-devel: []
    python27-devel: []
    ruby-devel: []
    postgresql93: []
    postgresql93-devel: []
    # nginx: []
  rubygems:
    bundler: '1.7.3'
    compass-core: '1.1.0.alpha.3'
    neat-compass: '1.6.0'

answered Nov 20 '14 at 23:32

yolk

729
7
10

I'm actually using a MySQL database so I don't think this solution applies. – vaisaghvt Dec 01 '14 at 04:42
1

this worked for me, all i had to add in my config file was the lines postgresql93 and postgresql93-devel under the packages/yum/ heading. I'm running this aws beanstalk – Jock Oct 18 '17 at 16:08
I have postgresql 10.7 running on my local ubuntu machine. Should i add postgresql10:[ ] to the .ebextensions files? – Aseem Mar 23 '19 at 18:33
@Aseem that would make sense to me. – yolk May 07 '19 at 18:56

djvg · Answer 3 · 2018-09-18T16:46:51.243

Although a different scenario than described by the OP, it may be of use to someone:

We got the same ERROR Your requirements.txt is invalid. Snapshot your logs for details. message while trying to deploy a minimal test app to a newly created t1.micro instance on Elastic Beanstalk.

The test app consisted of the basic AWS Python sample app (from here), to which we added the .ebextensions/software.config and requirements.txt from our production environment. The sole purpose was to test these two files in deployment.

Closer inspection of the logs uncovered a MemoryError during pip install. After upgrading to a t2.small instance, the deployment succeeded without issue.

score 1 · Answer 4 · answered Nov 13 '13 at 10:45

1

Have you made sure mysql-devel package is installed? It doesn't seem to be included in the yum section of your config file.

answered Nov 13 '13 at 10:45

Anas

1,761
1
13
22

I have the same problem, but I have mysql-devel installed. – JacobF Jan 07 '14 at 12:29
Yep, it's installed. As I mentioned, it was working fine till this commit. No idea what went wrong, but a rebuild of the environment was the only thing that worked. – vaisaghvt Jan 30 '14 at 03:32

score 0 · Answer 5 · answered Jul 21 '17 at 22:50

For whatever reason AWS wouldn't install postgresql-devel packets from my .extensions/01_packets.config as yolk suggested. So instead I had to add

container_commands:
  01_addPostgreSQLPacket:
    command: 'yum install postgresql95-devel'
    leader_only: true

to my 02_django.config. After That it worked just fine.

Invalid requirements.txt on deploying django app to aws beanstalk

5 Answers5

Linked