psycopg2 on elastic beanstalk - can't deploy app

Question

I'm deploying my first app(Flask) to EB and am hitting a snag. Using PostgreSQL requires that I get psycopg2 installed but I can't figure out how to get it running.

I saw in this post (Psycopg2 on Amazon Elastic Beanstalk) as well as this post(https://realpython.com/blog/python/deploying-a-django-app-to-aws-elastic-beanstalk/) that I need to add a file name .elasticbeanstalk/01_packages.config with the following code:

packages:
  yum:
    postgresql-devel: []

I've tried different variations with postgresql93-devel: [] and still no luck. Attached below are my error logs.

[2015-11-16T21:03:24.749Z] INFO  [2680]  - [Application update/AppDeployStage0/AppDeployPreHook/03deploy.py] : Starting activity...
[2015-11-16T21:03:27.024Z] INFO  [2680]  - [Application update/AppDeployStage0/AppDeployPreHook/03deploy.py] : Activity execution failed, because: Collecting psycopg2==2.6.1 (from -r /opt/python/ondeck/app/requirements.txt (line 1))
    Using cached psycopg2-2.6.1.tar.gz
      Complete output from command python setup.py egg_info:
      running egg_info
      creating pip-egg-info/psycopg2.egg-info
      writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt
      writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt
      writing pip-egg-info/psycopg2.egg-info/PKG-INFO
      writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'
      warning: manifest_maker: standard file '-c' not found

      Error: pg_config executable not found.

      Please add the directory containing pg_config to the PATH
      or specify the full executable path with the option:

          python setup.py build_ext --pg-config /path/to/pg_config build ...

      or with the pg_config option in 'setup.cfg'.

      ----------------------------------------
  Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-z20gvx7_/psycopg2
  2015-11-16 21:03:27,010 ERROR    Error installing dependencies: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 1
  Traceback (most recent call last):
    File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 22, in main
      install_dependencies()
    File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 18, in install_dependencies
      check_call('%s install -r %s' % (os.path.join(APP_VIRTUAL_ENV, 'bin', 'pip'), requirements_file), shell=True)
    File "/usr/lib64/python2.7/subprocess.py", line 540, in check_call
      raise CalledProcessError(retcode, cmd)
  CalledProcessError: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 1 (ElasticBeanstalk::ExternalInvocationError)
caused by: Collecting psycopg2==2.6.1 (from -r /opt/python/ondeck/app/requirements.txt (line 1))
    Using cached psycopg2-2.6.1.tar.gz
      Complete output from command python setup.py egg_info:
      running egg_info
      creating pip-egg-info/psycopg2.egg-info
      writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt
      writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt
      writing pip-egg-info/psycopg2.egg-info/PKG-INFO
      writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'
      warning: manifest_maker: standard file '-c' not found

      Error: pg_config executable not found.

      Please add the directory containing pg_config to the PATH
      or specify the full executable path with the option:

          python setup.py build_ext --pg-config /path/to/pg_config build ...

      or with the pg_config option in 'setup.cfg'.

      ----------------------------------------
  Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-z20gvx7_/psycopg2
  2015-11-16 21:03:27,010 ERROR    Error installing dependencies: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 1
  Traceback (most recent call last):
    File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 22, in main
      install_dependencies()
    File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 18, in install_dependencies
      check_call('%s install -r %s' % (os.path.join(APP_VIRTUAL_ENV, 'bin', 'pip'), requirements_file), shell=True)
    File "/usr/lib64/python2.7/subprocess.py", line 540, in check_call
      raise CalledProcessError(retcode, cmd)
  CalledProcessError: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 1 (Executor::NonZeroExitStatus)

It seems pretty clear that I need to get psycopg2 installed but I'm not sure what other steps I can take to make this work.

I have the same. I'll let you know if I solve it. – Cephlin Dec 06 '15 at 17:22 — Cephlin, Dec 06 '15 at 17:22

score 19 · Accepted Answer · answered Jan 22 '16 at 23:45

19

I solved it, you need to open you EC2 instance first, then install like this

sudo yum install postgresql-devel

Then deploy your EB

answered Jan 22 '16 at 23:45

Muke

694
8
15

was able to complete `sudo install psycopg2` only after the command you provided! – Vadim Apr 04 '16 at 01:47
Thanks. Just wondering, if we have multiple ec2 instances when using EB, do we need to run this command on each isntance? – Fawwaz Yusran Nov 16 '20 at 03:09
1

This does not solve the problem. If you have more instances, or when the instance is replaced then this would fail again – alacret Jun 18 '21 at 23:39

Nick Brady · Answer 2 · 2020-08-05T19:20:17.637

Funny enough, the solution for me for Amazon Linux 2 ended up being what the original question was - adding a 01_package.conf with the contents of

packages:
  yum:
    postgresql-devel: []

Furthmore, this was contrary to almost all answers I found online which directed to use postgresql96-devel or some other number. I found out I needed to make the package name postgresql-devel after SSHing onto the EC2 instance and viewing all available Yum packages.

[ec2-user@ip-xxx-xxx-xxx-xxx ~]$ yum list available | grep 'postgresql'
freeradius-postgresql.x86_64           3.0.13-10.amzn2                amzn2-core
pcp-pmda-postgresql.x86_64             3.12.2-5.amzn2                 amzn2-core
postgresql.x86_64                      10.4-5.amzn2.0.2               amzn2extra-postgresql10
postgresql-contrib.x86_64              10.4-5.amzn2.0.2               amzn2extra-postgresql10
postgresql-devel.x86_64                10.4-5.amzn2.0.2               @amzn2extra-postgresql10
postgresql-docs.x86_64                 10.4-5.amzn2.0.2               amzn2extra-postgresql10
...

and I saw it was just postgres-devel. Once I changed it to that it deployed alright. Make sure you commit your change if you're using eb with git before running eb deploy - or at least add the changes to your git staging area and run with eb deploy --staged as well. That one cost me a run around thinking I had already tried the package name when I hadn't

I found the answers on this stackoverflow post here to be helpful as well. Good luck!

I ended up needing a newer psql client (11.x) than what was default using the above answer (9.2 at the time of posting). This answer here was the only way I could make it work. Answer pasted here for posterity...

packages:
    yum:
        amazon-linux-extras: []

commands:
    01_postgres_activate:
        command: sudo amazon-linux-extras enable postgresql11
    02_postgres_install:
        command: sudo yum install -y postgresql-devel

Your comment solved the issue for me, you're amazing. For others out there using Amazon Linux 2 - this is the solution. — Sahar, Jul 22 '20 at 12:00
thank you this one gave me a lot of trouble - especially since most documentation says to use `postgresql96-devel` (including AWS's) or some other version... even put up a bounty without luck. Glad my efforts helped someone else!! — Nick Brady, Jul 22 '20 at 13:38
Thank you for your answer! Took a long time to figure out that the version number was messing my deployment, specially when so many tutorials point to a postgresql with a version — Eduardo Matsuoka, Nov 26 '20 at 14:04
What is 11 at the end of `postgresql11`? Should I use 12 instead if I try to connect to RDS with postgres 12.5? — Sergey Zakharov, Mar 31 '21 at 08:20
I think you may be out of luck unless you want to build it yourself. If you speak with support staff please comment out. I checked on AL2 with `amazon-linux-extras list | grep 'postgres'` and only see 9.6, 10, and 11. I remember having a hell of a time getting elastic beanstalk going... good luck. — Nick Brady, Mar 31 '21 at 12:51

score 3 · Answer 3 · answered Mar 05 '16 at 15:13

3

Even after installing postgresql93-devel, I kept getting an error "no module named psycopg2". To solve this I had to connect to my enviroment with eb ssh and then execute sudo pip install psycopg2 to solve this.

answered Mar 05 '16 at 15:13

joalquipe

31
2

score 0 · Answer 4 · answered Jul 18 '22 at 22:39

You mis-named the directory. Config files go in the .ebextensions directory, you have yours in the .elasticbeanstalk directory. That won't work. Try:

mkdir .ebextensions && mv .elasticbeanstalk/01_packages.config .ebextensions

Then deploy your app again.

score 0 · Answer 5 · answered Jun 29 '23 at 09:13

0

i solved the issue by: 1.run : eb ssh 2. In the ec2 terminal run: sudo yum install -y postgresql-devel

redeploy your app. it should deploy without the error

answered Jun 29 '23 at 09:13

maluki

1

Can you elaborate? – TheTridentGuy supports Ukraine Jul 03 '23 at 16:11

psycopg2 on elastic beanstalk - can't deploy app

5 Answers5

Linked