Deploy Postgres11 to Elastic Beanstalk - Requires /etc/redhat-release

Question

I'm having a hell of a time deploying my first app to Elastic Beanstalk and could really use some help. I cannot get Postgres11 to install though it is officially supported with RDS.

ISSUE
If I run eb deploy I get the message saying pg_config executable not found. It is required to build psycopg2 from source.

/usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'project_urls'
  warnings.warn(msg)
running egg_info
creating pip-egg-info/psycopg2.egg-info
writing pip-egg-info/psycopg2.egg-info/PKG-INFO
writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt
writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt
writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'

Error: pg_config executable not found.

pg_config is required to build psycopg2 from source.  Please add the directory
containing pg_config to the $PATH or specify the full executable path with the
option: ...

I suppose I'll need to add the repo? Fair enough. Next I try adding the repo as I have found in other posts across the internet:

[ec2-user@ip-... etc]$ sudo yum -y install https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7-x86_64/pgdg-centos11-11-2.noarch.rpm
Loaded plugins: priorities, update-motd, upgrade-helper
pgdg-centos11-11-2.noarch.rpm                                          | 5.6 kB  00:00:00     
Examining /var/tmp/yum-root-cQJP_4/pgdg-centos11-11-2.noarch.rpm: pgdg-redhat-repo-42.0-4.noarch
Marking /var/tmp/yum-root-cQJP_4/pgdg-centos11-11-2.noarch.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package pgdg-redhat-repo.noarch 0:42.0-4 will be installed
--> Processing Dependency: /etc/redhat-release for package: pgdg-redhat-repo-42.0-4.noarch
--> Processing Dependency: /etc/redhat-release for package: pgdg-redhat-repo-42.0-4.noarch
--> Finished Dependency Resolution
Error: Package: pgdg-redhat-repo-42.0-4.noarch (/pgdg-centos11-11-2.noarch)
           Requires: /etc/redhat-release

From here I'm stuck. I've tried symlinking /etc/system-release -> /etc/redhat-release with no luck. Nobody else seems to have this problem? I also don't seem to have the coveted amazon-linux-extras for some reason?

Environment

Environment tier: Web Server
Platform: Python 3.6 running on 64bit Amazon Linux/2.8.2

---

.ebextensions/packages.config

packages:
  yum:
    postgresql11-devel: []

requirements.txt

Django==2.2
psycopg2==2.8.2
pytz==2019.1
sqlparse==0.3.0

---

[ec2-user@ip-... etc]$ cat /etc/os-release 
NAME="Amazon Linux AMI"
VERSION="2018.03"
ID="amzn"
ID_LIKE="rhel fedora"
VERSION_ID="2018.03"
PRETTY_NAME="Amazon Linux AMI 2018.03"
ANSI_COLOR="0;33"
CPE_NAME="cpe:/o:amazon:linux:2018.03:ga"
HOME_URL="http://aws.amazon.com/amazon-linux-ami/"

[ec2-user@ip-... etc]$ cat /etc/system-release 
Amazon Linux AMI release 2018.03

Answer 1

Alternatively, you can build posgresql from source:

wget https://ftp.postgresql.org/pub/source/v11.5/postgresql-11.5.tar.gz

tar zxvf postgresql-11.5.tar.gz

cd postgresql-11.5

./configure --without-readline

make

make install

Answer 2

PostgreSQL 11 is not yet available from Amazon but PostgreSQL 10 is. I'm using Amazon Linux release 2 (Karoo) reported by cat /etc/system-release. To enable installation:

$ sudo amazon-linux-extras enable postgresql10

Once you enable this extra, you'll then see many packages for PostgreSQL 10 available through yum that can be installed normally:

$ yum list postgresql*
Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
Installed Packages
postgresql.x86_64                    10.4-5.amzn2.0.2    @amzn2extra-postgresql10
postgresql-devel.x86_64              10.4-5.amzn2.0.2    @amzn2extra-postgresql10
postgresql-libs.x86_64               10.4-5.amzn2.0.2    @amzn2extra-postgresql10
Available Packages
postgresql-contrib.x86_64            10.4-5.amzn2.0.2    amzn2extra-postgresql10
postgresql-docs.x86_64               10.4-5.amzn2.0.2    amzn2extra-postgresql10
postgresql-libs.i686                 10.4-5.amzn2.0.2    amzn2extra-postgresql10
postgresql-plperl.x86_64             10.4-5.amzn2.0.2    amzn2extra-postgresql10
postgresql-plpython.x86_64           10.4-5.amzn2.0.2    amzn2extra-postgresql10
postgresql-plpython3.x86_64          10.4-5.amzn2.0.2    amzn2extra-postgresql10
postgresql-pltcl.x86_64              10.4-5.amzn2.0.2    amzn2extra-postgresql10
postgresql-server.x86_64             10.4-5.amzn2.0.2    amzn2extra-postgresql10
postgresql-static.x86_64             10.4-5.amzn2.0.2    amzn2extra-postgresql10
postgresql-test.x86_64               10.4-5.amzn2.0.2    amzn2extra-postgresql10
postgresql-test-rpm-macros.x86_64    10.4-5.amzn2.0.2    amzn2extra-postgresql10
postgresql-upgrade.x86_64            10.4-5.amzn2.0.2    amzn2extra-postgresql10
postgresql-upgrade-devel.x86_64      10.4-5.amzn2.0.2    amzn2extra-postgresql10

Answer 3

I have the same problem with PostgreSQL 10 for a Django 2.1 project on AWS Elastic Beanstalk.

The issue was introduced last week around 17 Apr 2019 to make sure the operating system is an actual Red Hat release (which Amazon Linux is not). I found some details on the PostgreSQL mailing list:

"Amazon Linux support was removed years ago actually. I just made sure that our repo file reflects that."
(BUG #15768: Removed rpms and now require /etc/redhat-release)

One poster on the mailing list suggested the following fix:

"We have temporarily mitigated the issue by using rpm and explicitly ignoring the repositories dependencies, but that seems like a band-aid fix for the real problem which is that dependency shouldn’t be there."

Personally, I've done the same as you, Scott, I've simply reverted to the PostgreSQL 9.6 client packages that AWS supplies directly. As long as Django and psycopg support that release, this works fine.

However, the long-term fix is for AWS to finally provide platforms with Amazon Linux 2...

Answer 4

Here is my maverick solution for the latest Amazon Linux 2 ami:

sudo su
cd /etc/yum.repos.d
nano pgdg.repo

Then I pasted this into the nano editor with its newly created pgdg.repo file:

[pgdg11]
name=PostgreSQL 11 $releasever - $basearch
baseurl=https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7.5-x86_64
enabled=1
gpgcheck=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-PGDG-11

Further commands are:

sed -i "s/rhel-\$releasever-\$basearch/rhel-7.5-x86_64/g" "/etc/yum.repos.d/pgdg.repo"
yum groupinstall "PostgreSQL Database Server 11 PGDG"

Create a new PostgreSQL database cluster:

/usr/pgsql-11/bin/postgresql-11-setup initdb

Now execute the following command to start and enable the postgresql services:

systemctl enable postgresql-11
systemctl start postgresql-11

Still, keep in mind the notes at this address: Link to Postgresql.org

Answer 5

What did not work for me:

1. Postgre10 and above is not available on Amazon Linux 2.
2. Amazon-linux-extras is not available on (public.ecr.aws/lambda/python:3.11) python3.11 lambda image for Amazon Linux 2. So you cant use this library to install Postgres in python3.11 (python3 more preferably) lambda image for Amazon Linux 2

What worked for me: (installing postgres11 in public.ecr.aws/lambda/python:3.11), but underlying Linux is Amazon Linux 2, so applicable for all instances using AL2 - EC2, Elastic Beanstalk etc.

Dockerfile:

FROM public.ecr.aws/lambda/python:3.11.2023.08.02.09-x86_64
# install dependencies
RUN yum -y groupinstall "Development Tools"
RUN yum -y update
RUN echo -e "\
[pgdg11] \n\
name            = PostgreSQL 11 $releasever - $basearch \n\
baseurl         = https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7.5-x86_64 \n\
enabled         = 1 \n\
gpgcheck        = 0 \n\
" > /etc/yum.repos.d/pgdg.repo

RUN sed "s/rhel-\$releasever-\$basearch/rhel-7.5-x86_64/g" "/etc/yum.repos.d/pgdg.repo"
RUN yum groupinstall "PostgreSQL Database Server 11 PGDG" -y
RUN yum install postgresql11 postgresql11-server -y
COPY systemctl.py /usr/bin/systemctl
RUN chmod a+x /usr/bin/systemctl
RUN systemctl enable postgresql11
RUN systemctl start postgresql11
RUN export PATH=/usr/pgsql-11/bin/psql:$PATH

Now you need to create a file called systemctl.py in the same directory as Dockerfile and copy the content in that from here

Reference: this article