1

I am getting the below error trying to deploy my app to gcloud app engine. The whole problem is from trying to add the GDAL library to my app.

File "/opt/python3.7/lib/python3.7/ctypes/init.py", line 377, in getattr func = self.getitem(name) File "/opt/python3.7/lib/python3.7/ctypes/init.py", line 382, in getitem func = self._FuncPtr((name_or_ordinal, self)) AttributeError: /usr/lib/libgdal.so.1: undefined symbol: OGR_F_GetFieldAsInteger64

I followed all the directions I could possibly find online. But nothing seems to work at all. Here is my app.yml files:

runtime: custom

entrypoint: gunicorn -b :8080 app.wsgi
env: flex 

# any environment variables you want to pass to your application.
# accessible through os.environ['VARIABLE_NAME']
env_variables:
 ... 
  
beta_settings:
  cloud_sql_instances: site:asia-northeast2:site-db

handlers:
- url: /.*
  script: auto
  secure: always
manual_scaling: 
  instances: 1

runtime_config:
  python_version: 3

And Dockerfile:

FROM gcr.io/google-appengine/python
#FROM python:3.7
#FROM python:3.8.0-slim-buster

EXPOSE 8080
ENV PYTHONUNBUFFERED 1

# Install GDAL dependencies
#RUN apt-get update && apt-get install --yes libgdal-dev
RUN apt-get update && apt-get install --reinstall -y \
  #libopenjp2-7 \
  #libopenjp2-7-dev \
  #libgdal-dev \
  binutils \
  gdal-bin \
  python-gdal \
  python3-gdal 


# Update C env vars so compiler can find gdal
#ENV CPLUS_INCLUDE_PATH=/usr/include/gdal
#ENV C_INCLUDE_PATH=/usr/include/gdal
RUN export CPLUS_INCLUDE_PATH=/usr/include/gdal
RUN export C_INCLUDE_PATH=/usr/include/gdal

# Create a virtualenv for dependencies. This isolates these packages from
# system-level packages.
# Use -p python3 or -p python3.7 to select python version. Default is version 2.
RUN virtualenv /env -p python3

# Setting these environment variables are the same as running
# source /env/bin/activate.
ENV VIRTUAL_ENV /env
ENV PATH /env/bin:$PATH

# Copy the application's requirements.txt and run pip to install all
# dependencies into the virtualenv.
ADD requirements.txt /app/requirements.txt
RUN pip install -r /app/requirements.txt
# Add the application source code.
ADD . /app

# Run a WSGI server to serve the application. gunicorn must be declared as
# a dependency in requirements.txt.
CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 main:app --timeout 0 --preload

Update: As most suggestions online of using Python3.8 image.

FROM python:3.8

When I try that, I get the following error:

/bin/sh: 1: virtualenv: not found The command '/bin/sh -c virtualenv /env -p python3' returned a non-zero code: 127 ERROR ERROR: build step 0 "gcr.io/cloud-builders/docker" failed: step exited with non-zero status: 127

Someone help me here. What is it I need to do to get this working?

LearnToday
  • 111
  • 2
  • First search the web for that error, then try the solutions indicated. – Michael Hampton Aug 22 '20 at 14:46
  • I've been searching since yesterday. No suggestion has worked. – LearnToday Aug 22 '20 at 14:59
  • There's nothing in your question about the things you have tried! – Michael Hampton Aug 22 '20 at 15:00
  • The talk online has been mostly about the Google python image version. Seems GDAL upgraded and version 3.8 pf python should be suitable for GDAL. Unfortunately I can't get any VM with Python 3.8. I didn't think I had to add all this because tried so many other recommendations. Do you need me to update the question? – LearnToday Aug 22 '20 at 15:05
  • if you look at the question code, I even commented out other Gcloud python images I tried ```#FROM python:3.7 #FROM python:3.8.0-slim-buster``` – LearnToday Aug 22 '20 at 15:07
  • I found this [Stackoverflow](https://stackoverflow.com/questions/61412885/app-engine-flexible-docker-file-fails-to-install-gdal) post, maybe this one could helps – Andie Vanille Aug 25 '20 at 15:34

1 Answers1

0

GDAL dependencies with GAE can be pretty painful. Also, I've noticed the GAE image published for the flex environment does not include python3.8 unlike the standard env which is supposed to include 3.8 (it goes all the way up to python3.7 for flex, you can see them all in /opt/python3.*) So, you should install 3.8 yourself plus a few necessary GIS dependencies.

Give this Dockerfile a try:

FROM gcr.io/google-appengine/python
ENV PYTHONUNBUFFERED 1
ENV DEBIAN_FRONTEND noninteractive

RUN apt -y update && apt -y upgrade\
    && apt-get install -y software-properties-common \
    && add-apt-repository -y ppa:ubuntugis/ppa \
    && add-apt-repository -y ppa:deadsnakes/ppa \
    && apt -y update && apt -y upgrade\
    && apt-get -y install python3.8 python3.8-distutils python3.8-venv \
       gdal-bin libgdal-dev python3-gdal  \ 
    && apt-get autoremove -y \
    && apt-get autoclean -y \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

RUN virtualenv /env -p python3.8
ENV VIRTUAL_ENV /env
ENV PATH /env/bin:$PATH

ADD requirements.txt /app/requirements.txt
RUN python3 -m pip install -r /app/requirements.txt 
ADD . /app/
WORKDIR /app
CMD gunicorn -b :$PORT app.wsgi

This uses the Deadsnakes PPA to install the python3.8 binary & co. in the container as well as the newer GDAL/GIS dependencies as GAE's container is based off Ubuntu 16.04 LTS (xenial) which comes with some rather dated packages. Note that you don't need to set the compiler flags because you are not compiling anything in your container.

Hendrik F
  • 101