2

I am trying to reduce the size of my python docker image file using multi-stage. This is my Dockerfile.

Dockerfile

FROM ubuntu:18.04 AS builder

WORKDIR /usr/app

RUN apt-get update \
    && apt-get install -y python3-pip python3-dev python3-venv \
    && pip3 install --upgrade pip \
    && python3 -m venv /opt/venv

ENV PATH="/opt/venv/bin:$PATH"

RUN apt-get update && \
    DEBIAN_FRONTEND=noninteractive \
    apt-get -y install \
        build-essential \
        libboost-all-dev \
        cmake \
        clang \
        libcurl4-gnutls-dev \
        curl \
        gnupg2 \
        libssl-dev \
        zlib1g-dev \
        libbz2-dev \
        libreadline-dev \
        libsqlite3-dev \
        llvm \
        libncurses5-dev \
        libncursesw5-dev \
        xz-utils \
        tk-dev \
        libffi-dev \
        git \
        wget \
        ssh \
        libgcrypt-dev \
        gnutls-dev

ENV CC /usr/bin/clang
ENV CXX /usr/bin/clang++

ARG key
RUN mkdir ~/.ssh && echo "Host github.com\n IdentityFile ~/.ssh/id_rsa\n IdentitiesOnly yes" > ~/.ssh/config && \
    echo "$key" > ~/.ssh/id_rsa && \
    chmod 0600 ~/.ssh/id_rsa && \
    ssh-keyscan github.com >> ~/.ssh/known_hosts

COPY requirements.txt .
RUN pip install -r requirements.txt

FROM python:3.7.3-slim

COPY --from=builder /opt/venv /opt/venv
ENV PYTHONPATH="/opt/venv/lib"
ENV PATH="/opt/venv/bin:$PATH"

WORKDIR /usr/app
COPY . .
RUN ./test.sh && rm -rf test*
USER 10001
CMD PYTHONPATH="${PYTHONPATH}:$(pwd)" python prediction_engine/process.py

In the first stage which python outputs /opt/venv/bin/python When it gets to the second stage which python outputs /usr/local/bin/python

The symlink that gets defined in the first stage for the virtualenv does not get copied over properly. Can anyone please help me figure out why.

I did attempt to use python:3.6-jessie AS builder in the first stage so that both stages use a python image, but many libs are missing in that image and some python library need c++17 to compile. This is the most ideal set-up but the virtualenv doesn't work in the second stage

Nicol Bolas
  • 449,505
  • 63
  • 781
  • 982
Justin
  • 444
  • 4
  • 10
  • 4
    You don't need a virtual environment inside a Docker image. The container itself provides a layer of filesystem isolation from the host. You might consider building your local packages into [wheels](https://packaging.python.org/discussions/wheel-vs-egg/) in the first stage, and then `pip install` those wheels into the "system" Python in the second stage. – David Maze Jun 28 '19 at 13:29
  • Right so I tried that once upon a time and it was a huge failure. Virtualenv approach to multistage builds was the best and most elegant – Justin Jun 28 '19 at 14:14

0 Answers0