Docker build taking too long when installing grpcio via pip

Question

I have a Dockerfile which installs a few packages via pip. Some of them are requiring grpcio, and it takes a few minutes only to build this part.

Does anyone have a tip to speed up this part?

Installing collected packages: python-dateutil, azure-common, azure-nspkg, azure-storage, jmespath, docutils, botocore, s3transfer, boto3, smmap2, gitdb2, GitPython, grpcio, protobuf, googleapis-common-protos, grpc-google-iam-v1, pytz, google-api-core, google-cloud-pubsub
Found existing installation: python-dateutil 2.7.3
  Uninstalling python-dateutil-2.7.3:
    Successfully uninstalled python-dateutil-2.7.3
Running setup.py install for grpcio: started
Running setup.py install for grpcio: still running...
Running setup.py install for grpcio: still running...
Running setup.py install for grpcio: still running...

Thanks.

What docker image are you using, `alpine`? Also, what is the target Python version in the image? — hoefling, Oct 20 '18 at 10:43
please use the cache and you dont need to build thoese layers everytime , put new things in the end of dockerfile — Ijaz Ahmad, Oct 20 '18 at 12:55
Having the same problem building the wheel for grpcio, see code here: https://github.com/ohduran/Promenade/blob/master/instarest/compose/local/django/Dockerfile — ohduran, Mar 20 '19 at 07:54
You're right @Hyperbole, new link: https://github.com/ohduran/Promenade/blob/master/compose/local/django/Dockerfile — ohduran, Mar 31 '19 at 10:44
If you are using alpine, it takes time to compile from scratch https://github.com/grpc/grpc/issues/22815#issuecomment-649690837 — Sachin G., Oct 30 '21 at 12:22

score 39 · Answer 1 · answered Aug 24 '20 at 16:55

39

I had the same issue and it was solved by upgrading pip:

$ pip3 install --upgrade pip

Here's a word from one of the maintainers of grpc project:

pip grpcio install is (still) very slow #22815

answered Aug 24 '20 at 16:55

ivanleoncz

9,070
7
57
49

4

you made my day! – Uvuvwevwevwe Nov 11 '20 at 22:17
Thanks bro. Indeed, upgrading pip is essential for getting rid of this issue. Have no idea of what might be going wrong with grpcio and Docker. – ivanleoncz Nov 18 '20 at 21:06
1

still no aarch64 (armv8) wheels for grpcio, so this doesn't work on most newer arm processers – toing_toing Dec 14 '20 at 20:52
why isn't this marked as the answer?? – Anirudh Goel Nov 06 '21 at 00:49
You mean upgrade pip on the host or in the Dockerfile like `RUN pip3 install --upgrade pip`? – Oleg Yablokov Jan 26 '22 at 09:20
That depends, Oleg. If your `python3-pip` installed on the container is outdated, you might consider RUNning a command for upgrading pip. I'd say that, having the latest version of `pip` would do the trick. Worth testing. – ivanleoncz Jan 26 '22 at 13:42

score 11 · Answer 2 · answered Jun 04 '19 at 15:32

Had the same issue, fixed it by using a virtualenv and a multistage dockerfile :

FROM python:3.7-slim as base

# ---- compile image -----------------------------------------------
FROM base AS compile-image
RUN apt-get update \
  && apt-get install -y --no-install-recommends \
  build-essential \
  gcc

RUN python -m venv /app/env
ENV PATH="/app/env/bin:$PATH"

COPY requirements.txt .
RUN pip install --upgrade pip
# pip install is fast here (while slow without the venv) :
RUN pip install -r requirements.txt

# ---- build image -----------------------------------------------
FROM base AS build-image
COPY --from=compile-image /app/env /app/env

# Make sure we use the virtualenv:
ENV PATH="/app/env/bin:$PATH"
COPY . /app
WORKDIR /app

Here is my requirements.txt :

fastapi==0.27.*
grpcio-tools==1.21.*
uvicorn==0.7.*

I am curious why virtualenv would make a difference here. – Alexei Masterov Sep 12 '20 at 14:33 — Alexei Masterov, Sep 12 '20 at 14:33

Gleno · Answer 3 · 2022-01-18T11:35:09.837

Some docker images (looking at you, Alpine) can't pull prebuilt wheels. Use a docker image that can, like Debian.

Check out this nice write-up on why. I'll reproduce a quote from it, that's especially apt:

But for Python, as Alpine doesn't use the standard tooling used for building Python extensions, when installing packages, in many cases Python (pip) won't find a precompiled installable package (a "wheel") for Alpine. And after debugging lots of strange errors you will realize that you have to install a lot of extra tooling and build a lot of dependencies just to use some of these common Python packages. -- Sebastián Ramírez

Docker build taking too long when installing grpcio via pip

3 Answers3