Docker-compose does not reflect changes in requirements.txt

Question

Changes in my requirements.txt are not being reflected when I run:

docker-compose -f docker-compose-dev.yml up -d

docker-compose-dev.yml

version: '3.6'

services:

  web:
    build:
      context: ./services/web
      dockerfile: Dockerfile-dev
    volumes:
      - './services/web:/usr/src/app'
    ports:
      - 5001:5000
    environment:
      - FLASK_ENV=development
      - APP_SETTINGS=project.config.DevelopmentConfig
      - DATABASE_URL=postgres://postgres:postgres@web-db:5432/web_dev 
      - DATABASE_TEST_URL=postgres://postgres:postgres@web-db:5432/web_test  
    depends_on:  
      - web-db

  web-db:  
    build:
      context: ./services/web/project/db
      dockerfile: Dockerfile
    ports:
      - 5435:5432
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres

  nginx:
    build:
      context: ./services/nginx
      dockerfile: Dockerfile-dev
    restart: always
    ports:
      - 80:80
    depends_on:
      - web
      - client

  client:
    build:
      context: ./services/client
      dockerfile: Dockerfile-dev
    volumes:
      - './services/client:/usr/src/app'
      - '/usr/src/app/node_modules'
    ports:
      - 3007:3000
    environment:
      - NODE_ENV=development
      - REACT_APP_WEB_SERVICE_URL=${REACT_APP_WEB_SERVICE_URL}
    depends_on:
      - web

Dockerfile-dev

# base image
FROM python:3.6-alpine

# install dependencies
RUN apk update && \
    apk add --virtual build-deps gcc python-dev musl-dev && \
    apk add libffi-dev && \
    apk add postgresql-dev && \
    apk add netcat-openbsd && \
    apk add bind-tools && \
    apk add --update --no-cache g++ libxslt-dev && \
    apk add jpeg-dev zlib-dev

ENV PACKAGES="\
    dumb-init \
    musl \
    libc6-compat \
    linux-headers \
    build-base \
    bash \
    git \
    ca-certificates \
    freetype \
    libgfortran \
    libgcc \
    libstdc++ \
    openblas \
    tcl \
    tk \
    libssl1.0 \
    "

ENV PYTHON_PACKAGES="\
    numpy \
    matplotlib \
    scipy \
    scikit-learn \
    nltk \
    " 

RUN apk add --no-cache --virtual build-dependencies python3 \
    && apk add --virtual build-runtime \
    build-base python3-dev openblas-dev freetype-dev pkgconfig gfortran \
    && ln -s /usr/include/locale.h /usr/include/xlocale.h \
    && python3 -m ensurepip \
    && rm -r /usr/lib/python*/ensurepip \
    && pip3 install --upgrade pip setuptools \
    && ln -sf /usr/bin/python3 /usr/bin/python \
    && ln -sf pip3 /usr/bin/pip \
    && rm -r /root/.cache \
    && pip install --no-cache-dir $PYTHON_PACKAGES \
    && pip3 install 'pandas<0.21.0' \
    && apk del build-runtime \
    && apk add --no-cache --virtual build-dependencies $PACKAGES \
    && rm -rf /var/cache/apk/*

# set working directory
WORKDIR /usr/src/app

# add and install requirements
COPY ./requirements.txt /usr/src/app/requirements.txt  # <--- refer to EDIT
RUN pip install -r requirements.txt

# add entrypoint.sh
COPY ./entrypoint.sh /usr/src/app/entrypoint.sh

RUN chmod +x /usr/src/app/entrypoint.sh

# add app
COPY . /usr/src/app

# run server
CMD ["/usr/src/app/entrypoint.sh"]

what am I missing?

EDIT

Like the accepted answer in [Docker how to run pip requirements.txt only if there was a change?, I'm already copying the requirements.txt file in a separate build step before adding the entire application into the image, but it does not seem to work.

Docker uses layer caching https://semaphoreci.com/docs/docker/docker-layer-caching.html you need to copy your requirements file `COPY` — Yan Khonski, Feb 23 '19 at 19:30
Or alternatively, you can disable cache https://stackoverflow.com/a/31782658/4587961 — Yan Khonski, Feb 23 '19 at 19:32
is this not being already done above in the line `COPY ./requirements.txt /usr/src/app/requirements.txt`? — 8-Bit Borges, Feb 23 '19 at 19:34

score 15 · Accepted Answer · answered Feb 24 '19 at 01:57

I think the problem likely is that $ docker-compose up alone will not rebuild your images if you make changes. In order to get docker-compose to include your changes to your requirements.txt you will need to pass the --build flag to docker-compose.

I.e instead run:

docker-compose -f docker-compose-dev.yml up --build -d

Which will force a docker-compose rebuild the image. However this will rebuild all images in the docker-compose file which may or may not be desired.

If you only want to rebuild the image of a single service you can first run docker-compose -f docker-compose-dev.yml build web, then afterwards just run your original docker-compose command.

More info on the build command here.

Yan Khonski · Answer 2 · 2019-02-23T21:38:28.277

2

Try to install requirements from the copied file

https://docs.docker.com/develop/develop-images/dockerfile_best-practices/

It is an example of their Dockerfile

COPY requirements.txt /tmp/
RUN pip install --requirement /tmp/requirements.txt

This is what you have

RUN pip install -r requirements.txt

Then after you have changed your docker file, you have to stop your container, remove your image, build a new one, and run container from it.

Stop container and remove the image.

docker-compose down
docker-compose --rmi all

--rmi all - removes all images. You might want to use --rmi IMAGE_NAME

And to start it (if you use not default parameters, change these commands with your arguments).

docker-compose up

Update

In case you have running docker and you do not want to stop it and rebuild an image (if you just want to install a package or run some commands or even start a new application), you can connect the container from your local machine and run command line commands.

docker exec -it [CONTAINER_ID] bash

To get [CONTAINER_ID], run

docker ps

Note docker-compose ps will give you containers names, but you need container id to ssh the container.

edited Feb 23 '19 at 21:38

answered Feb 23 '19 at 19:58

Yan Khonski

12,225
15
76
114

could you please add the `$ docker-compose `to your answer in order to run these changes? – 8-Bit Borges Feb 23 '19 at 20:04
hum..do not wish that. is it strictly necessary? – 8-Bit Borges Feb 23 '19 at 20:10
sure, but I was hoping to install only what is inside `requirements.txt`, which is what changes during develpoment, whereas all other core packages, previously installed (all of which take ages to install) remain fixed. – 8-Bit Borges Feb 23 '19 at 20:16
just one last thing. the application above consists of 4 images/containers. is it ok to bash only into the container where the new package is needed? – 8-Bit Borges Feb 23 '19 at 20:45
Yes, you can ssh only one container in your environment. The same thing is about removing, instead of `--rmi all`, you can remove only one image. `--rmi your_image_tag` – Yan Khonski Feb 23 '19 at 21:40
1

I accepted the other answer because it ended up being more useful to my needs, but I upvoted yours and I thank you for your effort. – 8-Bit Borges Feb 24 '19 at 22:45

Docker-compose does not reflect changes in requirements.txt

2 Answers2

Update