Why use requirements.txt in a Docker image

Question

There is a similar question from last year but I don't think the responses are widely applicable and it's not accepted.

This is in the context of developing small jobs that will only be run in Docker in-house; I'm not talking about sharing work with anyone outside a small team, or about projects getting heavy reuse.

What advantage do you see in using requirements.txt to install instead of pip install commands in Dockerfile? I see one: your Dockerfile for various projects is more cookie-cutter.

I'm not even thinking of the use of setup envisioned in the question I linked.

What downside is there to naming the packages in Dockerfile:

 RUN   pip install --target=/build  django==3.0.1 Jinja2==2.11.1 .  . .

@superstormer asked "what are the upsides to putting it in Dockefile?". It is a fair question. I read coworkers' Dockerfiles in GitLab and have to navigate to the requirements. I don't have it locally in an editor. Note to self: so clone it and look at it in an editor.

simple, "better maintenance" , you don't need to go to docker file to see which package to install in docker env at single or multi lvl — sahasrara62, Mar 07 '21 at 01:27
also having a requirements.txt allows you to setup your local dev machine as well as anyone else's who would like to contribute — DevLounge, Mar 07 '21 at 01:29
@sahasrara62, if you're looking at Dockerfile though, you don't need to go to requirements.txt. And requirements.txt you can live without. I don't know what you mean about single/multi level, that might be why I'm not getting the advantage you see. — Levin Magruder, Mar 07 '21 at 01:35
@AnthonyPerot -- Having the requirements in Dockerfile does as well? I guess if someone wanted to make the image not to run in docker, that wouldn't be relevant in the work I do (little scripts that can't be scheduled any way other than Docker) — Levin Magruder, Mar 07 '21 at 01:38
What are the upsides to keeping it in the Dockerfile? However, if you put requirements in reqs.txt, then its much easier to see precisely what the reqs are at a glance. — SuperStormer, Mar 07 '21 at 01:42
@LevinMagruder by single/multi mean, single/multi stage docker env setup. plus let's say i don't want to implement/use Docker in my project then how would i know which packkage to install ? so requirement.txt help us to maintain all packages and install them in any env, production stage or dev or docker one, without modifying/writing different packages/version for different env — sahasrara62, Mar 07 '21 at 01:42
@sahasrara62 thanks, see your point. I claified in my comment I'm looking from perspective of writing scripts that only ever execute in a docker environment, no one would ever want to run them standalone, or they'd be working in a virtualenv where they're set up already. — Levin Magruder, Mar 07 '21 at 02:02
@SuperStormer good question -- I often look at co-workers' dockerfile in a gitlab web page, and I have to use clumsy web navigation to get to the requirements file, often on projects that are set up with the wrong default branch. — Levin Magruder, Mar 07 '21 at 02:05
@LevinMagruder Are you considering embedding pip requirements in all the docker files as a workaround for gitlab web UI clumsiness? — Jerry101, Mar 08 '21 at 20:42
@Jerry101, Yes, that is the context the prompted the question. Additionally, we make a lot of small scripts with little variation in libraries, and I spend more time thinking about docker deployment than script requirements. I'm convinced by your answer and Adam's (and all the commenters) that I'm wrong to object to the practice. Will mark question answered soon. — Levin Magruder, Mar 09 '21 at 02:05

score 3 · Accepted Answer · answered Mar 07 '21 at 02:13

First consider going with the flow of the tools:

To manually install those packages, inside or outside a Docker Container, or to test that it works without building a new Docker Image, do pip install -r requirements.txt. You won't have to copy/paste the list of packages.
To "freeze" on specific versions of the packages to make builds more repeatable, pip freeze will create (or augment) that requirements.txt file for you.
PyCharm will look for a requirements.txt file, let you know if your currently installed packages don't match that specification, help you fix that, show you if updated packages are available, and help you update.

Presumably other modern IDEs do the same, but if you're developing in plain text editors, you can still run a script like this to check the installed packages (this is also handy in a git post-checkout hook):

echo -e "\nRequirements diff (requirements.txt vs current pips):"
diff --ignore-case <(sed 's/ *#.*//;s/^ *--.*//;/^$/d' requirements.txt | sort --ignore-case) \
  <(pip freeze 2>/dev/null | sort --ignore-case) -yB --suppress-common-lines

Hopefully this makes it clearer that requirements.txt declares required packages and usually the package versions. It's more modular and reusable to keep it separate than embed it inside a Dockerfile.

score 1 · Answer 2 · answered Mar 07 '21 at 02:12

It's a question of single responsibility.

Dockerfile's job is to package an application up to be built as an image. That is: it should describe every step needed to turn an application into a container image.

requirements.txt's job is to list every dependency of a Python application, regardless of its deployment strategy. Many Python workflows expect a requirements.txt and know how to add new dependencies while updating that requirements.txt file. Many other workflows can at least interoperate with requirements.txt. None of them know how to auto-populate a Dockerfile.

In short, the application is not complete if it does not include a requirements.txt. Including that information in the Dockerfile is like writing documentation that teaches your operations folks how to pull and install every individual dependency while deploying the application, rather than including it in a dependency manager that packages into the binary you deliver to ops.

Why use requirements.txt in a Docker image

2 Answers2