91

When I run pip freeze > requirements.txt it seems to include all installed packages. This appears to be the documented behavior.

I have, however, done something wrong as this now includes things like Django in projects that have no business with Django.

How do I get requirements for just this project? or in the future how do I install a package with pip to be used for this project. I think I missed something about a virtualenv.

Marshall Davis
  • 3,337
  • 5
  • 39
  • 47
  • 8
    `pip` has no way to understand, that your project have no business with, say, Django. So generally you want to create an isolated virtual environment without access to system site-packages and run pip with `--isolated` flag to ignore stuff in `PYTHONPATH`. And of course you should not install packages into that environment other than required by your project – Konstantin Sep 04 '15 at 05:07
  • 1
    @Alik Alright, that's what I was afraid of. I'll look into creating the virtual environments later. If you want to submit this as an actual answer, I'll accept it. – Marshall Davis Sep 04 '15 at 05:58

12 Answers12

143

pipreqs can save the day for a specific project. Just

pip install pipreqs
#then
pipreqs path/to/project

Github Page

Smart Manoj
  • 5,230
  • 4
  • 34
  • 59
unlockme
  • 3,897
  • 3
  • 28
  • 42
  • 3
    I might not be understanding this correctly, but doesn't this confuse install_requires and requirement.txt? requirement.txt are intended to be complete and exhaustive, where as install_requires are intended to be minimal. https://packaging.python.org/discussions/install-requires-vs-requirements/ – onesiumus Aug 01 '19 at 20:42
  • @Daniel Maksimovich, I think you mean to imply that the command will not output all the `desired - pinned` packages as should be in requirements.txt. This is true because the package `sniffs` the requirements based on imports in the files located in the target directory. But no it does not confuse the two. In any case, the `install requires`, will ensure that re-using the generated requirements.txt will install all requirements which may have been missed. – unlockme Oct 22 '20 at 11:15
  • this produced a file with none of the packages that were in the python file, like `google-cloud-storage` – Zimano Nov 19 '20 at 15:44
30

I have tried both pipreqs and pigar and found pigar is better because it also generates information about where it is used, it also has more options.

ishandutta2007
  • 16,676
  • 16
  • 93
  • 129
15

I use this command

EDIT: Thanks Addisson Klinke for suggestion

pip freeze -r requirements.txt | grep -B100 "pip freeze" | grep -v "pip freeze"

pip freeze -r requirements.txt | sed '/freeze/,$ d'

When I ran pip freeze -r requirements.txt the output is something like

APScheduler==3.2.0
Eve==0.6.4
Eve-Elastic==0.3.8
## The following requirements were added by pip freeze:
arrow==0.8.0
Cerberus==0.9.2

I have a requirements file like this

APScheduler
Eve
Eve-Elastic

So I get this output and sed to remove the dependencies that I don`t want.

First output this to a file

pip freeze -q -r requirements.txt | sed '/freeze/,$ d' > requirements-froze.txt

That will output just the libs with version

APScheduler==3.2.0
Eve==0.6.4
Eve-Elastic==0.3.8

Then replace requirements file

mv requirements-froze.txt requirements.txt
LuisComS
  • 452
  • 4
  • 20
  • I'll have to look into this, but I believe all the requirements were added by `pip freeze`, this would mean that the line being searched for would be the first. – Marshall Davis Oct 24 '16 at 13:56
  • 1
    Thanks for sharing the `pip freeze -r` option - didn't know that! I think the `grep` pipes are somewhat verbose/fragile though - a better approach is probably `pip freeze -r requirements.txt | sed '/freeze/,$ d`. That deletes the freeze line and all following lines to the end of the list – Addison Klinke May 14 '21 at 15:43
  • 1
    this breaks if you have something like `some-package[some-extra]` as the extra package is not included in the about after sed. – The Fool Apr 01 '22 at 15:02
11

I still suggest using the official pip freeze > requirements.txt (documentation) compared to the two alternative features using pigar and pipreqs mentioned in the other answers because pip freeze lists the effective package names.

Incomplete comparison among the three as per February 2022

Criteria \ Tool pip freeze > requirements.txt pigar pipreqs
Name mismatch (1) Package my-package==1.0.0 Module my_package == 1.0.0 Module my_package==1.0.0
Module overloading (2) All packages my-package1==1.0.0, my-package2==2.0.0 None Top-level module (version shows 0.0.0) my==0.0.0
Showing only directly used packages No Yes Yes
Minimal contents Yes No Yes

(1) There can be a mismatch between module and package name such as my-package (package name) vs my_package (module name).

(2) There can be several packages using the same top level folder such as my-package1 and my-package2 (package names) which are installed under my/package1 and my/package2, which are imported by Python's command import my.package1 and import my.package2. Note that pipreqs notes version 0.0.0 for the not existing package my.

I know these are very particular cases but I hope giving people this overview can help understanding limitations and avoid possible mistakes.

notfancy
  • 157
  • 1
  • 7
6
pip install pipreqs 
pipreqs>requirements.txt

That works easily

swateek
  • 6,735
  • 8
  • 34
  • 48
Victor Ashioya
  • 61
  • 1
  • 5
2

Here is a simplified version based on the previous comment: https://stackoverflow.com/a/40026169/4052041

mv requirements.txt requirements.txt.bak
pip freeze -q -r requirements.txt.bak | awk '/.*pip freeze.*/  {exit} {print}' > requirements.txt
jeremie
  • 81
  • 1
  • 2
1

if you are using linux then do it with sed

pip freeze | sed 's/==.*$/''/' > requirements.txt

duylamvo
  • 179
  • 5
1

I had the same issue with pip freeze. In my case the problem was that I ran pip freeze without activating my projects virtual environment. I activated the virtual environment and pip freeze > requirements.txt worked fine.

So do make sure you activate your projects virtual environment by running <virtualenv folder name>\Scipts\activate on Windows or source <virtualenv folder name>\bin\activate on Linux.

If the virtualenv has global access you should run pip freeze with the -l or --local option, pip freeze -l which according to the pip freeze docs

-l, --local
If in a virtualenv that has global access, do not output globally-installed packages.

vaylon fernandes
  • 11
  • 3
1

With your virtualenv activated, do python -E -m pip freeze. The -E flag means, according to python --help, ignore PYTHON* environment variables (such as PYTHONPATH), so pip it's not going to access to global site-packages.

Guillermo Z
  • 38
  • 1
  • 5
1

There is a package that works for me in windows and Lunix Pipdeptree After the installation you can use Pipdeptree to generate requirements.txt file with just top-level packages.

Windows:

pipdeptree -f --warn silence | findstr  /r  "^[a-zA-Z0-9\-]" > requirements.txt

Lunix:

pipdeptree -f --warn silence | grep -E '^[a-zA-Z0-9\-]+' > requirements.txt
Mohssine SERRAJI
  • 21
  • 2
0

I just had the same issue, here's what I've found to solve the problem.

First create the venv in the directory of your project, then activate it.

For Linux/MacOS : python3 -m venv ./venv source myvenv/bin/activate

For Windows : python3 -m venv .\venv env\Scripts\activate.bat

Now pip freeze > requirements.txt should only takes the library used in the project.

NB: If you have already begin your project you will have to reinstall all the library to have them in pip freeze.

Severin Robert
  • 13
  • 2
0

I do came across same situation. After activating the virtual enviroment and doing

pip3 freeze > requirements.txt

still its collect all package install on my WSL ubuntu 22.04

After install "pipreqs" and typing pipreqs>requirments.txt My Wsl got stuck.

The best solution is pip3 freeze -l > requirments.txt its only collected my local package used for my development.

Mrsreez
  • 77
  • 12