2

There is syntax used in requirements.txt that allows dependencies from a github project (see here). I want to work with conda and keep requirements file compatible with pip (for others), so I don't want to use yml environment files.

Is there a way to create a file that both pip and conda could install from?

borgr
  • 20,175
  • 6
  • 25
  • 35

2 Answers2

2

Is there a way to create a file that both pip and conda could install from?

No, not generally. This is because Conda additionally manages non-Python dependencies (e.g., dynamic libraries compiled from other languages), which are things that typically are not provided by PyPI. So, even though Conda can output a Pip-like requirements.txt (i.e., with conda list --export), some of the packages are specific to Conda.

As for installing GitHub projects into Conda envs, anything one can do with Pip can be accomplished in a YAML file, e.g., working from the referenced question:

YAML

name: my_env
channels:
 - defaults
dependencies:
 - pip
 - pip:
   - -e git://github.com/mozilla/elasticutils.git#egg=elasticutils

would install a GitHub-hosted package using Pip. See the Advanced Pip Example in the Conda repository, for additional examples.

Using a Pip Requirements File in Conda

One can also use a Pip requirements.txt indirectly in Conda by creating a YAML wrapper for it. For example,

requirements.txt

-e git://github.com/mozilla/elasticutils.git#egg=elasticutils

my_env.yaml

name: my_env
channels:
 - defaults
dependencies:
 - pip
 - pip:
   - -r requirements.txt

which you can use either to create a new env:

conda env create -f my_env.yaml

or to update an existing env:

conda env update -f my_env.yaml

and it will use Pip to install from GitHub.

merv
  • 67,214
  • 13
  • 180
  • 245
  • Just to make it clear, although pip *can* install from a requirement file that has direct git links, there is no way to change the requirements.txt to make also conda understand it, correct? – borgr Jan 23 '20 at 09:13
  • Thanks, it is a solution, I still wonder, isn't that a not recommended solution as it will not try to `conda install ` the requirements and only when not understood pip install them. Instead it uses pip for all the requirements and not conda, even when available. – borgr Jan 30 '20 at 09:26
  • @borgr that's right, it is recommended to install as much as you can from Conda first. You could go through the `requirements.txt` and add whatever is definitely available through Conda as a direct dependency in the YAML. Conda will install those first and when the pip section runs, it will recognize they are already there – merv Jan 30 '20 at 15:10
  • any way to do that automatically in the yaml? (i.e. without creating a shell script) – borgr Feb 02 '20 at 07:21
  • @borgr coming back to this, my opinion has evolved a bit: if one has a pure Python environment fully defined by the requirements file, I would only use pip-installed packages, rather than trying to mix in what is available through Conda. You can still manage environment activation with Conda, but subsequent changes to packages should be done through Pip. – merv Aug 01 '21 at 04:29
0

for this add -e in front of the link and add the link normally.

-e https://github.com/something.git

like this you have to add .

the requirements file can be used for both pip and conda

in pip

   pip install -r requirements.txt

in conda

  conda install --yes --file  requirements.txt
i_am_deesh
  • 448
  • 3
  • 12
  • Could the link be git+ssh:git@github.com...? because my format was not html format and conda failed to understand it (despite the -e notation) – borgr Jan 23 '20 at 09:11