10

I use Python 3.6.6 :: Anaconda, Inc.

I try to use subprocess to call other python script.

subprocess.run("python -V", shell=True)

I tried this code but outcome is

Python 2.7.12

My local python get caught

I also tried

subprocess.run("bash -c 'source /home/hclee/anaconda3/envs/calamari/lib/python3.6/venv/scripts/common/activate
/home/hclee/anaconda3/envs/calamari && python -V && source deactivate'", shell=True)

but I got the same result

Cœur
  • 37,241
  • 25
  • 195
  • 267
hochan
  • 103
  • 1
  • 1
  • 5
  • 1
    you default python alias is set to the system python, which is Python 2.7.12, if you want to change your default `python` to 3.6.6, you would have to change the python alias. – user2906838 Aug 13 '18 at 10:01
  • Anaconda has by default python 2.7.12 but in also includes python 3.6.6. If you want to choos python 3 then you may create and environment like `conda create -n myenv python=3.6` and then, to use the environment you need to "activate" it. type `conda activate myenv`. Then if you execute `python -V` you should get python 3.6.6 When you are finish you may use `conda deactivate' to exit the environment. – jalazbe Aug 13 '18 at 10:33
  • can you post the output of `env` and `subprocess.run("env")`? – Nils Werner Aug 13 '18 at 10:44
  • 1
    @jalazbe that statement is not correct. It depends on which version of Anaconda you install. As you can see [here](https://www.anaconda.com/download/) there is one download with a default python3 version and a different download with a default python2 version – FlyingTeller Aug 13 '18 at 10:54
  • @FlyingTeller thanks for the information. You are right. It depends on the installation package selected. – jalazbe Aug 13 '18 at 10:59
  • @NilsWerner My output of subprocess.run("env") PYTHONPATH=/home/hclee/calamari PYTHONIOENCODING=UTF-8 – hochan Aug 17 '18 at 01:06

4 Answers4

11

Run source activate root in linux, or activate root in Windows to activate the environment before running your code.

If this does not help you, try for a quick and dirty fix e.g.:

subprocess.run('bash -c "source activate root; python -V"', shell=True)

The reason you need to call bash is that python's run will not call the bash environment, but another which is a bit more constrained and does not contain source, so here we need to call bash... But as mentioned, if you need this command, either you are doing something special, or something is wrong with your environment...

deactivate is not needed, it does nothing cause the shell it was run on will be destroyed...

Note: For newest conda versions, the provided code will work, but there are also these options that work similarly:

conda deactivate:

subprocess.run('bash -c "conda deactivate; python -V"', shell=True)

conda activate root or base:

subprocess.run('bash -c "conda activate root; python -V"', shell=True)
subprocess.run('bash -c "conda activate base; python -V"', shell=True)
ntg
  • 12,950
  • 7
  • 74
  • 95
  • I beleive that `conda` basic environment (if none are created) is called `base`. I just checked that you can either use `conda activate env` or `conda activate root` or `source path-to-conda/env-name/bin/activate` – jalazbe Aug 13 '18 at 10:35
  • I am running linux mint (Ubuntu), so can only speak for it, but just tried running it and it accepted 'root' but did not recognize 'base'. Also, I could not do conda activate: https://i.stack.imgur.com/CK4UN.png – ntg Aug 13 '18 at 11:20
  • 1
    what version of conda do you have? The changes mentioned by @jalazbe occurred in conda 4.4. – darthbith Aug 13 '18 at 15:07
  • It works in my terminal but It didn't work my pycharm. Other modules that I import is from my anaconda env. But subprocess is from "/pycharm-community-2018.1.4/helpers/typeshed/stdlib/3" Idk what's going on here – hochan Aug 14 '18 at 01:36
  • @darthbith my conda version is 4.5.4 – hochan Aug 14 '18 at 01:37
  • @hochan: Did you set up the correct interpreter in pycharm? https://stackoverflow.com/questions/19679150/how-to-set-default-pycharm-interpreter – ntg Aug 14 '18 at 13:17
  • @ntg I check my interpreter in pycharm. I use conda env. I check my tenosrflow version, The version info printed that I installed. It's weird... – hochan Aug 17 '18 at 02:27
  • @hochan :S weird... one thing to check is what print(sys.executable) outputs – ntg Aug 22 '18 at 07:36
  • @ntg I check in my terminal and pycharm both. But print(sys.executable) outputs are the same. – hochan Aug 23 '18 at 09:49
  • I had the same problem like @hochan in terminal no problem to activate the environment but it fails in pycharm. So one solution could be to make the correct configuration in the Run/Debug Configurations window - any ideas what to enter here? I tried to set python=conda_python_path but although it is set it correctly by os.environ() , finally the subprocess() command didn't pick it up – Agile Bean Oct 06 '18 at 08:21
  • Note that pycharm has options on which conda env to run... (check properties etc) There is one property for running environment, another for the console... The configuration has also a hierarchy (you can define default, then project specifics, then run specifics...) But this is another (few) question(s) ... Search for the pycharm question(s) that relate to your needs... – ntg Dec 05 '20 at 02:53
4

I don't think sourcing a conda env in every subprocess call of your code is a good idea.

Instead you can find the bin dir of your current sourced env and grab the full path to binaries from there. Then pass these to subprocess when you want to call them. 

import os
import sys
import subprocess

# what conda env am I in (e.g., where is my Python process from)?
ENVBIN = sys.exec_prefix

# what binaries am I looking for that are installed in this env?
BIN1 = os.path.join(ENVBIN, "bin", "ipython")
BIN2 = os.path.join(ENVBIN, "bin", "python")
BIN3 = os.path.join(ENVBIN, "bin", "aws")

# let's make sure they exist, no typos.
for bin in (BIN1, BIN2, BIN3):
    assert os.path.exists(bin), "missing binary {} in env {}".format(bin, ENVBIN)

# then use their full paths when making subprocess calls
for bin in (BIN1, BIN2, BIN3):
    cmd = ["which", bin]
    proc = subprocess.Popen(cmd, stdout=subprocess.PIPE)
    print(proc.communicate()[0].decode())

The printed results show that subprocess is using the binaries from the conda environment instead of from the base (default) env. 

.../miniconda3/envs/aws/bin/ipython
.../miniconda3/envs/aws/bin/python
.../miniconda3/envs/aws/bin/aws
Flower-doctor
  • 73
  • 6
0

If your console/notebook is already using the correct environment, you can call subprocess with sys.executable to use current environment:

import sys
import subprocess
subprocess.run(f"{sys.executable} -V", shell=True)
ronkov
  • 1,263
  • 9
  • 14
  • sys.executable will give only the current folder where jupyter runed not the env_path as desribed in https://stackoverflow.com/questions/36539623/how-do-i-find-the-name-of-the-conda-environment-in-which-my-code-is-running – MosQuan Oct 29 '22 at 02:18
  • @MosQuan `sys.executable` returns the full path to the python.exe in the current environment, so you ensure that you're calling python with the correct version. In the example you link they split sys.executable to get the folder – ronkov Nov 08 '22 at 10:29
0

The above solutions didn't work for me. What did work is:

import subprocess
output = subprocess.run('''python -c "import sys; print(sys.executable)"
                  source <your-conda-base-dir>/etc/profile.d/conda.sh
                  conda activate <name-of-the-conda-environment-you-want-to-activate>
                  python -c "import sys; print(sys.executable)"
                  conda env export''', executable='/bin/bash', shell=True, capture_output=True)
print(bytes.decode(output.stdout))

Substitute <your-conda-base-dir> by the output of which conda executed in a the interactive terminal you normally use for executing conda commands, but remove /condabin/conda from the end of the output you get, such that the /etc/profile.d/conda.sh location will be found when the above subprocess.run() is executing.

Sander Vanden Hautte
  • 2,138
  • 3
  • 22
  • 36