2

Is it possible at all to launch a Python script using conda environment ENV1 and at some point within the script to switch to environment ENV2 and the code that follows that point to be executed within ENV2 instead of ENV1? I have tried the following suggested solution but it doesn't work:

https://unix.stackexchange.com/questions/622383/subprocess-activate-conda-environment-from-python-script?newreg=191cf527472141d2a76a244969897af8

Below is an example script. Assuming that I launch the script while having ENV1 as my active environment:

import subprocess

print("Changing Conda virtual environment to 'ENV2'.")
cmd = '. $CONDA_PREFIX_1/etc/profile.d/conda.sh && conda activate ENV2 && echo $CONDA_PREFIX'
subprocess.call(cmd, shell=True, executable='/bin/bash')
print(os.environ['CONDA_PREFIX'])

The only viable solution I could think of is to save all code that occurs after "subprocess.call(cmd, shell=True, executable='/bin/bash')" into a separated script named "script_for_ENV2.py" and replace the above script with this:

import subprocess

cmd = 'conda run -n ENV2 script_for_ENV2.py'
subprocess.call(cmd, shell=True, executable='/bin/bash')
tevang
  • 518
  • 1
  • 4
  • 17
  • 1
    This seems like an [XY Problem](https://xyproblem.info/). If you explain why you want to do this, maybe we can suggest a better solution to your original problem. – Code-Apprentice Mar 11 '22 at 22:41

2 Answers2

0

Is it possible at all to launch a Python script using conda environment ENV1 and at some point within the script to switch to environment ENV2 and the code that follows that point to be executed within ENV2 instead of ENV1?

No, this isn't possible, at least not at a practical level, since launching a python script creates a process running in the underlying operating system.

One possibility is that you can launch another python script using whatever conda environment you want. But why do you want to do this?

Code-Apprentice
  • 81,660
  • 23
  • 145
  • 268
  • Briefly, I created a software that calls various individual codes (neural networks), each one developed using various versions of TensorFlow, PyTorch, numpy, etc. So there are a lot of version conflicts that render execution of my software from a single conda environment impossible. That's why I thought that the simplest solution would be to create an individual conda environment for each of these neural networks, switch to the appropriate environment whenever I launch the NN, retrieve the output and then switch back to the initial environment. Is it clear enough now? – tevang Mar 11 '22 at 23:00
  • 1
    @tevang Ideally, I think you should unify all of the python cod to use a single set of dependencies. I understand that can potentially be a large undertaking, though. To run a python script in a separate conda environment, you will need to fork a new process for it rather than calling the python code directly. – Code-Apprentice Mar 12 '22 at 00:10
  • what you describe is practically impossible. I have been struggling with that problem for years. Either I will focus on developing my own software or updating other people's codes every time a new version of TensorFlow, Keras, PyTorch, numpy etc. comes out. Forking the external codes, saving the output into files, and loading it to process it sounds OK. – tevang Mar 12 '22 at 08:51
0

I've tried looking into this... as I had a similar problem. Here's what I've found.

  1. using os.system("activate < env-name >") and any subprocess or system command after seems to performed in that env.

  2. This is the path I went with since I wanted to visually see the commands being done in that environment (performed on windows from python script) Maybe some sort of this method could be incorporated on linux? I'm not too sure about this but I will test it out and add to this if it was possible or if I find another solution. Nonetheless, using this for loop I was able to run/append as many commands on the separate window/environment. One can also replace the "cmd /k" with "cmd /c" to hide the command prompt. I hope one of these methods helps or provides you with a path. best, -G

    my_env=[]
cmdStr=""
my_env.append("echo \" Running Command \"")
my_env.append("activate <env-path or name>")
my_env.append(" echo \" do stuff on this env\"")
for ii in my_env:
    cmdStr = cmdStr + ii + " ^&^& "
os.system("start " + cmdStr + "cmd /k")
Everydaycoder
  • 1
  • 1