Development environment version alignment can be such a nightmare given the understandable lack of coordination between various moving parts such as OS version, CUDA
version, driver version, package version. What I've found works relatively well is to use conda
to install packages and only if a version is not available there, install using pip
. Your mileage may vary of course and I'm not looking for opinions.
However, with the recent update to CUDA 11.7, a sort of catch-22 has arisen with Pytorch
requiring that one downgrade one's CUDA installation to 11.6. While there are solutions out there for how to do this, it became apparent what I really needed was some way to solve for a set of constraints on all the moving parts, including OS version, CUDA
version, driver version, etc. to create a named environment and then activate them by name in a manner similar to conda activate <name>
.
Of course, one of the advantages of conda
over docker
is that it shares one's files by default so that, say, .bashrc
can set up portions of the development environment in common between conda
environments. This obviously conflicts with the philosophy behind docker
so I'm not hoping for a trivial solution here -- just one that is easier than I've found thus far in the various "how to" instructions on using various docker
images to run systems like pytorch
for development.