2

I am trying to create a requirements.txt to use pytorch but would like it to work on both GPU and non-GPU platforms.

I do something like on my Linux GPU system:

--find-links https://download.pytorch.org/whl/cu113/torch_stable.html

torch==1.10.2+cu113
torchvision==0.11.3+cu113
pytorch-lightning==1.5.10

This works fine and the packages are installed and I can use the GPU-enabled pytorch.

I wonder how I can modify this for the mac and non GPU users to install the non cuda package for torch and torchvision? Do I need to maintain separate requirements.txt files?

Luca
  • 10,458
  • 24
  • 107
  • 234

1 Answers1

5

Check https://pytorch.org/. You will see that "MacOS Binaries dont support CUDA, install from source if CUDA is needed". However, you can still get performance boosts (this will depend on your hardware) by installing the MPS accelerated version of pytorch by:

# MPS acceleration is available on MacOS 12.3+
pip3 install --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cpu

This command can be generated here: https://pytorch.org/

enter image description here

And in order to install different Torch versions on different platorms you can use conditionals in your requirements.txt like this

# for CUDA 11.3 torch on Linux
--find-links https://download.pytorch.org/whl/cu113/torch_stable.html; sys_platform == "linux"
torch==1.10.2; sys_platform == "linux"
torchvision==0.11.3; sys_platform == "linux"
pytorch-lightning==1.5.10; sys_platform == "linux"

# for MPS accelerated torch on Mac
--pre --extra-index-url https://download.pytorch.org/whl/nightly/cpu; sys_platform == "darwin"
torch==1.10.2; sys_platform == "darwin"
torchvision==0.11.3; sys_platform == "darwin"
pytorch-lightning==1.5.10; sys_platform == "darwin"

# for CPU torch on Mac
# torch==1.10.2; sys_platform == "darwin"
# torchvision==0.11.3; sys_platform == "darwin"
# pytorch-lightning==1.5.10; sys_platform == "darwin"

This will install CUDA enabled torch and torchvision on Linux but the MPS accelerated version of them on MacOS

Mike B
  • 2,136
  • 2
  • 12
  • 31
  • I don't think this works out of the box. I'm getting `ERROR: No matching distribution found for torch==1.10.2+cu113` on my M1 mac. – pypae Jul 12 '22 at 09:50
  • 1
    CUDA is not supported on MacOS so you should get rid of +cu113 – Mike B Jul 12 '22 at 10:33
  • Yes, but this is the question...how should one have a standardized requirements file for such cases rather than different ones for each OS. – Luca Jul 13 '22 at 13:13
  • Use conditions in your requirements.txt as in my edit @Luca – Mike B Jul 13 '22 at 13:29
  • '--pre --extra-index-url https://download.pytorch.org/whl/nightly/cpu; sys_platform == "darwin" torch==1.10.2; sys_platform == "darwin" torchvision==0.11.3; sys_platform == "darwin" pytorch-lightning==1.5.10; sys_platform == "darwin"' it worked for me – lam vu Nguyen Jan 13 '23 at 03:49