Load Environment Modules into Jupyter Notebook/Lab

Question

I am running JupyterLab through a server (which is managed remotely, I am a user without sudo access). I want to use CUDA but in order to do so I have to load it in as a module using module load bash command when in terminal.

The CUDA package is already installed, so I'm presuming that module load merely adds paths that point to where the package is. I cannot use !module load cuda in Jupyter, so I cannot use CUDA at all. CUDA installation also requires sudo access.

I've tried using %env to add environment variables but beyond that I am really stuck.

Is there a way I can use module load or properly point to where the package is, because the Python library I'm using unfortunately keeps throwing this error when trying to import it

libcudart.so.9.2: cannot open shared object file: No such file or directory

All help appreciated

@DivyanshuSrivastava It modifies the shell Environment Variables, particularly the `LD_LIBRARY_PATH` to point to the CUDA package. It seems to be a command that is mostly used in computing clusters, like Universities. It may not be a command that many are familiar with. — Justin Cunningham, Aug 23 '20 at 14:21
Could you share the output of the 2 following commands when run in Jupyter : `module load cuda` and `module avail` — Xavier Delaruelle, Aug 24 '20 at 18:18
@XavierDelaruelle Jupyterlab can't find the `module` command, so all it says is `/bin/bash: module: command not found` — Justin Cunningham, Aug 24 '20 at 21:15

score 1 · Accepted Answer · answered Aug 25 '20 at 03:47

It seems module command is not recognized in the environment you try to use. So first you need to define this shell function by sourcing module initialization script. Such script can be found in /usr/share/Modules/init on Red Hat like systems.

After running:

source /usr/share/Modules/init/bash

You should get the module shell function defined, then doing a module avail will tell you if a cuda modulefile is available for load with module load cuda.

Load Environment Modules into Jupyter Notebook/Lab

1 Answers1