2

reticulate is an R package to call Python code from R, that I find easy to use on my local computer and I sometimes get working within a Singularity container.

In this question I will post a simple reprex about code that worked two months ago, at https://github.com/richelbilderbeek/reticulate_on_singularity and I hope you can help to get it to work again.

Here is the part of the Singularity script (the full script can be found here, so one can confirm I use -for example- a fresh Miniconda install) that creates the Conda environment in the /opt/ormr folder (Singularity recommends to use a system folder, hence the opt, ormr is Swedish for snake), which works fine:

%post
    # ...
    Rscript -e 'reticulate::conda_create(envname = "/opt/ormr")'
    Rscript -e 'reticulate::use_condaenv(condaenv = "/opt/ormr")'
    Rscript -e 'reticulate::use_python(python = reticulate:::python_binary_path("/opt/ormr"), required = TRUE)'
    Rscript -e 'reticulate::conda_install(packages = "scipy", envname = "/opt/ormr")'
    Rscript -e 'reticulate:::conda_list_packages(envname = "/opt/ormr")'

However, in the test section of the Singularity script, this goes wrong, probably because the section above (the post section) has full access rights, where test does not have this:

%test
    # ...
    Rscript -e 'reticulate::use_condaenv(condaenv = "/opt/ormr")'

The -I think- relevant part of the error message is shown below (copied from this GitHub Actions log and full error message is at the bottom of this post):

    OSError: [Errno 30] Read-only file system: '/opt/ormr/.tmphx9hr3g2'

`$ /miniconda/bin/conda run --prefix /opt/ormr --no-capture-output python -c import os; print(os.environ['PATH'])`

I would conclude that Conda creates a temporary file in the now-read-only folder /opt/ormr.

How need I fix this problem?

Or, in general, how can use reticulate from within a Singularity container? A working example would be great (I have found none that work)!

Thanks, Richel Bilderbeek

Full error message

# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "/miniconda/lib/python3.9/site-packages/conda/exceptions.py", line 1080, in __call__
        return func(*args, **kwargs)
      File "/miniconda/lib/python3.9/site-packages/conda/cli/main.py", line 84, in _main
        exit_code = do_call(args, p)
      File "/miniconda/lib/python3.9/site-packages/conda/cli/conda_argparse.py", line 83, in do_call
        return getattr(module, func_name)(args, parser)
      File "/miniconda/lib/python3.9/site-packages/conda/cli/main_run.py", line 25, in execute
        script_caller, command_args = wrap_subprocess_call(on_win, context.root_prefix, prefix,
      File "/miniconda/lib/python3.9/site-packages/conda/utils.py", line 403, in wrap_subprocess_call
        with Utf8NamedTemporaryFile(mode='w', prefix=tmp_prefix, delete=False) as fh:
      File "/miniconda/lib/python3.9/site-packages/conda/auxlib/compat.py", line 88, in Utf8NamedTemporaryFile
        return NamedTemporaryFile(
      File "/miniconda/lib/python3.9/tempfile.py", line 541, in NamedTemporaryFile
        (fd, name) = _mkstemp_inner(dir, prefix, suffix, flags, output_type)
      File "/miniconda/lib/python3.9/tempfile.py", line 251, in _mkstemp_inner
        fd = _os.open(file, flags, 0o600)
    OSError: [Errno 30] Read-only file system: '/opt/ormr/.tmphx9hr3g2'

`$ /miniconda/bin/conda run --prefix /opt/ormr --no-capture-output python -c import os; print(os.environ['PATH'])`

  environment variables:
                 CIO_TEST=<not set>
               CONDA_ROOT=/miniconda
           CURL_CA_BUNDLE=<not set>
        GITHUB_EVENT_PATH=/github/workflow/event.json
              GITHUB_PATH=/__w/_temp/_runner_file_commands/add_path_10663761-8f12-43e3-b986-5f70
                          909c4322
                   GOPATH=/go
          LD_LIBRARY_PATH=/usr/lib/R/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/default-
                          java/lib/server:/.singularity.d/libs
                     PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
       REQUESTS_CA_BUNDLE=<not set>
            SSL_CERT_FILE=<not set>
                USER_PATH=/go/bin:/usr/local/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/us
                          r/bin:/sbin:/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/loc
                          al/sbin

     active environment : None
       user config file : /root/.condarc
 populated config files : 
          conda version : 4.11.0
    conda-build version : not installed
         python version : 3.9.5.final.0
       virtual packages : __linux=5.11.0=0
                          __glibc=2.33=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /miniconda  (read only)
      conda av data dir : /miniconda/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /miniconda/pkgs
                          /root/.conda/pkgs
       envs directories : /root/.conda/envs
                          /miniconda/envs
               platform : linux-64
             user-agent : conda/4.11.0 requests/2.27.1 CPython/3.9.5 Linux/5.11.0-1027-azure debian/ glibc/2.33
                UID:GID : 0:0
             netrc file : None
           offline mode : False


An unexpected error has occurred. Conda has prepared the above report.

Error in Sys.setenv(PATH = new_path) : wrong length for argument
Calls: <Anonymous> ... use_python -> python_config -> python_munge_path -> Sys.setenv
In addition: Warning message:
In system2(conda, c("run", in_env, run_args, shQuote(cmd), args),  :
  running command ''/miniconda/bin/conda' run --prefix '/opt/ormr' --no-capture-output 'python' -c "import os; print(os.environ['PATH'])"' had status 1
Execution halted
richelbilderbeek
  • 349
  • 3
  • 10

0 Answers0