Deploying an Rmarkdown file containing Python code (enabled by reticulate) to shinyapp.io

Question

I have a problem that I have been trying to fix for a really long time now and I would really appreciate any help with it. So, I wrote an rmarkdown with both R and Python code. Everything runs fine locally. The document knits nicely (it also displays a shiny app I created inside it). The problem comes when I try to deploy it to shinyapp.io. I get a warning and an error when I do that.

Warning:

using reticulate but python was not specified; will use python at /usr/bin/python. Did you forget to set the RETICULATE_PYTHON environment variable in your .Rprofile before publishing?

Error:

Error in py_call_impl: ImportError: No module named spacy. When I start installing numPy first, it gives the same error saying it can't find numPy.

(I got these from the log in shinyapp.io. I have put the entire log at the bottom)

Here is the relevant part of my code:

reticulate::virtualenv_create(envname = 'test', python = "python")
reticulate::virtualenv_install(envname = 'test', packages = 
c('spacy','numpy'))
reticulate::use_virtualenv(virtualenv = 'test', required = FALSE)

And I also have a .Rprofile file that contains the line Sys.setenv(RETICULATE_PYTHON = ".venv/bin/python"). This file is in the same directory as the Rmd file I am deploying.

I have searched for a solution everywhere and everything I have found hasn't been useful. Some similar Github issues are still open and some similar questions on Stack Overflow don't have answers. The most useful thing I have found is this and I have tried everything they suggest there. Here is what I have tried so far:

I have downgraded rsconnect. Initially, I couldn't even deploy the Rmd. I was able to do that after downgrading rsconnect, but this didn't solve the two problems that came next.
I read that I could change the file name .Rprofile to .rsconnect_profile but this hasn't helped at all.
I have tried using Python 3.6 by specifying it when creating the virtual environment (virtualenv_create(envname = 'test', python = "python3.6"). This made things worse. Moreover, shinyappy.io keeps overriding my choice of Python. It always defaults to Python2.7.

What is that I am missing here? Did I specify the virtual environments wrong? What is the difference between the virtual environment in .Rprofile and test, the one that I create in my code and install Python packages in? How are they related? At some point, test was actually RETICULATE_PYTHON and that didn't make a difference. Having read about all of this, I am actually even more confused.

This is the rest of the log:

 Server version: 1.7.6-6

 LANG: en_US.UTF-8

 R version: 3.6.1

 shiny version: 1.3.2

 httpuv version: 1.5.1

 rmarkdown version: 1.14

 knitr version: 1.23

 jsonlite version: 1.6

 RJSONIO version: (none)

 htmltools version: 0.3.6

 warning: using reticulate but python was not specified; will use python at /usr/bin/python 

 Did you forget to set the RETICULATE_PYTHON environment variable in your .Rprofile before publishing?

 Using pandoc at /opt/connect/ext/pandoc2

 Using jsonlite for JSON processing

 Starting R with process ID: '23'

 Listening on http://127.0.0.1:46584

 processing file: index.Rmd

  |                                                                       

  |                                                                 |   0%

  |                                                                       

  |...                                                              |   5%

   ordinary text without R code

  |                                                                       

  |......                                                           |  10%

 label: setup (with options) 

 List of 1

  $ include: logi FALSE

 New python executable in /home/shiny/.virtualenvs/test/bin/python2

 Also creating executable in /home/shiny/.virtualenvs/test/bin/python

 Installing setuptools, pkg_resources, pip, wheel...done.

 Running virtualenv with interpreter /usr/bin/python2

 DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.

 Collecting spacy

   Downloading https://files.pythonhosted.org/packages/0f/bf/f76eed483f5bcb9772869a002b0029eb2888dd662f75538b3bcc197bb655/spacy-2.1.6-cp27-cp27mu-manylinux1_x86_64.whl (30.8MB)

 Collecting numpy

   Downloading https://files.pythonhosted.org/packages/1f/c7/198496417c9c2f6226616cff7dedf2115a4f4d0276613bab842ec8ac1e23/numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl (17.0MB)

 Collecting blis<0.3.0,>=0.2.2 (from spacy)

   Downloading https://files.pythonhosted.org/packages/61/b7/6f32b1e2506937525802d94136eb73dec2cacd4a21c9bec9c90549e2b413/blis-0.2.4-cp27-cp27mu-manylinux1_x86_64.whl (3.2MB)

 Collecting thinc<7.1.0,>=7.0.8 (from spacy)

   Downloading https://files.pythonhosted.org/packages/c4/d9/944e0d409e8af994d8d09268a3b7fb9eacbdd08f9bc72b9b0b66c405c05a/thinc-7.0.8-cp27-cp27mu-manylinux1_x86_64.whl (2.1MB)

   Downloading https://files.pythonhosted.org/packages/ac/aa/9b065a76b9af472437a0059f77e8f962fe350438b927cb80184c32f075eb/pathlib-1.0.1.tar.gz (49kB)

 Collecting pathlib==1.0.1; python_version < "3.4" (from spacy)

   Downloading https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl (57kB)

 Collecting requests<3.0.0,>=2.13.0 (from spacy)

 Collecting preshed<2.1.0,>=2.0.1 (from spacy)

   Downloading https://files.pythonhosted.org/packages/df/b1/4ff2cbd423184bd68e85f1daa6692753cd7710b0ba68552eb64542906a57/cymem-2.0.2-cp27-cp27mu-manylinux1_x86_64.whl

 Collecting cymem<2.1.0,>=2.0.2 (from spacy)

 Collecting plac<1.0.0,>=0.9.6 (from spacy)

 Collecting murmurhash<1.1.0,>=0.28.0 (from spacy)

   Downloading https://files.pythonhosted.org/packages/ed/31/247b34db5ab06afaf5512481e77860fb4cd7a0c0ddff9d2566651c8c2f07/murmurhash-1.0.2-cp27-cp27mu-manylinux1_x86_64.whl

   Downloading https://files.pythonhosted.org/packages/9e/9b/62c60d2f5bc135d2aa1d8c8a86aaf84edb719a59c7f11a4316259e61a298/plac-0.9.6-py2.py3-none-any.whl

 Collecting srsly<1.1.0,>=0.0.6 (from spacy)

   Downloading https://files.pythonhosted.org/packages/3d/17/e003b2e9122500762a9ba6f3dfe4db912604e6be840c7d3041ae72787ae3/srsly-0.0.7-cp27-cp27mu-manylinux1_x86_64.whl (175kB)

 Collecting wasabi<1.1.0,>=0.2.0 (from spacy)

   Downloading https://files.pythonhosted.org/packages/25/b1/9098d07e70b960001a8a9b99435c6987006d0d7bcbf20523adce9272f66e/preshed-2.0.1-cp27-cp27mu-manylinux1_x86_64.whl (80kB)

 Collecting tqdm<5.0.0,>=4.10.0 (from thinc<7.1.0,>=7.0.8->spacy)

   Downloading https://files.pythonhosted.org/packages/be/ba/08c53c55cc97f62310ed83e1a4d91e424f221645c88c2dddd41f179fd1f7/wasabi-0.2.2.tar.gz

   Downloading https://files.pythonhosted.org/packages/9f/3d/7a6b68b631d2ab54975f3a4863f3c4e9b26445353264ef01f465dc9b0208/tqdm-4.32.2-py2.py3-none-any.whl (50kB)

 Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests<3.0.0,>=2.13.0->spacy)

   Downloading https://files.pythonhosted.org/packages/e6/60/247f23a7121ae632d62811ba7f273d0e58972d75e58a94d329d51550a47d/urllib3-1.25.3-py2.py3-none-any.whl (150kB)

 Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.13.0->spacy)

   Downloading https://files.pythonhosted.org/packages/69/1b/b853c7a9d4f6a6d00749e94eb6f3a041e342a885b87340b79c1ef73e3a78/certifi-2019.6.16-py2.py3-none-any.whl (157kB)

 Collecting chardet<3.1.0,>=3.0.2 (from requests<3.0.0,>=2.13.0->spacy)

   Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)

 Collecting idna<2.9,>=2.5 (from requests<3.0.0,>=2.13.0->spacy)

   Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB)

 Building wheels for collected packages: pathlib, wasabi

   Building wheel for pathlib (setup.py): started

   Building wheel for pathlib (setup.py): finished with status 'done'

   Stored in directory: /home/shiny/.cache/pip/wheels/f9/b2/4a/68efdfe5093638a9918bd1bb734af625526e849487200aa171

   Building wheel for wasabi (setup.py): started

   Building wheel for wasabi (setup.py): finished with status 'done'

   Stored in directory: /home/shiny/.cache/pip/wheels/b3/2c/d1/78fd1255da73ff77b372ecc56bcdb15115ab0882bb6f67af17

 Successfully built pathlib wasabi

 Installing collected packages: numpy, blis, tqdm, wasabi, pathlib, plac, cymem, preshed, murmurhash, srsly, thinc, urllib3, certifi, chardet, idna, requests, spacy

 Successfully installed blis-0.2.4 certifi-2019.6.16 chardet-3.0.4 cymem-2.0.2 idna-2.8 murmurhash-1.0.2 numpy-1.16.4 pathlib-1.0.1 plac-0.9.6 preshed-2.0.1 requests-2.22.0 spacy-2.1.6 srsly-0.0.7 thinc-7.0.8 tqdm-4.32.2 urllib3-1.25.3 wasabi-0.2.2

  |                                                                       

  |.........                                                        |  14%

   ordinary text without R code

 label: load_packages (with options) 

 List of 2

  $ include: logi FALSE

  |                                                                       

  |............                                                     |  19%

  $ engine : chr "python"

 Quitting from lines 22-29 (index.Rmd) 

 Warning: Error in py_call_impl: ImportError: No module named spacy

   File "<string>", line 1, in <module>
 Detailed traceback: 
   149: <Anonymous>

in your terminal or command prompt try this `pip install spacy` — Not_Dave, Jul 22 '19 at 22:24
Thanks @Not_Dave! But I have done that. Everything installs fine, it's just retrieving them that's failing. In shinyapp.io, spacy installs but importing it fails. — rev_tesh, Jul 22 '19 at 23:16
Hi! have you solved this issue? I think I met the same error .... — YihanBao, Jun 21 '20 at 18:31

score 0 · Answer 1 · answered Mar 19 '20 at 18:28

0

Have you tried: reticulate::use_virtualenv(virtualenv = 'test', required = TRUE)

So far it seems logical to enforce. I also came across this one: https://github.com/ranikay/shiny-reticulate-app

answered Mar 19 '20 at 18:28

Ofelia

91
1
4

Deploying an Rmarkdown file containing Python code (enabled by reticulate) to shinyapp.io

1 Answers1