2

I am trying to get tensorflow up and running in R but several attempts have failed. First my specs: Computer: MacBook Pro M1 Max OS: MacOS Ventura 13.1 R version: version 4.2.2 (2022-10-31) [arch: aarch64] with RStudio 2022.12.0 Build 353

I followed initially these guidelines: https://tensorflow.rstudio.com/install/ and https://tensorflow.rstudio.com/install/local_gpu

In R I then run the following in a new session:

library(reticulate)
virtualenv_create("r-reticulate")

library(tensorflow)
tf$constant("Hello Tensorflow!")

I receive the following output:

> library(reticulate)
> virtualenv_create("r-reticulate")
virtualenv: r-reticulate
> library(tensorflow)
> tf$constant("Hello Tensorflow!")
Error: Valid installation of TensorFlow not found.

Python environments searched for 'tensorflow' package:
 /Users/daniel/Library/r-miniconda-arm64/envs/r-reticulate/bin/python3.8

Python exception encountered:
 Traceback (most recent call last):
  File "/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/library/reticulate/python/rpytools/loader.py", line 39, in _import_hook
    module = _import(
  File "/Users/daniel/Library/r-miniconda-arm64/envs/r-reticulate/lib/python3.8/site-packages/tensorflow/__init__.py", line 443, in <module>
    _ll.load_library(_plugin_dir)
  File "/Users/daniel/Library/r-miniconda-arm64/envs/r-reticulate/lib/python3.8/site-packages/tensorflow/python/framework/load_library.py", line 151, in load_library
    py_tf.TF_LoadLibrary(lib)
tensorflow.python.framework.errors_impl.NotFoundError: dlopen(/Users/daniel/Library/r-miniconda-arm64/envs/r-reticulate/lib/python3.8/site-packages/tensorflow-plugins/libmetal_plugin.dylib, 0x0006): symbol not found in

When I type 'python3 --version' in the Terminal, it returns: Python 3.10.9 and when I type 'type -a python3' it gives me the following paths:

 python3 is /opt/homebrew/bin/python3
 
 python3 is /Library/Frameworks/Python.framework/Versions/3.11/bin/python3 

 python3 is /usr/local/bin/python3 

 python3 is /usr/bin/python3

This looked a bit odd to me to have these different paths, but I don't know because I haven't worked with Python installations before; also note that the initial Error Message returns this path (/Users/daniel/Library/r-miniconda-arm64/envs/r-reticulate/bin/python3.8), which is not listed

In additional trials, I specified the virtual environment path to any of these python paths, such as:

virtualenv_create("r-reticulate", python = "/Library/Frameworks/Python.framework/Versions/3.11/bin/python3")

The Error message is the same though.

I also tried the following solutions to reinstall and specify differently.

https://github.com/rstudio/tensorflow/issues/new?assignees=&labels=Installation&template=installation-issue.md&title=Installation+Issue

https://gist.github.com/juliasilge/035d54c59436604d6142ebebf29aa224

Installing TensorFlow on M1 Chip - Issues. - PackagesNotFoundError: The following packages are not available from current channels:

In the global menu --> python: I have selected '/usr/local/bin/python3.11'. When I try to install tensorflow for Python it returns the following:

py_install('tensorflow') Using virtual environment '~/.virtualenvs/r-reticulate' ...

  • '/Users/daniel/.virtualenvs/r-reticulate/bin/python' -m pip install --upgrade --no-user 'pip' 'wheel' 'setuptools' sh: /Users/daniel/.virtualenvs/r-reticulate/bin/python: No such file or directory Error: Error installing package(s): "'pip'", "'wheel'", "'setuptools'" In addition: Warning message: In system2(command, args, ...) : error in running command

Thanks for any pointers you can provide!

After reinstalling miniconda and tensorflow dependencies for metal (following apple's developer instructions). I get the following: tf$constant("Hello Tensorflow!") List of 20 $ python : chr "/Users/daniel/miniconda3/bin/python3.10" $ libpython : chr "/Users/daniel/miniconda3/lib/libpython3.10.dylib" $ pythonhome : chr "/Users/daniel/miniconda3:/Users/daniel/miniconda3" $ pythonpath : chr "/Users/daniel/miniconda3/lib/python310.zip:/Users/daniel/miniconda3/lib/python3.10:/Users/daniel/miniconda3/lib"| truncated $ prefix : chr "/Users/daniel/miniconda3" $ exec_prefix : chr "/Users/daniel/miniconda3" $ base_exec_prefix : chr "/Users/daniel/miniconda3" $ virtualenv : chr "" $ virtualenv_activate : chr "" $ version_string : chr "3.10.8 | packaged by conda-forge | (main, Nov 22 2022, 08:25:13) [Clang 14.0.6 ]" $ version : chr "3.10" $ architecture : chr "64bit" $ anaconda : logi FALSE $ conda : logi TRUE $ numpy :List of 2 ..$ path : chr "/Users/daniel/miniconda3/lib/python3.10/site-packages/numpy" ..$ version:Class 'numeric_version' hidden list of 1 .. ..$ : int [1:3] 1 22 3 $ required_module : chr "tensorflow" $ required_module_path: chr "/Users/daniel/miniconda3/lib/python3.10/site-packages/tensorflow" $ available : logi TRUE $ python_versions : chr "/Users/daniel/miniconda3/bin/python3.10" $ forced : chr "use_python function"

  • attr(*, "class")= chr "py_config" Error: Python module tensorflow was not found.

Detected Python configuration:

Dan
  • 59
  • 8
  • 1
    Go to Global Options (cmd + ,) -> Python (left side, middle). What interpreter is indicated there? If you select 'Select...' you can see different versions & virtual environments that exist, well... that RStudio sees. If you have Python installed, there's no reason for Anaconda (IMO... full body shudder on conda). When you create a virtual environment, you still have to say you want to use it. You also have to install packages into it. (Installing packages into Python, in general, will do nothing for you.) For example, `py_install('tensorflow')` (using the `reticulate` R package). – Kat Dec 30 '22 at 18:26
  • Thank you. I tried various paths in there but I now have selected "/usr/local/bin/python3.11" in the menu. When I try to install tensorflow it gives me an error. I have updated the original post. – Dan Dec 30 '22 at 21:23
  • 1
    Tensorflow isn't out for 3.11; you need anything before 3.11 (like 3.10). **That's not the error, though.** After changing Python version ensure you have the Python package `virtualenv`: `pip install virtualenv` or `python3.10 -m pip install virtualenv` (whichever works for you). Then return to RStudio, with `library(reticulate)` called, try `py_available()`; if this returns `FALSE`, run `py_available(initialize = T)` <- if this one returns F, reinstall Python. Let me know what error you run into next! – Kat Dec 30 '22 at 22:09
  • I removed all Python versions so that there is only Python 3.9.6 in /usr/bin/python3. `py_available(initialize = T)` returns TRUE. But now I get an error when running `py_instal('tensorflor')`l: > py_install('tensorflow') Using virtual environment '/Users/daniel/.virtualenvs/base' ... + '/Users/daniel/.virtualenvs/base/bin/python' -m pip install --upgrade --no-user 'tensorflow' ERROR: Could not find a version that satisfies the requirement tensorflow (from versions: none) ERROR: No matching distribution found for tensorflow Error: Error installing package(s): "'tensorflow'" – Dan Dec 31 '22 at 02:26
  • 1
    Have you tried other packages? For example, install `seaborn` or `pandas`? In the terminal window of RStudio, (shift + opt + r), run `pip install tensorflow`. Does that work? If so, then try `virtualenv_install('tensorflow')` in the console or a script. – Kat Dec 31 '22 at 20:06
  • I was able to install other packages. The rest didn't work. I also just followed the apple developer instructions from scratch to make sure tensorflow metal dependencies are installed, which installs miniconda as environment. I went through everytging again and get a "different" error message -- but it seems that is there but not found ... (please see my updated post for the full error message). Thanks!!! – Dan Dec 31 '22 at 23:09
  • Per the Apple instructions, did you run the Python code in terminal? Did everything work in the terminal? – Kat Jan 01 '23 at 00:38
  • Yes, I ran all in the terminal using these instructions (step 1-3) and all seemed fine. [link](https://developer.apple.com/metal/tensorflow-plugin/) – Dan Jan 01 '23 at 19:45
  • Instead of `py_install('tensorflow')`, did you try `py_install('tensorflow-macos')`? If not, you need the two packages it states for `metal` in your environment (like you did for Python), not vanilla `tensorflow`. – Kat Jan 02 '23 at 02:11
  • 1
    I retried various steps and now it seems to work. The last thing that I did was to change the python path (per your earlier comment) under global tools in RStudio but not using a system path but a virtual one (the middle tab): ~/.virtualenvs/base/bin/python instead of creating/calling a virtual environment from the R console. But I can't be sure what my mistake was earlier. I am still a bit confused by all these different Python versions and paths but it seems to work now. Thank you! – Dan Jan 02 '23 at 02:48

1 Answers1

1

I got it working for tensorflow 2.8.0, doing the following steps:

  1. Install Miniconda: https://docs.conda.io/en/main/miniconda.html
  2. Open Terminal
  3. Create Conda env: conda create --name rstudio-tf-2.8
  4. Activate env: conda activate rstudio-tf-2.8
  5. Downgrade Python: conda install python=3.8
  6. Install tensorflow-deps: conda install -c apple tensorflow-deps==2.8.0
  7. Install tensorflow-macos: python -m pip install tensorflow-macos==2.8.0
  8. Install tensorflow-metal: python -m pip install tensorflow-metal==0.4.0
  9. Install tensorflow: conda install -c anaconda tensorflow==2.8.0
  10. Edit ~/.Renviron: echo "RETICULATE_PYTHON=~/miniforge3/envs/rstudio-tf-2.8/bin/python" >> ~/.Renviron
  11. Check installed packages: conda list | grep tensorflow

It should look like this:

➜  ~ conda list | grep tensorflow                 
tensorflow                2.8.0           cpu_py39h2839aeb_0    conda-forge
tensorflow-base           2.8.0           cpu_py39hcf13c84_0    conda-forge
tensorflow-deps           2.8.0                         0    apple
tensorflow-estimator      2.8.0           cpu_py39hfad2b03_0    conda-forge
tensorflow-macos          2.8.0                    pypi_0    pypi
tensorflow-metal          0.4.0                    pypi_0    pypi
  1. Go to R-Studio and restart R-Session Session -> Restart R
  2. Init and test tensorflow with following R script:
reticulate::use_python("~/miniforge3/envs/rstudio-tf-2.8/bin/python")
reticulate::use_condaenv("rstudio-tf-2.8", conda="~/miniforge3/bin/conda", required = TRUE)

library(tensorflow)
tf_config() 
tf_version() 
tf$config$list_logical_devices()
  1. If tf_config() crashes go back to Terminal and upgrade to python 3.9 conda install python=3.9
  2. Check again if all deps are listed conda list | grep tensorflow, for me tensorflow-macos and metal were missing after the python upgrade, I had to repeat step 8 and 9
  3. Repeat steps 12 and 13 - it should work now

I don't know how to avoid the crash, if I install from the beginning python 3.9 the crash still happens and get fixed by downgrading to python 3.8. My guess is come dependency issue, which I couldn't figure out.

Joba
  • 828
  • 9
  • 28