I installed Anaconda on a CentOS VM on an Ubuntu server. I then created a conda environment for all of my python data science libraries on this VM. I have a matching conda environment on my laptop.
I then followed this guide to allow me to perform data analytics on the server using a Jupyter notebook remotely on my laptop: https://amber-md.github.io/pytraj/latest/tutorials/remote_jupyter_notebook
I am able to successfully load my Jupyter notebook on my laptop and perform analytics on the server. I can also mark my completed app as .servable() in the notebook and then use panel serve --show ... on ther VM terminal to serve the completed interactive app to LAN users via a web browser.
My issue is that when I instead mark my app with .show() to allow me to view the app/report that I am currently working on in the notebook, I get the following message outputted: <bokeh.server.server.Server at 0x7faa92ef5590> but the app does not load in a new browser tab as expected.
How can I make app.show() work when using a Jupyter notebook remotely instead of locally?
Update 23/01/20 in response to @SandervandenOord's answer: "You need to specify the port to .show() and make sure the user can access that port on the remote server"
For the illustration of this question(not actuals), say my server VM is address 192.168.0.1 and my laptop is 192.168.0.2 and the port that I want to use with app.show() is 3333.
1. Setup the port:
On the server (192.168.0.1):
- Check that port 3333 is not being used:
sudo lsof -i:3333 - Check that iptables aren't showing the port as open:
sudo iptables-save | grep 3333 - Add the port to /etc/services file:
sudo nano /etc/services - I added the following line to the top of the file and saved:
# service-name port/protocol [aliases ...] [# comment]
bokeh-server 3333/tcp # Open port to allow app.show() to work on a remotely executed Jupyter Notebook
- Enable the firewall:
sudo systemctl enable firewalld - Start the firewall:
sudo systemctl start firewalld - Check the status of the firewall:
sudo systemctl status firewalld Confirm that this port is now open:
sudo iptables-save | grep 3333
Output:
-A IN_public_allow -p tcp -m tcp --dport 3333 -m conntrack --ctstate NEW -j ACCEPT
- Test that the user can access the port on the server:
I installed ncat on the server (192.168.0.1):
sudo yum install nmap-ncat -y
Then I set up messaging between the server and my laptop:
- On server:
nc -l 3333 - On laptop:
nc 192.168.0.1 3333
and also between the laptop and the server:
- On laptop:
nc -l 3333 On server:
nc 192.168.0.2 3333Messages successfully make it from server to laptop and vice versa.
Confirmation of these two connections:
sudo lsof -i:3333
which outputted:
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
nc 1776 XXX 5u IPv4 23627 0t0 TCP vmserver:bokeh-server->192.168.0.2:55358 (ESTABLISHED)
nc 1846 XXX 3u IPv4 22895 0t0 TCP vmserver:33658->192.168.0.2:bokeh-server (ESTABLISHED)
showing that the server on port 3333 can communicate with the laptop on a different port and the laptop on port 3333 can communicate with the server on a different port.
I'm not sure how to make them connect on the same port though? ie both on port 3333? Is this the source of my issue?
- Then in my jupyter notebook, I replaced
app.show()with
app.show(port=3333, websocket_origin=None, threaded=False)
However I am still experiencing the same behaviour as previously in that the app does not load in a new browser tab as expected.
I then experimented with parameters websocket_origin (tried "192.168.0.1","192.168.0.2" and "*") and threaded (True/False) but changing these didn't give my desired result of the app loading in a new browser tab.
A couple of interesting points:
1. If I run my notebook which executes
app.show(port=3333, websocket_origin=None, threaded=False)
and then on the server run:
sudo lsof -i:3333
then the following is returned:
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
ZMQbg/1 19328 xxx 45u IPv4 83214 0t0 TCP *:bokeh-server (LISTEN)
ZMQbg/1 19328 xxx 54u IPv6 83215 0t0 TCP *:bokeh-server (LISTEN)
It seems like the server is waiting for something to happen on port 3333? ie LISTEN? Unlike when I was messaging between the server and my laptop (and vice versa) there doesn't actually seem to be an ESTABLISHED connection though.
Please note that those 2 lines above only appear after I run the notebook.
- After I have executed my notebook (and the app in the new tab hasn't opened), if I try to execute the notebook again then I get this warning in my jupyter notebook:
OSError: [Errno 98] Address already in use
So it does seem like the connection between the server and laptop has been ESTABLISHED or at very least reserved? To fix this, I then have to close the connection to my server and then on my laptop run the command:
lsof -ti:8888 | xargs kill -1 before then reestablishing my connection between the server and my laptop as per https://amber-md.github.io/pytraj/latest/tutorials/remote_jupyter_notebook
@SandervandenOord do you have any pointers as to what I could try next?
Many thanks in advance.
Software versions on the conda environment on the VM:
# Name Version Build Channel
_anaconda_depends 2019.03 py37_0
_libgcc_mutex 0.1 main
alabaster 0.7.12 py37_0
anaconda custom py37_1
anaconda-client 1.7.2 py37_0
anaconda-project 0.8.4 py_0
arrow-cpp 0.11.1 py37h0e61e49_1004 conda-forge
asn1crypto 1.3.0 py37_0
astroid 2.3.3 py37_0
astropy 4.0 py37h7b6447c_0
atomicwrites 1.3.0 py37_1
attrs 19.3.0 py_0
babel 2.8.0 py_0
backcall 0.1.0 py37_0
backports 1.0 py_2
backports.os 0.1.1 py37_0
backports.shutil_get_terminal_size 1.0.0 py37_2
beautifulsoup4 4.8.2 py37_0
bitarray 1.2.0 py37h7b6447c_0
bkcharts 0.2 py37_0
blas 1.0 mkl
bleach 3.1.0 py37_0
blosc 1.16.3 hd408876_0
bokeh 1.4.0 py37_0
boost-cpp 1.68.0 h11c811c_1000 conda-forge
boto 2.49.0 py37_0
bottleneck 1.3.1 py37hdd07704_0
bzip2 1.0.8 h7b6447c_0
ca-certificates 2019.11.28 hecc5488_0 conda-forge
cairo 1.14.12 h8948797_3
cartopy 0.17.0 py37hbb7e04d_1
certifi 2019.11.28 py37_0
cffi 1.13.2 py37h2e261b9_0
cftime 1.0.4.2 py37hdd07704_0
chardet 3.0.4 py37_1003
click 7.0 py37_0
click-plugins 1.1.1 py_0
cligj 0.5.0 py37_0
cloudpickle 1.2.2 py_0
clyent 1.2.2 py37_1
colorama 0.4.3 py_0
colorcet 2.0.2 py_0 pyviz
contextlib2 0.6.0.post1 py_0
cryptography 2.8 py37h1ba5d50_0
curl 7.67.0 hbc83047_0
cycler 0.10.0 py37_0
cython 0.29.14 py37he6710b0_0
cytoolz 0.10.1 py37h7b6447c_0
dask 2.9.1 py_0
dask-core 2.9.1 py_0
datashader 0.9.0 py_0 pyviz
datashape 0.5.4 py37_1
dbus 1.13.12 h746ee38_0
decorator 4.4.1 py_0
defusedxml 0.6.0 py_0
distributed 2.9.1 py_0
docutils 0.15.2 py37_0
entrypoints 0.3 py37_0
et_xmlfile 1.0.1 py37_0
expat 2.2.6 he6710b0_0
fastcache 1.1.0 py37h7b6447c_0
feather-format 0.4.0 py_1003 conda-forge
filelock 3.0.12 py_0
fiona 1.8.4 py37hc38cc03_0
flask 1.1.1 py_0
fontconfig 2.13.0 h9420a91_0
freetype 2.9.1 h8a8886c_1
freexl 1.0.5 h14c3975_0
fribidi 1.0.5 h7b6447c_0
fsspec 0.6.2 py_0
gdal 2.3.3 py37hbb2a789_0
geopandas 0.6.1 py_0
geos 3.7.1 he6710b0_0
geoviews 1.6.6 py_0 pyviz
geoviews-core 1.6.6 py_0 pyviz
get_terminal_size 1.0.0 haa9412d_0
gevent 1.4.0 py37h7b6447c_0
giflib 5.1.4 h14c3975_1
glib 2.63.1 h5a9c865_0
glob2 0.7 py_0
gmp 6.1.2 h6c8ec71_1
gmpy2 2.0.8 py37h10f8cd9_2
graphite2 1.3.13 h23475e2_0
greenlet 0.4.15 py37h7b6447c_0
gst-plugins-base 1.14.0 hbbd80ab_1
gstreamer 1.14.0 hb453b48_1
h5py 2.9.0 py37h7918eee_0
harfbuzz 1.8.8 hffaf4a1_0
hdf4 4.2.13 h3ca952b_2
hdf5 1.10.4 hb1b8bf9_0
heapdict 1.0.1 py_0
holoviews 1.12.7 py_0 pyviz
html5lib 1.0.1 py37_0
hvplot 0.5.2 py_0 pyviz
hypothesis 4.54.2 py37_0
icu 58.2 h9c2bf20_1
idna 2.8 py37_0
imageio 2.6.1 py37_0
imagesize 1.2.0 py_0
importlib_metadata 1.3.0 py37_0
intel-openmp 2019.4 243
ipykernel 5.1.3 py37h39e3cac_1
ipython 7.11.1 py37h39e3cac_0
ipython_genutils 0.2.0 py37_0
ipywidgets 7.5.1 py_0
isort 4.3.21 py37_0
itsdangerous 1.1.0 py37_0
jbig 2.1 hdba287a_0
jdcal 1.4.1 py_0
jedi 0.15.2 py37_0
jeepney 0.4.2 py_0
jinja2 2.10.3 py_0
joblib 0.14.1 py_0
jpeg 9b h024ee3a_2
json-c 0.13.1 h1bed415_0
json5 0.8.5 py_0
jsonschema 3.2.0 py37_0
jupyter 1.0.0 py37_7
jupyter_client 5.3.4 py37_0
jupyter_console 6.0.0 py37_0
jupyter_core 4.6.1 py37_0
jupyterlab 1.2.4 pyhf63ae98_0
jupyterlab_server 1.0.6 py_0
kealib 1.4.7 hd0c454d_6
keyring 20.0.0 py37_0
kiwisolver 1.1.0 py37he6710b0_0
krb5 1.16.4 h173b8e3_0
lazy-object-proxy 1.4.3 py37h7b6447c_0
ld_impl_linux-64 2.33.1 h53a641e_7
libarchive 3.3.3 h5d8350f_5
libboost 1.67.0 h46d08c1_4
libcurl 7.67.0 h20c2e04_0
libdap4 3.19.1 h6ec2957_0
libedit 3.1.20181209 hc058e9b_0
libffi 3.2.1 hd88cf55_4
libgcc-ng 9.1.0 hdf63c60_0
libgdal 2.3.3 h2e7e64b_0
libgfortran-ng 7.3.0 hdf63c60_0
libkml 1.3.0 h590aaf7_4
liblief 0.9.0 h7725739_2
libnetcdf 4.6.1 h11d0813_2
libpng 1.6.37 hbc83047_0
libpq 11.2 h20c2e04_0
libprotobuf 3.6.1 hdbcaa40_1001 conda-forge
libsodium 1.0.16 h1bed415_0
libspatialindex 1.9.3 he6710b0_0
libspatialite 4.3.0a hb08deb6_19
libssh2 1.8.2 h1ba5d50_0
libstdcxx-ng 9.1.0 hdf63c60_0
libtiff 4.1.0 h2733197_0
libtool 2.4.6 h7b6447c_5
libuuid 1.0.3 h1bed415_2
libxcb 1.13 h1bed415_1
libxml2 2.9.9 hea5a465_1
libxslt 1.1.33 h7d1a2b0_0
llvmlite 0.31.0 py37hd408876_0
locket 0.2.0 py37_1
lxml 4.4.2 py37hefd8a0e_0
lz4-c 1.8.1.2 h14c3975_0
lzo 2.10 h49e0be7_2
markdown 3.1.1 py37_0
markupsafe 1.1.1 py37h7b6447c_0
matplotlib 3.1.1 py37h5429711_0
mccabe 0.6.1 py37_1
mistune 0.8.4 py37h7b6447c_0
mkl 2019.4 243
mkl-service 2.3.0 py37he904b0f_0
mkl_fft 1.0.15 py37ha843d7b_0
mkl_random 1.1.0 py37hd6b4f25_0
mock 3.0.5 py37_0
more-itertools 8.0.2 py_0
mpc 1.1.0 h10f8cd9_1
mpfr 4.0.1 hdf1c602_3
mpmath 1.1.0 py37_0
msgpack-python 0.6.1 py37hfd86e86_1
multipledispatch 0.6.0 py37_0
munch 2.5.0 py_0
nbconvert 5.6.1 py37_0
nbformat 4.4.0 py37_0
ncurses 6.1 he6710b0_1
netcdf4 1.4.2 py37h808af73_0
networkx 2.4 py_0
nltk 3.4.5 py37_0
nose 1.3.7 py37_2
notebook 6.0.2 py37_0
numba 0.47.0 py37h962f231_0
numexpr 2.7.0 py37h9e4a6bb_0
numpy 1.17.4 py37hc1035e2_0
numpy-base 1.17.4 py37hde5b4d6_0
numpydoc 0.9.2 py_0
olefile 0.46 py37_0
openjpeg 2.3.0 h05c96fa_1
openpyxl 3.0.2 py_0
openssl 1.1.1d h7b6447c_3
owslib 0.18.0 py_0
packaging 20.0 py_0
pandas 0.25.3 py37he6710b0_0
pandoc 2.2.3.2 0
pandocfilters 1.4.2 py37_1
panel 0.7.0 py_0 pyviz
pango 1.42.4 h049681c_0
param 1.9.2 py_0 pyviz
parquet-cpp 1.5.1 3 conda-forge
parso 0.5.2 py_0
partd 1.1.0 py_0
patchelf 0.10 he6710b0_0
path 13.1.0 py37_0
path.py 12.4.0 0
pathlib2 2.3.5 py37_0
patsy 0.5.1 py37_0
pcre 8.43 he6710b0_0
pep8 1.7.1 py37_0
pexpect 4.7.0 py37_0
pickleshare 0.7.5 py37_0
pillow 7.0.0 py37hb39fc2d_0
pip 19.3.1 py37_0
pixman 0.38.0 h7b6447c_0
pkginfo 1.5.0.1 py37_0
plotly 4.4.1 py_0 plotly
plotly_express 0.4.1 py_0 plotly
pluggy 0.13.1 py37_0
ply 3.11 py37_0
poppler 0.65.0 h581218d_1
poppler-data 0.4.9 0
proj4 5.2.0 he6710b0_1
prometheus_client 0.7.1 py_0
prompt_toolkit 2.0.9 py37_0
psutil 5.6.7 py37h7b6447c_0
ptyprocess 0.6.0 py37_0
py 1.8.1 py_0
py-lief 0.9.0 py37h7725739_2
pyarrow 0.11.1 py37hbbcf98d_1002 conda-forge
pycodestyle 2.5.0 py37_0
pycosat 0.6.3 py37h7b6447c_0
pycparser 2.19 py37_0
pycrypto 2.6.1 py37h14c3975_9
pyct 0.4.6 py_0 pyviz
pyct-core 0.4.6 py_0 pyviz
pycurl 7.43.0.3 py37h1ba5d50_0
pyepsg 0.4.0 py37_0
pyflakes 2.1.1 py37_0
pygments 2.5.2 py_0
pykdtree 1.3.1 py37hdd07704_2
pylint 2.4.4 py37_0
pyodbc 4.0.27 py37he6710b0_0
pyopenssl 19.1.0 py37_0
pyparsing 2.4.6 py_0
pyproj 1.9.6 py37h14380d9_0
pyqt 5.9.2 py37h05f1152_2
pyrsistent 0.15.6 py37h7b6447c_0
pyshp 2.1.0 py_0
pysocks 1.7.1 py37_0
pytables 3.6.1 py37h71ec239_0
pytest 5.3.2 py37_0
pytest-arraydiff 0.3 py37h39e3cac_0
pytest-astropy 0.7.0 py_0
pytest-astropy-header 0.1.1 py_0
pytest-doctestplus 0.5.0 py_0
pytest-openfiles 0.4.0 py_0
pytest-remotedata 0.3.2 py37_0
python 3.7.6 h0371630_2
python-dateutil 2.8.1 py_0
python-libarchive-c 2.8 py37_13
pytz 2019.3 py_0
pyviz_comms 0.7.2 py_0 pyviz
pywavelets 1.1.1 py37h7b6447c_0
pyyaml 5.2 py37h7b6447c_0
pyzmq 18.1.0 py37he6710b0_0
qt 5.9.7 h5867ecd_1
qtawesome 0.6.0 py_0
qtconsole 4.6.0 py_1
qtpy 1.9.0 py_0
readline 7.0 h7b6447c_5
requests 2.22.0 py37_1
retrying 1.3.3 py37_2
rope 0.14.0 py_0
rtree 0.8.3 py37_0
ruamel_yaml 0.15.87 py37h7b6447c_0
scikit-image 0.15.0 py37he6710b0_0
scikit-learn 0.22.1 py37hd81dba3_0
scipy 1.3.2 py37h7c811a0_0
seaborn 0.9.0 pyh91ea838_1
secretstorage 3.1.1 py37_0
send2trash 1.5.0 py37_0
setuptools 44.0.0 py37_0
shapely 1.6.4 py37h86c5351_0
simplegeneric 0.8.1 py37_2
singledispatch 3.4.0.3 py37_0
sip 4.19.8 py37hf484d3e_0
six 1.13.0 py37_0
snappy 1.1.7 hbae5bb6_3
snowballstemmer 2.0.0 py_0
sortedcollections 1.1.2 py37_0
sortedcontainers 2.1.0 py37_0
soupsieve 1.9.5 py37_0
sphinx 2.3.1 py_0
sphinxcontrib 1.0 py37_1
sphinxcontrib-applehelp 1.0.1 py_0
sphinxcontrib-devhelp 1.0.1 py_0
sphinxcontrib-htmlhelp 1.0.2 py_0
sphinxcontrib-jsmath 1.0.1 py_0
sphinxcontrib-qthelp 1.0.2 py_0
sphinxcontrib-serializinghtml 1.1.3 py_0
sphinxcontrib-websupport 1.1.2 py_0
spyder 3.3.6 py37_0
spyder-kernels 0.5.2 py37_0
sqlalchemy 1.3.12 py37h7b6447c_0
sqlite 3.30.1 h7b6447c_0
statsmodels 0.10.1 py37hdd07704_0
sympy 1.5.1 py37_0
tbb 2019.8 hfd86e86_0
tblib 1.6.0 py_0
terminado 0.8.3 py37_0
testpath 0.4.4 py_0
thrift-cpp 0.12.0 h0a07b25_1002 conda-forge
tk 8.6.8 hbc83047_0
toolz 0.10.0 py_0
tornado 6.0.3 py37h7b6447c_0
tqdm 4.41.1 py_0
traitlets 4.3.3 py37_0
unicodecsv 0.14.1 py37_0
unixodbc 2.3.7 h14c3975_0
urllib3 1.25.7 py37_0
wcwidth 0.1.7 py37_0
webencodings 0.5.1 py37_1
werkzeug 0.16.0 py_0
wheel 0.33.6 py37_0
widgetsnbextension 3.5.1 py37_0
wrapt 1.11.2 py37h7b6447c_0
wurlitzer 2.0.0 py37_0
xarray 0.14.1 py_1
xerces-c 3.2.2 h780794e_0
xlrd 1.2.0 py37_0
xlsxwriter 1.2.7 py_0
xlwt 1.3.0 py37_0
xz 5.2.4 h14c3975_4
yaml 0.1.7 had09818_2
zeromq 4.3.1 he6710b0_3
zict 1.0.0 py_0
zipp 0.6.0 py_0
zlib 1.2.11 h7b6447c_3
zstd 1.3.7 h0b5b093_0
