How to enable automatic resampling on zoom in datashader with holoviews together with a (live) data stream through a pipe into a DynamicMap?

Question

While I have a data flow through a pipe together with a holoviews.DynamicMap containing a Curve, on which holoviews.operation.datashader.datashade() is applied: when use the zoom tool the view does not resample (as for static data) leading to a very pixelated visualization of my data. What do I need to do to enable this resampling?

I run the whole thing in a jupyter notebook with python3

When I setup my holoviews.DynamicMap just with static data and I have no pipe running it works properly.

When I start to fill the pipe (without ever using it) the resampling does not take place anymore. (I do not use the pipe at all)

Problem Scenario:

(3 cells in jupyter notebook)

(1) Import

import time
import numpy as np
import holoviews as hv

from holoviews.operation.datashader import datashade

from holoviews import opts
from holoviews.streams import Pipe

hv.extension('bokeh')

(2) Setup Pipe and Plot

#no of samples
N=100000

pipe2 = Pipe(data=[])
data_dmap = hv.DynamicMap(hv.Curve, streams=[pipe2])

data_dmap_opt = datashade(data_dmap, streams=[hv.streams.RangeXY])
data_dmap_opt.opts(width=900,xlim=(0, N),ylim=(0, 1))

(3) Generate Data Stream

def makeBigData(N):
    x = np.arange(N)
    y = np.random.rand(N)

    while True:
        time.sleep(1)
        y = np.random.rand(N)
        pipe2.send((x,y))

Debugging Scenarios:

alternative to cell (2)

(alternative 2) Setup Pipe and Plot with static Plot

#default Data
N=100000
x = np.arange(N)
y = np.random.rand(N)

pipe2 = Pipe(data=[])
data_dmap = hv.DynamicMap(hv.Curve((x,y)))


data_dmap_opt = datashade(data_dmap, streams=[hv.streams.RangeXY])
data_dmap_opt.opts(width=900,xlim=(0, 100000),ylim=(0, 1))

(this works as long as cell (3) is not executed, then this alternative stops working)

Expected result:

continuously updating plot with noise (at a later stage with real data)

Therefore the actual graph is sampled into an image, when zooming in the sampling should be adjusted to the actual view

Real results:

zooming in does not trigger adjustment of sampling into image.

score 0 · Accepted Answer · answered May 27 '19 at 11:00

The problem you are having is that if you trigger the updates from a while loop the kernel will be permanently busy, which means that it never gets freed up to respond to the events arriving from JS which tell it to resample. You need to schedule the events on the pipe asynchronously in some form. In the notebook you can do this using a tornado PeriodicCallback, e.g.:

from tornado.ioloop import PeriodicCallback
from tornado import gen

N = 100
x = np.arange(N)    

@gen.coroutine
def f():
    y = np.random.rand(N)
    pipe.send((x, y))

cb = PeriodicCallback(f, 1000)
cb.start()

How to enable automatic resampling on zoom in datashader with holoviews together with a (live) data stream through a pipe into a DynamicMap?

1 Answers1