5

The following code works to change the “min” of the color scale of a Plotly heatmap with a slider.

But it is very slow : we have a 1 fps movement when dragging the slider.

Solutions like replacing "heatmap" by "heatmapgl" don't really solve the problem (it maybe improves to 2 fps).

How to have a more responsive / faster change of the color scale with a slider?

For many applications, it is crucial to be able to see in "realtime" (more than 10 fps), the result of a change in the color scale, and it is possible to do this with Matplotlib, etc.

How to get the same behaviour in Plotly JS?

var z = Array.from({length: 500}, () => Array.from({length: 1000}, () => Math.floor(Math.random() * 500)));  
var steps = [], i;
for (i = 0; i < 500; i++)
    steps.push({label: i, method: 'restyle', args: ['zmin', i]});
Plotly.newPlot('myDiv',  [{z: z, colorscale: 'Jet', type: 'heatmap'}], {sliders: [{steps: steps}]});
<script src="https://cdn.plot.ly/plotly-2.16.2.min.js"></script>
<div id="myDiv"></div>
Basj
  • 41,386
  • 99
  • 383
  • 673
  • 2
    This is a tough one, here the heatmap needs to be entirely redrawn for each step/zmin, afaik for this kind of task we can't just bypass the API and do things manually (it would be a huuge work), in contrast with the opactity slider in your previous posts which is applied on the _rendered_ image. Maybe you can try to reduce the number of steps and add some frames to animate in between.. – EricLavault Mar 13 '23 at 12:39
  • @EricLavault Do you think there is a way? Even if it's very low level, and if I should go down the rabbit hole... because I really need more speed than 1 frame per second for this slider :) I know it's possible because in many software, we can change a range slider and have the output re-rendered 20 ou 30 times per second if necessary. This would be a very useful feature for Plotly :) – Basj Mar 13 '23 at 13:28
  • 1
    Honestly I don't know, and i'm curious to know how it works with other lib (do you have an exmaple?). Heatmap traces end up in a base64 image, heamapgl (which is deprecated) in a canvas. In both cases, I have no idea right now how to hook into the rendering phase and if it's possible to speed up things by hand. But this is such an interesting subject (to me) I'm going to look deeper in the source code, hope to come back with good news, probably not today. – EricLavault Mar 13 '23 at 13:58
  • 1
    @EricLavault Yes this one will be a tricky one :) I'll try to write a MCVE showing how to do this with another library to have a comparison. Out of curiosity, why is `heatmapgl` deprecated? I thought OpenGL rendering would be the future of Plotly :) Also, something else: in the example code here, we have 500k points to be rendered. Rendering this shouldn't take more than 100 milliseconds, there is not much processing to be done. So 10 fps should realistically be possible. Of course I haven't looked precisely, but I don't see why rendering a half megapixel heatmap should take more than 100 ms. – Basj Mar 13 '23 at 16:19
  • 1
    Apparently Plotly's team would need contributors/sponsors to help rewrite entirely this trace type based on (Plotly's generic) cartesian features and using [regl](https://github.com/regl-project/regl) framework. It's likely that, in the meantime, with little/no sign of this happening any time soon, they don't want to maintain the current version anymore. Now from what I saw in the source code, whatever the type, the whole colorscale is rebuilt and the whole image is redrawn on a canvas, the heatmap type takes longer because the canvas ends up in an actual image in the DOM (costly). – EricLavault Mar 13 '23 at 17:36
  • 1
    Surprisingly, computing and applying the colorscale takes a long time and it might be a way forward to save some time, preventing such computation to occur on the fly for each zmin/step. For example, if we set `zsmooth: 'best'` we obtain ~10fps which is not logical unless i missed something, because the smoothing algorithm has to do the interpolation, it should take longer. – EricLavault Mar 13 '23 at 17:40
  • 1
    Interesting catch @EricLavault! I posted a Github issue here: https://github.com/plotly/plotly.js/issues/6521 – Basj Mar 14 '23 at 08:28
  • @EricLavault Also, maybe is there a way to make `heatmap` more lightweight? (i.e. remove the info box on hover, I don't know if adding these event listeners takes time or not) – Basj Mar 14 '23 at 08:35
  • 1
    Yes disabling hover interactions and other such feature could have an impact on performance. Though I just tried with `hoverinfo: 'skip'` and I see no difference. – EricLavault Mar 14 '23 at 14:09
  • Good idea. Do you know other things that we can disable to hope for a performance impact @EricLavault? – Basj Mar 14 '23 at 17:04
  • 1
    Nope.. There are probably other things to disable but nothing the api exposes afaik (except for the hover/click interactions), digging into the code might help. – EricLavault Mar 14 '23 at 17:12
  • 1
    @EricLavault I started a bounty (and I can start another one later if needed) in case you know someone who can help on this performance issue (a solution could be benefitial to the whole Plotly community). – Basj Mar 16 '23 at 14:50
  • @EricLavault Do you know someone in the Plotly team who has knowledge about this? – Basj Mar 20 '23 at 14:17
  • No, I don't know much people in general lol.. But I spotted the code involved, the thing is it's quite hard to get a (full) grasp of it, I will post more info on the issue thread. – EricLavault Mar 20 '23 at 15:30
  • @EricLavault Thanks :) Would you have a link to the relevant lines in the relevant source code file? I'll have a look too :) – Basj Mar 20 '23 at 15:40
  • Yes, the performance issue occurs in this [for loop](https://github.com/plotly/plotly.js/blob/master/src/traces/heatmap/plot.js#L306). This is the code responsible for the coloring of each individual brick. – EricLavault Mar 20 '23 at 15:42
  • Thanks @EricLavault. I also commented on the issue. TL;DR it seems that `zsmooth=false` is broken too, see here it should be 1000 x 500 pixels heatmap : https://jsfiddle.net/cpqhnjry/ but it gives this instead : https://user-images.githubusercontent.com/6168083/226556556-f91fb0d8-8816-41c6-8baf-123049cacde8.png – Basj Mar 21 '23 at 08:50
  • @EricLavault Out of curiosity, what were the main reasons of the "zsmooth=false" rendering bug? – Basj Mar 26 '23 at 19:01
  • Hi, sorry for having put aside this question for a while.. I wish I came up with a real solution, but as we've found other bugs in the meantime, it just makes things harder, so while several fixes are on their way, I added an answer below which is as I said in the issue description more like a workaround, but it works at least. – EricLavault Mar 28 '23 at 15:53

4 Answers4

2

Full credit to EricLavault for his great findings and his answer which gives a nice solution until the Plotly.JS heatmap code is improved.

Here is, for future reference, one more workaround to circuvent the bugs: using a HTML native slider (<input type="range">) instead of the Plotly slider. For me it gives another improvement of the number of frame per second from his answer by a factor at least 2.

var z = Array.from({length: 500}, () => Array.from({length: 1000}, () => Math.floor(Math.random() * 500)));  

const data = [{
  z: z, 
  colorscale: 'Jet', 
  type: 'heatmap', 
  zsmooth: 'fast'
}];

const layout = {
  width: 1200,
  height: 650,
  margin: {
    t: 40,
    b: 110,
    l: 90,
    r: 110
  }
};

Plotly.newPlot('myDiv', data, layout);

document.getElementById("slider").oninput = (e) => {
    Plotly.restyle('myDiv', { zmin: e.target.value });
};
.subplot.xy .heatmaplayer image { image-rendering: pixelated; }
input { width: 100%; }
<script src="https://cdn.plot.ly/plotly-2.16.2.min.js"></script>
<div id="myDiv"></div>
<input type="range" id="slider" min="0" max="500" />
Basj
  • 41,386
  • 99
  • 383
  • 673
1

The heatmap needs to be entirely redrawn for each slider step, which takes time particularly because of the number of points (500k).

However, as discussed in the comments, it appears that it takes longer when there is no smoothing algorithm (zsmooth:false) than when there is (zsmooth:'best' or zsmooth:'fast'), which is not logical : it should rather be the other way around because of the interpolation cost. Looking at the code, I can confirm there is a performance issue that could be addressed.

Until a PR fixes the issue, there is a workaround solution you can use, which consists of setting :

  1. zsmooth:'fast' in order to benefit from the (currently) fastest drawing method.
  2. image-rendering: pixelated to prevent the browser from doing the smoothing.

Of course, because it would be too easy, you will land on a bug, which is that high resolution heatmaps are somehow "truncated" when using zsmooth:'fast' (this has been fixed in version 2.21.0). To circumvent this one, we need to set explicit width, height and margins in the layout.

Nb. regarding the plot size, setting an explicit width, height and margins might be necessary if you want to give the user a chance to properly see the heatmap, ie. at least one pixel per brick obviously unless a specific (zooming) range is defined, and for the slider as well one pixel per step (in the end, the resolution might be an issue per se).

var z = Array.from({length: 500}, () => Array.from({length: 1000}, () => Math.floor(Math.random() * 500)));  
var steps = [], i;

for (i = 0; i < 500; i++)
    steps.push({label: i, method: 'restyle', args: ['zmin', i]});

const data = [{
  z: z, 
  colorscale: 'Jet', 
  type: 'heatmap', 
  zsmooth: 'fast'
}];

const layout = {
  sliders: [{steps: steps}],
  width: 1200,
  height: 650,
  margin: {
    t: 40,
    b: 110,
    l: 90,
    r: 110
  }
};

Plotly.newPlot('myDiv', data, layout);
.subplot.xy .heatmaplayer image {
  image-rendering: pixelated;  
}
<script src="https://cdn.plot.ly/plotly-2.16.2.min.js"></script>
<div id="myDiv"></div>
EricLavault
  • 12,130
  • 3
  • 23
  • 45
  • Wonderful answer, thanks a lot for your findings! Let's hope your future PR will be accepted, it will really improve Plotly! I posted another answer with your method + HTML slider, which also increases the FPS - I only posted it for future reference, of course full credit to you. – Basj Mar 28 '23 at 21:09
  • After some tests, in my context, having to set a fixed `width`, `height` is a bit restrictive in my layout. @EricLavault `zsmooth=best` is the best solution so far. Do you see a quick workaround to disable the interpolation in `zsmooth=best`? Something around https://github.com/plotly/plotly.js/blob/master/src/traces/heatmap/plot.js#L272? – Basj Mar 29 '23 at 08:37
  • 1
    Thanks for the feedback, I'd never have thought that just using a standard HTML slider would improve the end result without queuing or throttling the handler calls ! it seems that standard input events fire at a rate that better matches the time needed for plotly to restyle() than that rate of plotly slider events :) – EricLavault Mar 29 '23 at 18:54
  • 1
    Regarding your question, you could comment out lines 249 to 264 and fix the for loops so that they just map z values to their respective color using setColor().. but if you wait a bit, I will soon open an issue and post a PR for the zsmooth="fast" bug, even if the PR is not merged quicky, you will be able to pull the relevant code. – EricLavault Mar 29 '23 at 18:56
  • Thanks again @EricLavault. About commenting out lines 249 to 264, using `setColor()`, etc. I've already tried something like that, but without much success. If you have a small example it would be wonderful :) My problem was: how to map `i, j` canvas coordinates from `for(j = 0; j < imageHeight; j++) for(i = 0; i < imageWidth; i++, pxIndex += 4)` to `m, n` coordinates in `z[m][n]`? – Basj Mar 30 '23 at 14:37
  • Yeah I didn't try that yet but I guess you can use one of the (many) mapping functions, see the axis info objects `xa`, `ya`. – EricLavault Mar 31 '23 at 13:49
  • Just in case you have an idea for this one @EricLavault, I started a bounty :) https://stackoverflow.com/questions/75792497/zoom-on-a-plotly-heatmap – Basj Apr 18 '23 at 12:08
0

My advice to improve the responsiveness of the color scale slider in a Plotly.js heatmap:

Replace 'restyle' with 'animate' method for slider steps. Create custom frames array to store slider frames. Set duration and transition time to 0 for instantaneous updates. Use a separate function to update the color scale based on the slider value.

Saad Zafar
  • 238
  • 3
  • 8
  • Thank you for your answer, but this would not be possible in my context: I have ~ 1000 slider steps, and for each one, I have a 1000 x 1000 heatmap. This would require 1 billion data points, it's overkill. I really need on-the-fly rendering. – Basj Mar 22 '23 at 07:16
  • 1
    In this case, I recommend using WebGL-based rendering for the heatmap by changing the plot type to 'heatmapgl'. Although you mentioned it only improved the performance slightly, it's the best option for handling large datasets in Plotly.js. Here's an alternative approach: Use 'heatmapgl' instead of 'heatmap' for faster rendering. Throttle the slider input to limit the number of updates while dragging the slider. Update the color scale only after a short delay from the last slider change. – Saad Zafar Mar 26 '23 at 23:28
0

var z = Array.from({length: 500}, () => Array.from({length: 1000}, () => Math.floor(Math.random() * 500)));
var steps = [], i;
for (i = 0; i < 500; i++)
    steps.push({label: i, method: 'restyle', args: ['zmin', i]});

var myPlot = Plotly.newPlot('myDiv', [{z: z, colorscale: 'Jet', type: 'heatmap'}], {sliders: [{steps: steps}]});

function slider(func, w) {
  let timeElasped;
  return function executor(...args) {
    const l = () => {
      clearTimeout(timeElasped);
      func(...args);
    };
    clearTimeout(timeElasped);
    timeElasped = setTimeout(l, w);
  };
}

var updateZ = function(z) {
  Plotly.restyle('myDiv', 'zmin', z);
}

var sliderUpdateZ = slider(updateZ, 50);

myPlot.on('plotly_sliderchange', function(e){
  sliderUpdateZ(e.step.value);
});
<script src="https://cdn.plot.ly/plotly-2.16.2.min.js"></script>
<div id="myDiv"></div>
Basj
  • 41,386
  • 99
  • 383
  • 673
Rithik Banerjee
  • 447
  • 4
  • 16
  • Thank you for your answer. In order to improve it, can you explain in a few words what you do? Also I did a conversion into a runnable snippet, it's easier for future readers and future reference. But the code doesn't run properly. Can you please edit? Thanks! – Basj Mar 23 '23 at 16:47