1

I want to plot time series data with the help of datashader.
I have all the data in the form of a NetCDF file.
Since the file is huge - about 2 gb long I don't want all the things to be loaded in the memory at the same time, so I am using Dask to use chunks of the data instead.
I tried looking up for resources online that uses datashader and Dask at the same time but I could not find anything.
Is the thing that I am doing possible? Also if it is possible can you point me to some resources?

Sander van den Oord
  • 10,986
  • 5
  • 51
  • 96
Tushar Poddar
  • 55
  • 6
  • Please see https://datashader.org/user_guide/Performance.html and report back – mdurant May 15 '20 at 13:16
  • @mdurant I went over the page. So basically it says that I can use xaarys and dask array with datashader. It also says that we might find it useful to use the fastparquet library with Snappy compression. Thanks a lot for the help. Is there any other resource that i might find helpful? – Tushar Poddar May 15 '20 at 15:12
  • @mdurant also I was thinking of plotting the spectrogram of a huge sound file using that data. Can you tell me how should I use the chunk data of dask to show the spectrogram of the part that the user wants to see and as the user slides the datashader window to the left or right only then the other part of the spectrogram is made visible. – Tushar Poddar May 15 '20 at 15:54
  • There are quie a few examples of dask/holoviews/datashader around, not sure of anything quite like yours. Critical will be choosing an appropriate chunking, which I suppose would be along the time (only?) axis; but experimenting will tell you what works best. Merging spectra between chunks may be hard, if it must be coherent. – mdurant May 15 '20 at 17:22

0 Answers0