Geoviews Dataset class taking too much time for medium size xarray grid

Question

I'm trying to plot an xarray grid using the Geoviews Dataset class. The data has shape: (12,1300,1936) with coordinates (months,Longitude, Latitude).

The instantiation takes way too much time (almost 7 hours in an i5 32GB Ram computer). Ploting a smaller dataset works (taking some seconds).

This is the string representation of the xarray (named xmam):

<xarray.DataArray (MeanTemp Month:: 12, Latitude: 1300, Longitude: 1936)>
array([[[ nan,  nan, ...,  nan,  nan],
        [ 14.,  14., ...,  nan,  nan],
        ..., 
        [ nan,  nan, ...,  nan,  nan],
        [ nan,  nan, ...,  nan,  nan]],

       [[ nan,  nan, ...,  nan,  nan],
        [ 16.,  16., ...,  nan,  nan],
        ..., 
        [ nan,  nan, ...,  nan,  nan],
        [ nan,  nan, ...,  nan,  nan]],

       ..., 
       [[ nan,  nan, ...,  nan,  nan],
        [ 17.,  17., ...,  nan,  nan],
        ..., 
        [ nan,  nan, ...,  nan,  nan],
        [ nan,  nan, ...,  nan,  nan]],

       [[ nan,  nan, ...,  nan,  nan],
        [ 14.,  14., ...,  nan,  nan],
        ..., 
        [ nan,  nan, ...,  nan,  nan],
        [ nan,  nan, ...,  nan,  nan]]], dtype=float32)
Coordinates:
  * MeanTemp Month:  (MeanTemp Month:) |S9 'January' 'February' ... 'December'
  * Latitude         (Latitude) float64 25.57 25.56 25.55 ... 14.76 14.75 14.74
  * Longitude        (Longitude) float64 -103.6 -103.6 -103.6 ... -87.49 -87.48

This is how I'm instantiating the Geoviews Dataset

gvds = gv.Dataset(xmam,kdims=['Latitude', 'Longitude'],vdims=['MeanTemp Month:'],dynamic=True)

I tried with and without the dynamic parameter giving similar results.

What do you think it's the problem ?

Note that Datashader has no role yet because I'm just creating the object, not even plotting it!

Your declaration of the Dataset is weird, "MeanTemp Month" is not a value dimension it's just another key dimension. I suspect that's forcing HoloViews to try to flatten your data in some weird way. I think it should probably just error instead but what you are doing is definitely incorrect. Try just declaring the Dataset with `gv.Dataset(xmam)` — philippjfr, May 14 '19 at 12:56

score 0 · Answer 1 · answered May 14 '19 at 16:01

The problem were two:

How the xarray (xa) was built. That is, the dims argument make reference to the name of the coordinates. The coords parameter (if it's a dictionary) needs to have the same name as dim.

Consider the following example: Assuming data is a numpy narray with shape (12,100,100)

coords_months = range(12)
coords_lon = np.linspace(-103,87,100)
coords_lat = np.linspace(14,25,100)
dims = ['months','longitude', 'latitude']
coords = {'months':coords_months,'latitude':coords_lat,'longitude':coords_lon }
xdata = xa.DataArray(data,coords=coords,dims=dims,name='MeanTemperature')

As philippjfr mentioned, the declaration of the Geoviews (gv) DataSet constructor was wrong.

Instead I used this:

gvds = gv.Dataset(xdata)

The plotting took a bit of time (2 mins). However I could use the regrid method defined in:

from holoviews.operation.datashader import regrid
image = gvds.to(gv.Image,['Longitude','Latitude'],dynamic=True)
regrid(image)

which worked perfectly.

Geoviews Dataset class taking too much time for medium size xarray grid

1 Answers1