Background: I use interpolated data to generate colored images for different terrains, where my input data is a set of latitudes, longitudes and a measurement at the given location. Using scipy's Interpolate.griddata, I have been generating nearest neighbor interpolations that work okay, but I would like to transition to something more like Kriging. The scipy Interpolate.rbf looks like it will work well for the job and I already have it inside my AWS lamda function. The AWS lambda function is right on the hairy edge of being too large to upload, so I would like to avoid additional imports if possible. When I implemented the rbf, though, I immediately hit a Memory error. I found an answer to a post that looks like it would possibly work, but only for smaller data sets. Here is the code for generating the interpolated data points:
def CreateInterpolatedData(longitudes,latitudes,data):
#Sparse input data
x = longitudes
y = latitudes
z = data #at x,y, locations
xi = np.linspace(xmin, xmax, num=lenx)
yi = np.linspace(ymin, ymax, num=leny)
xi, yi = np.meshgrid(xi, yi)
# This is what I am currently using, with good performance but less than desired curve fitting
zi = Interpolate.griddata((y, x), z, (yi, xi), method='nearest')
#This is what I would like to be using
rbf = Interpolate.Rbf(x,y,z,function='gaussian', norm=euclidean_norm_numpy)
zi = rbf(xi,yi)
return zi
When the above zi = rbf(xi,yi) is called, I hit a memory exception in AWS. The shape of xi and yi are both (850,850) for this sample set. (1200,1200) - (800,800) are normal ranges for the size of the grids I am trying to interpolate over.
I am okay with processing slower if it avoids the memory error.
