3

My xarray Dataset has three dimensions,

  • x,y,t

and 2 variables,

  • foo, bar

I would like to apply function baz() on every x, y coordinate pair's time series t

baz() will accept an array of foo-s and and array of bar-s for a given (x, y)

I'm having a tough time understanding whether or not built in structures to handle/distribute this exists in either, xarray, pandas, numpy, or dask.

Any hints?

My current approach is writing a python array iterator, or exploring ufunc:

  • The issues with iterating over something in python is that I have to deal with concurrency myself and that there is overhead in the iteration.

  • The issues with ufunc is that I don't understand it enough and seems like it's a function that is applied to individual elements of the array, not subsets along axes.

The hopeful part of me is looking for something that is xarray native or daskable.

Conic
  • 998
  • 1
  • 11
  • 26
  • A real world example here would help a lot. – naught101 Oct 22 '18 at 05:34
  • Unrelated to what I actually need this for, I've come up with a dummy example. Lets say I have gridded population data spanning 30 years at a resolution of 1 year. For an area that is X by Y. For each (x,y) elem XY, I want to run linear regression on 30 time-stamped values for that specific coordinate. The result is a grid that stores the slope of each regression. This would represent a filtered linear population growth for that given area and highlight high population growth rather than just high populations. – Conic Oct 23 '18 at 20:05
  • In my case. It's high dimensional time series stats. Hence the need for multiple variables. – Conic Oct 23 '18 at 20:36

1 Answers1

0

Sounds (vaguely) like you are looking for Dataset.reduce:

ds.reduce(baz, dim=('x', 'y'))
j08lue
  • 1,647
  • 2
  • 21
  • 37
  • This is an interesting idea. I'll play around with it. I can be more specific( if you think I worded my problem vaguely) – Conic Dec 15 '17 at 17:49
  • A [MCVE](https://stackoverflow.com/help/mcve) would help. And a clear definition of the output you would like to have. :-) – j08lue Dec 19 '17 at 13:42