0

I have a .dat file containing a large (600000,6,8000) float array generated with numpy.memmap.

The third axis represents a date range. At runtime, the user specifies a narrower date range. The result is that I end up slicing along the 3rd dimension array[i, :, user_start:user_end] millions of times during execution.

I know it's possible to offset the memmap by a fixed number, but is there a way to confine the dimensions of the array at the beginning of the run such that I don't need to slice the third axis each time?

triphook
  • 2,915
  • 3
  • 25
  • 34

1 Answers1

2

A slice of a memory mapped array creates another memory mapped array with the same underlying file, so you could create a new array that is the slice defined by the user input, and use the new array from then on.

If a is the memory mapped array, define

b = a[:, :, user_start:user_end]

and then use b instead of repeatedly making the same slice of a. That is, instead of a[i, :, user_start:user_end], use b[i, :, :] (or b[i, :] or even just b[i], since the final : in a numpy index is optional).

Warren Weckesser
  • 110,654
  • 19
  • 194
  • 214