3

I want to look for a way to force a function to broadcast.

There are scenarios in which the function/method may be overwritten in a later instance, to constant function. In such case if

arr = np.arange(0, 1, 0.0001)
f = lambda x: 5
f(arr) # this gives just integer 5, i want [5, 5,..., 5]

I am aware of methods like np.vectorize which force the function to broadcast, but the problem is this is inefficient, as it is essentially for loop under the hood. (see documentation)

We can also use factory methods like np.frompyfunc which allows us to transform python function to numpy universal function ufunc See here for instance. This outperformed np.vectorize, but still is way less efficient than builtin ufunc methods.

I was wondering if there is any efficient numpy way of handling this, namely to force the function to broadcast?

mbauman
  • 30,958
  • 4
  • 88
  • 123
Wunderbar
  • 547
  • 5
  • 11

2 Answers2

1

If there was a better way to make arbitrary Python functions broadcast, numpy.vectorize would use it. You really have to write the function with broadcasting in mind if you want it to broadcast efficiently.

In the particular case of a constant function, you can write a broadcasting constant function using numpy.full:

def f(x):
    return numpy.full(numpy.shape(x), 5)

numba.vectorize can also vectorize functions more effectively than numpy.vectorize, but you need Numba, and you need to write your function in a way that Numba can compile efficiently.

user2357112
  • 260,549
  • 28
  • 431
  • 505
0

For those who can live without generic answer, the best answer would be np.full_like(arr, val) which improves by about 20% than np.full(arr.shape, val)

And after raising this issue to author, I found some best middle ground which achieves both generality while perform rather well:

np.broadcast_arrays(x, f(x))[1]

and here are some time analysis:

arr = np.arange(1, 2, 0.0001).reshape(10, -1)

def master_f(x): return np.broadcast_arrays(x, f(x))[-1].copy('K')
def master_f_nocopy(x): return np.broadcast_arrays(x, f(x))[-1]
def vector_f(x): return np.vectorize(f)(x)

%timeit arr+1 # this takes about 10microsec
%timeit master_f(arr) # this takes about 40 mircrosec
%timeit master_f_nocopy(arr) # this takes about 20 microsec

Note this allows one to apply to projection functions such as f(x,y):=y, which is beyond the help of np.full_like.

Moreover, when it comes more complicated function like np.sin and np.cos you'll notice the difference between f(arr) and master_f_nocopy(arr) is almost negligible.

Wunderbar
  • 547
  • 5
  • 11
  • 1
    `full_like(arr, val)` takes the dtype from `arr` instead of `val`, though. – user2357112 May 15 '19 at 23:47
  • 1
    The `broadcast_arrays` approach creates a read-only view of `numpy.asarray(f(x))` where every cell of the view aliases the same cell of the underlying array. This is only okay if you don't need to modify either the result array or the underlying array. (The one where you stuck a `copy` call on the end doesn't have this problem.) – user2357112 May 16 '19 at 19:00