0

I want to use HIPS autograd (https://github.com/HIPS/autograd) in Python 2.7 (in Jupyter notebook) to find a parameter x. My forward model (observations at given time points t as a function of the parameter x) is a piecewise function of t. Therefore, I elected to use the autograd.numpy.piecewise function. My loss (or objective) function is a straight-forward mean squared error. I am having trouble computing the automatic gradient using autograd.grad.

Simple code example below: 

import autograd.numpy as anp
from autograd import grad

def forward_model(x, t): # it's a rectangular box of width x and height 1/x centered at the origin
    y = anp.piecewise(t, [t < -x/2., (t >= -x/2.) & (t < x/2.), t >= x/2.], [0., 1/x, 0.])
    return y

def loss(x, t, y):
    y_hat = forward_model(x, t)
    return anp.mean( (y_hat - y)**2 ) # mean squared error loss

x_star = 1. # ground truth parameter x
t = anp.linspace(-1., 1., 1001) # time points to evaluate function
y = forward_model(x_star, t)

x_init = 0.5
loss_init = loss(x_init, t, y)
grad_loss = grad(loss)
grad_init = grad_loss(x_init, t, y)

The full error I get is:

ValueErrorTraceback (most recent call last) <ipython-input-507-e643ed94813b> in <module>()
     16 loss_init = loss(x_init, t, y)
     17 grad_loss = grad(loss)
---> 18 grad_init = grad_loss(x_init, t, y)

C:\Users\alan_dong\AppData\Local\Continuum\Anaconda2\lib\site-packages\autograd\wrap_util.pyc in nary_f(*args, **kwargs)
     18             else:
     19                 x = tuple(args[i] for i in argnum)
---> 20             return unary_operator(unary_f, x, *nary_op_args, **nary_op_kwargs)
     21         return nary_f
     22     return nary_operator

C:\Users\alan_dong\AppData\Local\Continuum\Anaconda2\lib\site-packages\autograd\differential_operators.pyc in grad(fun, x)
     22     arguments as `fun`, but returns the gradient instead. The function `fun`
     23     should be scalar-valued. The gradient has the same type as the argument."""
---> 24     vjp, ans = _make_vjp(fun, x)
     25     if not vspace(ans).size == 1:
     26         raise TypeError("Grad only applies to real scalar-output functions. "

C:\Users\alan_dong\AppData\Local\Continuum\Anaconda2\lib\site-packages\autograd\core.pyc in make_vjp(fun, x)
      8 def make_vjp(fun, x):
      9     start_node = VJPNode.new_root(x)
---> 10     end_value, end_node =  trace(start_node, fun, x)
     11     if end_node is None:
     12         def vjp(g): return vspace(x).zeros()

C:\Users\alan_dong\AppData\Local\Continuum\Anaconda2\lib\site-packages\autograd\tracer.pyc in trace(start_node, fun, x)
      8     with trace_stack.new_trace() as t:
      9         start_box = new_box(x, t, start_node)
---> 10         end_box = fun(start_box)
     11         if isbox(end_box) and end_box._trace == start_box._trace:
     12             return end_box._value, end_box._node

C:\Users\alan_dong\AppData\Local\Continuum\Anaconda2\lib\site-packages\autograd\wrap_util.pyc in unary_f(x)
     13                 else:
     14                     subargs = subvals(args, zip(argnum, x))
---> 15                 return fun(*subargs, **kwargs)
     16             if isinstance(argnum, int):
     17                 x = args[argnum]

<ipython-input-507-e643ed94813b> in loss(x, t, y)
      6 
      7 def loss(x, t, y):
----> 8     y_hat = forward_model(x, t)
      9     return anp.mean( (y_hat - y)**2 ) # mean squared error loss
     10 

<ipython-input-507-e643ed94813b> in forward_model(x, t)
      2 
      3 def forward_model(x, t): # it's a rectangular box of width x and height 1/x centered at the origin
----> 4     y = anp.piecewise(t, [t < -x/2., (t >= -x/2.) & (t < x/2.), t >= x/2.], [0., 1/x, 0.])
      5     return y
      6 

C:\Users\alan_dong\AppData\Local\Continuum\Anaconda2\lib\site-packages\autograd\tracer.pyc in f_wrapped(*args, **kwargs)
     46             return new_box(ans, trace, node)
     47         else:
---> 48             return f_raw(*args, **kwargs)
     49     f_wrapped.fun = f_raw
     50     f_wrapped._is_autograd_primitive = True

C:\Users\alan_dong\AppData\Local\Continuum\Anaconda2\lib\site-packages\numpy\lib\function_base.pyc in piecewise(x, condlist, funclist, *args, **kw)    1347         item
= funclist[k]    1348         if not isinstance(item, collections.Callable):
-> 1349             y[condlist[k]] = item    1350         else:    1351             vals = x[condlist[k]]

ValueError: setting an array element with a sequence.

I believe it has to do with the funclist argument of numpy.piecewise. When I change the forward model (so that none of the functions depend on x) to 

y = anp.piecewise(t, [t < -x/2., (t >= -x/2.) & (t < x/2.), t >= x/2.], [0., 1., 0.])

the error goes away. Any ideas? Thanks!

Alan Dong
  • 1

1 Answers1

0

It seems numpy.piecewise is not supported by autograd. I ended up changing it to an implementation that uses numpy.select, which is computing every function over the entire time window instead of just the region where its condition is active. It seems inefficient, but I suppose the alternative is to write a custom autograd primitive...

Alan Dong
  • 1