Custom op backward

Question

I am writing a custom op, and I got stucked when writing the backward part.

When I call out_grad[0].asnumpy() or do any manipulation of the out_grad, the program crash without any error message.

I tried fill the in_grad with zeros, the program run smoothly, but I need the grad to flow backward. def backward(self, req, out_grad, in_data, out_data, in_grad, aux): self.assign(in_grad[0], req[0], 0) self.assign(in_grad[1], req[1], 0)

What's going wrong here?

I use '.shape' to get the out_grad's shape, it turn out to be empty. Maybe something went wrong in the upstream. But how can I figure it out? — Zehao Shi, Jun 19 '17 at 14:08
I have found what is going wrong, I have post my solution below. — Zehao Shi, Jun 21 '17 at 08:02

score 1 · Answer 1 · answered Jun 21 '17 at 08:01

Custom Operator in MXNet show us how to define a loss function using custom op. The loss op is very special because it doesn't need grad to be flow into.

But in my situation, I need grad to flow into my op. So, the function below should return the dependency instead of empty as in loss op.

def declare_backward_dependency(self, out_grad, in_data, out_data):
    return [out_grad[0]]

In my opinion, the dependency is some variable which the gradient should be delievered to.

What do you mean by "grad to be flow into"? – MonsieurBeilto Dec 13 '18 at 21:37 — MonsieurBeilto, Dec 13 '18 at 21:37

score 0 · Answer 2 · answered Jun 20 '17 at 22:57

0

Have you tried to follow the tutorial here for developing a Custom Operator in MXNet.

If that does not help, provide your full code of the Custom operator along with some sample data and a simple model with which this issue can be easily reproduced.

answered Jun 20 '17 at 22:57

Naveen Swamy

109
8

Yes. The tutorial show us how to define a loss function using custom op. The loss op is very special because it doesn't need grad to be flow into. I have found what is going wrong. I'm now posting my solution. – Zehao Shi Jun 21 '17 at 07:51

Custom op backward

2 Answers2