1

I have a tensor of shape (number_of rays, number_of_points_per_ray, 3), let’s call it input. input is passed through a model and some processing (all of this is differentiable), let’s call this process inference. Finally, we get output = inference(input), which has a shape of (number_of_rays, number_of_points_per_ray, 300), where each “ray” in the output only depends on the same ray of the input. e.g. output[i] only depends on input[i]. This means that for each set of 3 elements on the input the output has 300 elements, so I would expect to get a gradient with the same shape as the output

As explained at https://discuss.pytorch.org/t/need-help-computing-gradient-of-the-output-with-respect-to-the-input/150950/5 , I tried grads=torch.autograd.grad (outputs = output, inputs = input, grad_outputs = None)

but the output I am getting is of shape (number_of rays, number_of_points_per_ray, 3) , which is the same as the input and not the same as the output.

Any clue what may I be doing wrong? Thanks in advance

malfonsoarquimea
  • 65
  • 4

1 Answers1

0

I am assuming that 3 in input is the size of the state that you forward to your model network, and 300 is the size of the output that your model network produces.

Now, you want to call a separate instance of your model network for each element in (number_of_rays)? If yes, then one way of getting the gradients for each element in your array, is to assign a separate optimizer to each instance of the model network that you assign to your array elements.

Here is my code:

class Environment():
    def __init__(self, N, input_dims, output_dims, lr, gamma):
        self.number_of_rays = N
        self.input_dims = input_dims
        self.output_dims = output_dims
        # assign to each ray a neural network
        self.rays = [YourModel(input_dims, output_dims, gamma) for i in range(N)]
        self.optimizer = []
        for i in range(N):
            optim = T.optim.Adam(self.rays[i].parameters(), lr=lr, betas=(0.92, 0.999))
            self.optimizer.append(optim)

    def run(self):
        # here you do your stuff 
        # ...

        # now update the networks
        for i in range(self.number_of_rays):
            prediction = self.rays[i](observation) # feed your input state to the model
            loss = loss_fn(prediction) # compute loss
            self.optimizer[i].zero_grad() # reset the gradients of model parameters
            loss.backward() # backpropagate the prediction loss
            self.optimizer[i].step() # adjust the parameters by the gradients collected in the backward pass
Ruslan Mukhamadiarov
  • 63
  • 1
  • 8