0

I'm working with linear transformation in the form of Y=Q(X+A), where X is the input tensor and Y is the output, Q and A are two tensors to be learned. Q is an arbitrary tensor, therefore I can use nn.Linear. But A is a (differentiable) tensor that has some specific pattern, as a short example,

A = [[a0,a1,a2,a2,a2], 
     [a1,a0,a1,a2,a2],
     [a2,a1,a0,a1,a2],
     [a2,a2,a1,a0,a1],
     [a2,a2,a2,a1,a0]].

So I cannot define such a pattern in nn.Linear. Is there any way to define such a tensor in Pytorch?

Shai
  • 111,146
  • 38
  • 238
  • 371
Zouzou
  • 1

1 Answers1

1

This looks like a Toeplitz matrix. A possible implementation in PyTorch is:

def toeplitz(c, r):
    vals = torch.cat((r, c[1:].flip(0)))
    shape = len(c), len(r)
    i, j = torch.ones(*shape).nonzero().T
    return vals[j-i].reshape(*shape)

In your case with a0 as 0, a1 as 1 and a2 as 2:

>>> toeplitz(torch.tensor([0,1,2,2,2]), torch.tensor([0,1,2,2,2]))
tensor([[0, 1, 2, 2, 2],
        [1, 0, 1, 2, 2],
        [2, 1, 0, 1, 2],
        [2, 2, 1, 0, 1],
        [2, 2, 2, 1, 0]])

For a more detailed explanation refer to my other answer here.

Ivan
  • 34,531
  • 8
  • 55
  • 100
  • I would like to update all elements with the same name together during backpropagation. For example, after one step, all `a0` are still the same. – Zouzou Nov 02 '21 at 15:04
  • @Zouzou Good question, this answer will indeed make a copy of `a0`, `a1` , ... since indexing makes a copy. I don't believe you can do a selection and return a view on the original data. – Ivan Nov 02 '21 at 16:27