Python numpy computing out matrix with shape 3,3,3 from input matrecies with shape 3,3

Question

I am currently building NeuralNetwork in python only using numpy.

This is the layout of the problem area:

I have one array holding the values for the input neurons in the columns and the rows represent the different training data points. It is of shape 3, 3:

in_a
array([['t_1a_1', 't_1a_2', 't_1a_3'],
       ['t_2a_1', 't_2a_2', 't_2a_3'],
       ['t_3a_1', 't_3a_2', 't_3a_3']], dtype='<U6')

Then I have an array for the weights, in where the columns are the connections going to the output1, 2 and 3 and the rows are the connections starting from 1, 2 and 3. It also has the shape 3, 3:

in_w
array([['w_11', 'w_12', 'w_13'],
       ['w_21', 'w_22', 'w_23'],
       ['w_31', 'w_32', 'w_33']], dtype='<U4')

Now I want to compute a matrix out with shape 3, 3, 3. That looks as follows:

out
array([[['t_1*a_1*w_11', 't_1*a_1*w_12', 't_1*a_1*w_13'],
        ['t_1*a_2*w_21', 't_1*a_2*w_22', 't_1*a_2*w_23'],
        ['t_1*a_3*w_31', 't_1*a_2*w_32', 't_1*a_2*w_33']],

       [['t_2*a_1*w_11', 't_2*a_1*w_12', 't_2*a_1*w_13'],
        ['t_2*a_2*w_21', 't_2*a_2*w_22', 't_2*a_2*w_23'],
        ['t_2*a_3*w_31', 't_2*a_2*w_32', 't_2*a_2*w_33']],

       [['t_3*a_1*w_11', 't_3*a_1*w_12', 't_3*a_1*w_13'],
        ['t_3*a_2*w_21', 't_3*a_2*w_22', 't_3*a_2*w_23'],
        ['t_3*a_3*w_31', 't_3*a_2*w_32', 't_3*a_3*w_33']]], dtype='<U12')

I tried numpy.dot, simple * multiplication, @ combination but nothing worked. I think a solution might be numpy.einsum or numpy.tensordot but I could not wrap my head around them. Does anybody know how to compute the out matrix based on the in matrices or can recommend a method and explanation? Thanks for you help

looks like an outer productthat can be done with `broadcasting`. It's not a sum-of-products that `dot` does. — hpaulj, Nov 13 '21 at 00:31
I tried np.outer(in_a, in_w) but this gives me an array of size 81 which is way too large, I think I need a function that can multiply the three variables t a and w together. How do you think broadcasting could help with that? — Phönix 64, Nov 13 '21 at 00:44
I'm getting lost in all the indices above. Could you tell me what the value of `out[i, j, k]` is in terms of `in_a` and `in_w`. I suspect you're just going to need something like `in_a[:,:,None] * in_w[None,:,:]`, but I'm a little bit uncertain which dimensions are being broadcast, and I probably have the Nones in the wrong place. — Frank Yellin, Nov 13 '21 at 00:46
`np.outer` isn't general enough.. It's a good idea to read its docs before trying to use it. — hpaulj, Nov 13 '21 at 01:33
Often it is easier to reason about dimensions like this when the dimensions differ; your example is 3 on all. But if `a` is (2,3)` and `w` is (3,4), the "outer' product to (2,3,4) almost jumps out at us. We need to multiply a (2,3,1) and (1,3,4), with `broadcasting` scaling the size 1 dimensions to match. — hpaulj, Nov 13 '21 at 03:00

Reti43 · Accepted Answer · 2021-11-13T01:19:10.567

All you need is

in_a[...,None] * in_w

If you think about this in_a has shape (training_sets, input_neurons) and in_w (input_neurons, output_neurons). And your output seems to be an element-wise multiplication of

    (T, I)       # in_a
*      (I, O)    # in_w

Let's demonstrate this for fun

class Variable:
    def __init__(self, name):
        self.name = name

    def __mul__(self, other):
        if not isinstance(other, Variable):
            raise ValueError
        return Variable(f'{self.name}*{other.name}')

    def __repr__(self):
        return self.name


def generate_array(fmt, rows, columns):
    return np.array([[Variable(fmt.format(i, j)) for j in range(1, columns+1)] 
                     for i in range(1, rows+1)])

in_a = generate_array('t_{}a_{}', 3, 3)
in_w = generate_array('ww_{}{}', 3, 3)
print(in_a[...,None] * in_w)

Which prints

[[[t_1a_1*ww_11 t_1a_1*ww_12 t_1a_1*ww_13]
  [t_1a_2*ww_21 t_1a_2*ww_22 t_1a_2*ww_23]
  [t_1a_3*ww_31 t_1a_3*ww_32 t_1a_3*ww_33]]

 [[t_2a_1*ww_11 t_2a_1*ww_12 t_2a_1*ww_13]
  [t_2a_2*ww_21 t_2a_2*ww_22 t_2a_2*ww_23]
  [t_2a_3*ww_31 t_2a_3*ww_32 t_2a_3*ww_33]]

 [[t_3a_1*ww_11 t_3a_1*ww_12 t_3a_1*ww_13]
  [t_3a_2*ww_21 t_3a_2*ww_22 t_3a_2*ww_23]
  [t_3a_3*ww_31 t_3a_3*ww_32 t_3a_3*ww_33]]]

Python numpy computing out matrix with shape 3,3,3 from input matrecies with shape 3,3

1 Answers1