Broadcasting over two arrays with different shapes (Numpy-Python)

Question

Suppose I have the following arrays:

first_array = array([[1, 8, 3, 9, 2],
                     [2, 6, 4, 1, 9],
                     [4, 2, 12, 8, 16],
                     [5, 3, 7, 18, 21],
                     [6, 20, 4, 8, 24]])

So an array with shape (5, 5)

Now I have a second array, which is a slice of the first:

second_array = array([[1, 8, 3, 9, 2],
                     [2, 6, 4, 1, 9]])

An array with shape (2, 5).

Now I want to subtract every vector of the first array by the vectors of the second array subsequently(excluding- in the first array- the vector of the second array I'm using to subtract), element-wise. I want to this for every vector of the second array.

So I want to have as an output:

subtracted_array = array([[[1, -2, 1, -8, 7],
                           [3, -6, 9, -1, 14],
                           [4, -5, 4, 9, 19],
                           [5, 12, 1, -1, 22]],
                          [[-1, 2, -1, 8, -7],
                           [2, -4, 8, 7, 7],
                           [3, -3, 3, 17, 12],
                           [4, 14, 0, 7, 15]]])

So this is an array with shape (2, 4, 5)

How do we do this with broadcasting?

Please add a [Minimal reproducible example](https://stackoverflow.com/help/minimal-reproducible-example) so that we can help you more. — yann ziselman, Jun 21 '21 at 12:37
Your example doesn't make sense. What are you subtracting off when you reach the end of the second array? — Mad Physicist, Jun 21 '21 at 12:57
Please show an example with arrays shaped (6, 3) and (3, 3) or something like that, and generate the expected output by hand — Mad Physicist, Jun 21 '21 at 12:57
I will edit it. But give me some time to do it, instead of closing to vote immediately. — Steven01123581321, Jun 21 '21 at 12:59
The close vote reflects the current state of the question. You were provided [ask] instructions while you were asking it. Once you fix the question to make sense, I will be more than happy to reverse my votes. — Mad Physicist, Jun 21 '21 at 13:02

yann ziselman · Accepted Answer · 2021-06-21T14:40:38.490

Does this code do what you meant you needed done?
You are welcomed to test it on your test case and update me if you need more help.

import numpy as np
arr = np.arange(50).reshape(10, 5)
arr_slice = arr[:2, :]

# "outer" tensor subtraction
arr_sub = arr_slice[:, None, :] - arr[None, :, :]
# create an index map of the vector combinations
idx0, idx1 = np.mgrid[:2, :10]
idx0, idx1 = idx0.flatten(), idx1.flatten()
# find the irrelevent combinations and remove them
remove_same = (1 - (idx0 == idx1)).astype(bool)
idx0, idx1 = idx0[remove_same], idx1[remove_same]

arr_sub = arr_sub[idx0, idx1, :].reshape(2, 9, 5)

EDIT: here's a more efficient method:

import numpy as np
arr = np.arange(50).reshape(10, 5)
arr_slice = arr[:2, :]

# create an index map of the vector combinations
idx0, idx1 = np.mgrid[:2, :10]
idx0, idx1 = idx0.flatten(), idx1.flatten()
# find the irrelevent combinations and remove them
remove_same = (1 - (idx0 == idx1)).astype(bool)
idx0, idx1 = idx0[remove_same], idx1[remove_same]

arr_sub = arr_slice[idx0, :] - arr[idx1, :]

arr_sub = arr_sub.reshape(2, 9, 5)

output:

arr = 
[[ 0  1  2  3  4]
 [ 5  6  7  8  9]
 [10 11 12 13 14]
 [15 16 17 18 19]
 [20 21 22 23 24]
 [25 26 27 28 29]
 [30 31 32 33 34]
 [35 36 37 38 39]
 [40 41 42 43 44]
 [45 46 47 48 49]]

arr_slice = 
[[0 1 2 3 4]
 [5 6 7 8 9]]

arr_sub = 
[[[ -5  -5  -5  -5  -5]
  [-10 -10 -10 -10 -10]
  [-15 -15 -15 -15 -15]
  [-20 -20 -20 -20 -20]
  [-25 -25 -25 -25 -25]
  [-30 -30 -30 -30 -30]
  [-35 -35 -35 -35 -35]
  [-40 -40 -40 -40 -40]
  [-45 -45 -45 -45 -45]]

 [[  5   5   5   5   5]
  [ -5  -5  -5  -5  -5]
  [-10 -10 -10 -10 -10]
  [-15 -15 -15 -15 -15]
  [-20 -20 -20 -20 -20]
  [-25 -25 -25 -25 -25]
  [-30 -30 -30 -30 -30]
  [-35 -35 -35 -35 -35]
  [-40 -40 -40 -40 -40]]]

EDIT 2: I subtracted the arrays in the wrong order (the whole array from the slice instead of the other way around). So here's a fix and this time I'm using your example:

import numpy as np

first_array = np.array(
    [[1, 8, 3, 9, 2],
     [2, 6, 4, 1, 9],
     [4, 2, 12, 8, 16],
     [5, 3, 7, 18, 21],
     [6, 20, 4, 8, 24]]
)

arr_slice = slice(None, 2)
second_array = first_array[arr_slice]

expected_subtracted_array = np.array(
    [[[[1, -2, 1, -8, 7],
       [3, -6, 9, -1, 14],
       [4, -5, 4, 9, 19],
       [5, 12, 1, -1, 22]],
      [[-1, 2, -1, 8, -7],
       [2, -4, 8, 7, 7],
       [3, -3, 3, 17, 12],
       [4, 14, 0, 7, 15]]]]
)

# create an index map of the vector combinations
idx0, idx1 = np.mgrid[:second_array.shape[0], :first_array.shape[0]]
idx0, idx1 = idx0.flatten(), idx1.flatten()
# find the irrelevent combinations and remove them
remove_same = (1 - (idx0 == idx1)).astype(bool)
idx0, idx1 = idx0[remove_same], idx1[remove_same]
arr_sub = first_array[idx1, :] - second_array[idx0, :]
arr_sub = arr_sub.reshape(second_array.shape[0], first_array.shape[0]-1, first_array.shape[1])

(arr_sub == expected_subtracted_array).all()

Output:

True

Computing a bunch of zeros is wasteful. Will post answer momentarily. — Mad Physicist, Jun 21 '21 at 12:54
@MadPhysicist, you're right. I added a slightly more efficient method — yann ziselman, Jun 21 '21 at 12:59
yes, I wrongly phrased the question in the beginning. Adjusted your code to what you have edited right now and it works indeed. Brilliant. Upvoted and accepted your answer. — Steven01123581321, Jun 21 '21 at 14:50

Broadcasting over two arrays with different shapes (Numpy-Python)

1 Answers1