1

I'm starting out with numpy and was trying to figure out how its arrays work for column vectors. Defining the following:

x1 = np.array([3.0, 2.0, 1.0])
x2 = np.array([-2.0, 1.0, 0.0])

And calling 

print("inner product x1/x2: ", np.inner(x1, x2))

Produces inner product x1/x2: -4.0 as expected - this made me think that numpy assumes an array of this form is a column vector and, as part of the inner function, tranposes one of them to give a scalar. However, I wrote some code to test this idea and it gave some results that I don't understand.

After doing some googling about how to specify that an array is a column vector using .T I defined the following:

x = np.array([1, 0]).T
xT = np.array([1, 0])

Where I intended for x to be a column vector and xT to be a row vector. However, calling the following:

print(x)
print(x.shape)

print(xT)
print(xT.shape)

Produces this:

[1 0]
(2,)
[1 0]
(2,)

Which suggests the two arrays have the same dimensions, despite one being the transpose of the other. Furthermore, calling both np.inner(x,x) and np.inner(x,xT) produces the same result. Am I misunderstanding the .T function, or perhaps some fundamental feature of numpy/linear algebra? I don't feel like x & xT should be the same vector.

Finally, the reason I initially used .T was because trying to define a column vector as x = np.array([[1], [0]]) and calling print(np.inner(x, x)) produced the following as the inner product:

[[1 0]
 [0 0]]

Which is the output you'd expect to see for the outer product. Am I misusing this way of defining a column vector?

twigonometry
  • 166
  • 1
  • 9
  • numpy doesn't have column vectors. nor row vectors. `shape` is a prperty of all arrays. – hpaulj Oct 20 '19 at 18:53
  • You are not---defining a column vector, that is. All the vectors you are defining are 1D. In 1D there is no such thing as rows and columns. Transpose is a nop. Some functions / ops take 1D vectors as arguments where a row or column should be (most notably matrix-vector multiplication) but that is merely a convenience. – Paul Panzer Oct 20 '19 at 18:53
  • So are you saying that they're treated just as arrays until I call functions like inner and then numpy interprets it as a vector? If so, that explains why calling shape produces the same answer but not why np.inner produces the same answer (unless I've misunderstood your comment). And what about the second case where I define a 2D array? – twigonometry Oct 20 '19 at 19:04
  • The second case is I'd say a pecularity of `np.inner`. It sum reduces the last axis, the normal inner product, and outer iterates over all the others, hence as the docs say: `out.shape = a.shape[:-1] + b.shape[:-1]` – Paul Panzer Oct 20 '19 at 20:44

1 Answers1

1

Look at the inner docs:

Ordinary inner product of vectors for 1-D arrays 
...
np.inner(a, b) = sum(a[:]*b[:])

With your sample arrays:

In [374]: x1 = np.array([3.0, 2.0, 1.0]) 
     ...: x2 = np.array([-2.0, 1.0, 0.0])                                       
In [375]: x1*x2                                                                 
Out[375]: array([-6.,  2.,  0.])
In [376]: np.sum(x1*x2)                                                         
Out[376]: -4.0
In [377]: np.inner(x1,x2)                                                       
Out[377]: -4.0
In [378]: np.dot(x1,x2)                                                         
Out[378]: -4.0
In [379]: x1@x2                                                                 
Out[379]: -4.0

From the wiki for dot/scalar/inner product:

https://en.wikipedia.org/wiki/Dot_product

two equal-length sequences of numbers (usually coordinate vectors) and returns a single number

enter image description here

If vectors are identified with row matrices, the dot product can also 
be written as a matrix product

Coming from a linear algebra world, it easy to think of everything in terms of matrices (2d) and vectors, which are 1 row or 1 column matrices. MATLAB/Octave works in that framework. But numpy is more general, with arrays with 0 or more dimensions, not just 2.

np.transpose does not add dimensions, it just permutes the existing ones. Hence x1.T does not change anything.

A column vector can be made with np.array([[1], [0]]) or:

In [381]: x1                                                                    
Out[381]: array([3., 2., 1.])
In [382]: x1[:,None]                                                            
Out[382]: 
array([[3.],
       [2.],
       [1.]])
In [383]: x1.reshape(3,1)                                                       
Out[383]: 
array([[3.],
       [2.],
       [1.]])

np.inner describes what happens when the inputs not 1d, such as your 2d (2,1) shape x. It says it uses np.tensordot which is a generalization of np.dot, matrix product.

In [386]: x = np.array([[1],[0]])                                               
In [387]: x                                                                     
Out[387]: 
array([[1],
       [0]])
In [388]: np.inner(x,x)                                                         
Out[388]: 
array([[1, 0],
       [0, 0]])
In [389]: np.dot(x,x.T)                                                         
Out[389]: 
array([[1, 0],
       [0, 0]])
In [390]: x*x.T                                                                 
Out[390]: 
array([[1, 0],
       [0, 0]])

This is the elementwise product of (2,1) and (1,2) resulting in a (2,2), or outer product.

hpaulj
  • 221,503
  • 14
  • 230
  • 353
  • So it essentially 'ignores' the dimensions and multiplies elementwise? I'm guessing this is because inner is only defined for vectors, which makes sense - thanks. I'm still not sure why the second example behaves as if I'm performing the outer product, however. – twigonometry Oct 20 '19 at 19:25
  • Dimensions do matter in `inner`. For 2 1d arrays, the shape has to match. If one is (2,1) (your `x`), the other has to be (n,1). We don't use `inner` as much as `np.dot` or its cousins (`einsum` and `matmul`). There's an `np.outer`, but broadcasted multiplication is just as easy. – hpaulj Oct 20 '19 at 20:33