Concatenating empty array in Numpy

Question

in Matlab I do this:

>> E = [];
>> A = [1 2 3 4 5; 10 20 30 40 50];
>> E = [E ; A]

E =

     1     2     3     4     5
    10    20    30    40    50

Now I want the same thing in Numpy but I have problems, look at this:

>>> E = array([],dtype=int)
>>> E
array([], dtype=int64)
>>> A = array([[1,2,3,4,5],[10,20,30,40,50]])

>>> E = vstack((E,A))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/shape_base.py", line 226, in vstack
    return _nx.concatenate(map(atleast_2d,tup),0)
ValueError: array dimensions must agree except for d_0

I have a similar situation when I do this with:

>>> E = concatenate((E,A),axis=0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: arrays must have same number of dimensions

Or:

>>> E = append([E],[A],axis=0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/lib/function_base.py", line 3577, in append
    return concatenate((arr, values), axis=axis)
ValueError: arrays must have same number of dimensions

In MATLAB arrays are always 2D or larger. In numpy they can have 1 or even 0 dimensions. — hpaulj, Mar 29 '14 at 18:46

behzad.nouri · Accepted Answer · 2014-03-29T15:23:16.403

132

if you know the number of columns before hand:

>>> xs = np.array([[1,2,3,4,5],[10,20,30,40,50]])
>>> ys = np.array([], dtype=np.int64).reshape(0,5)
>>> ys
array([], shape=(0, 5), dtype=int64)
>>> np.vstack([ys, xs])
array([[  1.,   2.,   3.,   4.,   5.],
       [ 10.,  20.,  30.,  40.,  50.]])

if not:

>>> ys = np.array([])
>>> ys = np.vstack([ys, xs]) if ys.size else xs
array([[ 1,  2,  3,  4,  5],
       [10, 20, 30, 40, 50]])

edited Mar 29 '14 at 15:23

answered Mar 29 '14 at 15:14

behzad.nouri

74,723
18
126
124

I dont understand how the second implementation works. It looks like ys.size should evaluate to False and then the statement is just ys=xs. – Nirvedh Meshram Dec 25 '18 at 20:31
3

@NirvedhMeshram The second statement says that if `ys.size` is a value, any value that is not zero or False, then execute the statement. – m13op22 Apr 04 '19 at 18:52
@NirvedhMeshram I think you're right. Logically, the second implementation doesn't need the first statement. It can just check if `ys` already exists: `ys = np.vstack([ys, xs]) if 'ys' in vars() else xs`. – flow2k Oct 04 '19 at 07:01
Also, I think `vstack` can be replaced with `concatenate` (and the use of `()` correspondingly); does anyone think otherwise? If I'm right - the essence of this method is to use `reshape` on an empty array. – flow2k Oct 04 '19 at 07:04
1

This whole idea of creating a 0 element array just so you can `vstack` is wrong. As a one time operation it is useless, and as part of a repeated `vstack` it is inefficient. Stick with list append. Don't slavishly imitate MATLAB. – hpaulj Jun 17 '20 at 16:45
@hpaulj, namely, code that uses list append, and then produces an np.array as above in the end – WurmD Sep 01 '20 at 11:10

score 4 · Answer 2 · answered Sep 01 '20 at 11:47

4

In Python, if possible to work with the individual vectors, to append you should use list.append()

>>> E = []
>>> B = np.array([1,2,3,4,5])
>>> C = np.array([10,20,30,40,50])
>>> E = E.append(B)
>>> E = E.append(C)
[array([1, 2, 3, 4, 5]), array([10, 20, 30, 40, 50])]

and then after all append operations are done, return to np.array thusly

>>> E = np.array(E)
array([[ 1,  2,  3,  4,  5],
   [10, 20, 30, 40, 50]])

answered Sep 01 '20 at 11:47

WurmD

1,231
5
21
42

Do you know if this is efficient? – lpnorm Jan 27 '22 at 16:58

score 3 · Answer 3 · edited Mar 18 '19 at 18:12

3

If you wanna do this just because you cannot concatenate an array with an initialized empty array in a loop, then just use a conditional statement, e.g.

if (i == 0): 
   do the first assignment
else:  
   start your contactenate

edited Mar 18 '19 at 18:12

assembler

3,098
12
43
84

answered Mar 18 '19 at 17:22

Ming

706
6
8

YoniChechik · Answer 4 · 2018-12-25T20:03:46.727

Something that I've build to deal with this sort of problem. It's also deals with list input instead of np.array:

import numpy as np


def cat(tupleOfArrays, axis=0):
    # deals with problems of concating empty arrays
    # also gives better error massages

    # first check that the input is correct
    assert isinstance(tupleOfArrays, tuple), 'first var should be tuple of arrays'

    firstFlag = True
    res = np.array([])

    # run over each element in tuple
    for i in range(len(tupleOfArrays)):
        x = tupleOfArrays[i]
        if len(x) > 0:  # if an empty array\list - skip
            if isinstance(x, list):  # all should be ndarray
                x = np.array(x)
            if x.ndim == 1:  # easier to concat 2d arrays
                x = x.reshape((1, -1))
            if firstFlag:  # for the first non empty array, just swich the empty res array with it
                res = x
                firstFlag = False
            else:  # actual concatination

                # first check that concat dims are good
                if axis == 0:
                    assert res.shape[1] == x.shape[1], "Error concating vertically element index " + str(i) + \
                                                       " with prior elements: given mat shapes are " + \
                                                       str(res.shape) + " & " + str(x.shape)
                else:  # axis == 1:
                    assert res.shape[0] == x.shape[0], "Error concating horizontally element index " + str(i) + \
                                                       " with prior elements: given mat shapes are " + \
                                                       str(res.shape) + " & " + str(x.shape)

                res = np.concatenate((res, x), axis=axis)
    return res


if __name__ == "__main__":
    print(cat((np.array([]), [])))
    print(cat((np.array([1, 2, 3]), np.array([]), [1, 3, 54+1j]), axis=0))
    print(cat((np.array([[1, 2, 3]]).T, np.array([]), np.array([[1, 3, 54+1j]]).T), axis=1))
    print(cat((np.array([[1, 2, 3]]).T, np.array([]), np.array([[3, 54]]).T), axis=1))  # a bad one

score 1 · Answer 5 · answered Oct 14 '21 at 10:15

A solution is to use the None object and np.concatenate, np.hstack or np.vstack.

>>> arr=None
>>> p=np.array([0,1,2,3])

>>> for i in range(0,2):
>>>     arr = (np.vstack((arr, p)) if (arr is not None) else p)

array([[ 0, 1, 2, 3],
      [[ 0, 1, 2, 3]])

score 0 · Answer 6 · answered Jan 01 '21 at 01:25

E = np.array([
    
]).reshape(0, 5)
print("E: \n{}\nShape {}\n".format(E, E.shape))

A = np.vstack([
    [1, 2, 3, 4, 5], 
    [10, 20, 30, 40, 50]]
)
print("A:\n{}\nShape {}\n".format(A, A.shape))

C = np.r_[
    E, 
    A
].astype(np.int32)

print("C:\n{}\nShape {}\n".format(C, C.shape))

E: 
[]
Shape (0, 5)

A:
[[ 1  2  3  4  5]
 [10 20 30 40 50]]
Shape (2, 5)

C:
[[ 1  2  3  4  5]
 [10 20 30 40 50]]
Shape (2, 5)

score -8 · Answer 7 · answered Mar 29 '14 at 20:26

-8

np.concatenate, np.hstack and np.vstack will do what you want. Note however that NumPy arrays are not suitable for use as dynamic arrays. Use Python lists for that purpose instead.

answered Mar 29 '14 at 20:26

Sturla Molden

1,124
7
7

Concatenating empty array in Numpy

7 Answers7

Linked