Defining empty numpy array when we do not know the size

Question

From my understanding, when we want to define a numpy array, we have to define its size.

However, in my case, I want to define a numpy array and then extend it based on my values in the for loop. The shape of values might differ in each run. So I cannot define the numpy array shape in advance.

Is there any way to overcome this?

I would like to avoid using lists.

Thanks

NumPy arrays cannot be created without size. You can start with a small (1-element?) array and then add items to it. — DYZ, Oct 19 '21 at 05:01
Sorry that's not how we do things in the numpy. Lists are for growing. Arrays have a fixed size. — hpaulj, Oct 19 '21 at 05:25
If you know the range for the `for` loop, you know the target array size. — hpaulj, Oct 19 '21 at 05:38
Why do you want to avoid lists? Growing a list with append is a lot faster than growing an array by making a new one, and copying data from the old. You apparently have worked with MATLAB. There you can grow a matrix by indexing `M[i,j]=0`, but I suspect that's doing a whole lot of work in the background. It's not something I routinely did when I worked with MATLAB years ago. — hpaulj, Oct 19 '21 at 07:03

score 1 · Answer 1 · answered Oct 19 '21 at 05:06

1

I think numpy array is just like array in clang or c++, I mean when you make numpy array you allocate memory depend on your request(size and dtype). So it is better to make array after the size of array is determinated.

Or you can try numpy.append https://numpy.org/doc/stable/reference/generated/numpy.append.html But I don't think it is preferable way because it keeps generate new arrays.

answered Oct 19 '21 at 05:06

TaekYoung

45
7

1

I hate `np.append` with a passion. It is not a list append clone. – hpaulj Oct 19 '21 at 05:36

score 1 · Accepted Answer · answered Oct 19 '21 at 05:19

1

import numpy as np  
myArrayShape = 2
myArray = np.empty(shape=2)

Note that this generates random values for each element in the array.

answered Oct 19 '21 at 05:19

InfoDaneMent

326
3
18

But `empty` here does not mean the same thing as the emlty list, `[]` – hpaulj Oct 19 '21 at 05:33

score 0 · Answer 3 · answered Oct 19 '21 at 07:19

From the Octave (free-MATLAB) docs, https://octave.org/doc/v6.3.0/Advanced-Indexing.html

In cases where a loop cannot be avoided, or a number of values must be combined to form a larger matrix, it is generally faster to set the size of the matrix first (pre-allocate storage), and then insert elements using indexing commands. For example, given a matrix a,

[nr, nc] = size (a);
x = zeros (nr, n * nc);
for i = 1:n
  x(:,(i-1)*nc+1:i*nc) = a;
endfor
is considerably faster than

x = a;
for i = 1:n-1
  x = [x, a];
endfor
because Octave does not have to repeatedly resize the intermediate result.

The same idea applies in numpy. While you can start with a (0,n) shaped array, and grow by concatenating (1,n) arrays, that is a lot slower than starting with a (m,n) array, and assigning values.

There's a deleted answer that illustrates how to create an array by list append. That is highly recommended.

Defining empty numpy array when we do not know the size

3 Answers3