13

In Python 3 I am importing several data files in a loop, and I would like to be able to store all the data in a single 2-dimensional array. I start with something like data = np.array([]) and on each iteration i want to add a new array datai = np.array([1,2,3]), how can I get my final array to look like this? [[1,2,3],[1,2,3],...,[1,2,3]]

I have tried np.append, np.concatenate, and np.stack, but none seem to work. An example code that I'm trying:

data = np.array([])
for i in range(datalen):
    datai = *func to load data as array*
    data = np.append(data, datai)

but of course this returns a flattened array. Is there any way I can get back a 2-dimensional array of length datalen with each element being the array datai?

Thanks!

Vicente Esnaola
  • 163
  • 1
  • 2
  • 5
  • 1
    Looks like you want `vstack`. Are all of your subarrays the same length? – user3483203 Aug 03 '18 at 16:11
  • yes! Ok i can get that to work for all iterations after the first, but since I start with an empty array it gives me a dimension error. I can just throw in an if statement to save the first array and vstack from then forward, but is there any cleaner way to do it? – Vicente Esnaola Aug 03 '18 at 16:26
  • 1
    Look at the `np.append` docs. What does it say about omitting the `axis` parameter? – hpaulj Aug 03 '18 at 16:28
  • You are trying to use a list append model for arrays. `np.empty([])` is not the same as the list `[]`. Look at its `shape` and `ndim`. For `concatenate` to work, the inputs have to have matching dimensions. `data1` is 1d (3,). It can be joined on axis 0 to another 1d array. To join on a new initial axis it needs to be 2d (which is what `vstack` addresses). In any case repeated list append is the right way, not a repeated concatenate. – hpaulj Aug 03 '18 at 16:45
  • What's the shape of `data1`? How do you load it? Often, when loaded from a txt file, the resulting array will be 2d. It could of course be 1d. – hpaulj Aug 03 '18 at 17:20
  • is there an explanation of this insane behaviour anywhere? (the flattening that is) – Emile Jan 07 '20 at 18:18

5 Answers5

5

The fastest way would be vstack

data = np.vstack((get_data() for i in range(datalen)))

vstack requires a tuple/iterable

data = np.vstack((data1, data2, data3))

or you can do this by appending with axis=0

data = np.empty(shape=(0, 3))
data = np.append(data, datai.reshape((-1, 3)), axis=0)  # -1 will make the rows automatic
justengel
  • 6,132
  • 4
  • 26
  • 42
1

You can reshape your array using np.reshape like this

flattened_array = np.array([1,2,3,1,2,3,1,2,3])

wanted_array = np.reshape(flattened_array, (-1, 3))

This would result in

[[1, 2, 3],[1, 2, 3],[1, 2, 3]]

Khaled Khazem
  • 23
  • 3
0

Solution 1 using list comprehensions:

data = []
datalen = 4
datai = range(1,4)
data = [list(datai) for _ in range(datalen)]
print (data)

Output

[[1, 2, 3], [1, 2, 3], [1, 2, 3], [1, 2, 3]]

Solution 2 (just a bit lengthy)

data = []
datalen = 4

for i in range(datalen):
    datai = range(1,4)
    data.append(list(datai))
print (data)

with the same output as above. In the second method you can also just simply use data.append(list(range(1,4))). You can choose if you want to convert datai to list or not. If you want the final output as an array, you can just use np.array()

Sheldore
  • 37,862
  • 7
  • 57
  • 71
0

You can try this-

data = np.zeros(shape=(datalen,len(datai))
for i in range(datalen):
   data[i] = datai
Kaustumbh Jaiswal
  • 1
  • 1
0

It's called numpy.tile.

From the official docs:

>>> c = np.array([1,2,3,4])
>>> np.tile(c,(3,1))
array([[1, 2, 3, 4],
       [1, 2, 3, 4],
       [1, 2, 3, 4]])

so, for your datai do np.tile(datai,(N_repeats,1))

Ufos
  • 3,083
  • 2
  • 32
  • 36