0

First I initialize a matrix called A with finite dimensions. While I try to assign values to matrix A as shown in the code, the process takes more and more RAM. I guess it is internally creating some temporary variables and not freeing up the memory once the assignment is done for a single iteration. Over time the process reaches the limit of my RAM (i.e. 32 GB) and my computer starts hanging. Can someone please comment on this problem? Is there a better way to do the following task? 

import numpy as np

A = np.zeros((10000, 200, 3000))

for i in range(A.shape[0]):
    print(i)
    A[i,:,:] = np.random.rand(A.shape[1], A.shape[2])
Nancy Mallika Nayak
  • 1
  • 1
  • Just as a test, what happens if you do `import gc` and add a `gc.collect()` in the loop body? Also, can you provide the _actual_ code that exhibits the problem? – Matteo Italia Oct 05 '19 at 05:41
  • 1
    What is (lowercase) `a`? – Klaus D. Oct 05 '19 at 05:41
  • @KlausD. That was by mistake. Sorry. – Nancy Mallika Nayak Oct 05 '19 at 05:48
  • @MatteoItalia Hi, Thanks for responding. After some basic debugging of my code I can say that the problem of my actual code can be exactly explained by the given code. If u check RAM usage, you can see over time it keeps increasing while the given program runs. And I checked with gc. It is not helping. – Nancy Mallika Nayak Oct 05 '19 at 06:02
  • 1
    An array with 10000×200×3000 doubles will take 47GB. If you really need that many values you need significantly more RAM. At least 94GB because you will often have two of them. – MSeifert Oct 05 '19 at 07:24
  • ok @MSeifert. So there is no other way to do this kind of assignment if I have less than 47Gb RAM. – Nancy Mallika Nayak Oct 06 '19 at 08:53

0 Answers0