0

Sorry if this better belongs on the page of my previous question: np.savetxt triggers ValueError. Why? but I don't think I'm going to get any more replies there at this point.

The allow_pickle suggestions still don't work, so what I can try instead?

Also, what's a pickle? I've only ever known it as a type of food.

Thanks

num3ri
  • 822
  • 16
  • 20
J.D.
  • 139
  • 4
  • 14
  • I suspect there's a problem with how you did the `save` or `savez`. `eig` returns two arrays, a 1d and 2d. You have to save them correctly. `numpy` `save` and python `pickle` work together. – hpaulj Aug 03 '19 at 08:13
  • When you ask, show the exact `save` command, so there's no confusion. – hpaulj Aug 03 '19 at 08:25
  • Can't seem to get multiline code to appear correctly in comments for some reason. Anyway, 1st line is: eigs=np.linalg.eig(P@K@P). 2nd line is: np.savez_compressed('eigs.npz',eigs,allow_pickle=True). P and K are 10000x10000 matrices that are calculated earlier. Thanks – J.D. Aug 03 '19 at 08:26
  • Save `*eigs`, not `eigs`. – hpaulj Aug 03 '19 at 08:29
  • Ok, trying np.savez_compressed('eigs.npz',*eigs) – J.D. Aug 03 '19 at 08:30
  • Well that appeared to work at first. I first executed the program with the savez command, and then commented out the save and put in the load command. That resulted in sensible looking 10000x10000 and 10000 arrays for eigvecs and eigvals in the Variable explorer. – J.D. Aug 03 '19 at 09:12
  • But then I tried to check with the following two lines: ```print(P@K@P@eigvecs[8721]) print(np.multiply(eigvals[8721],eigvecs[8721]))``` That threw up the following error: error: Error -3 while decompressing data: invalid bit length repeat – J.D. Aug 03 '19 at 09:14

1 Answers1

0

Pickle is a serialization technique converting python objects into byte streams. Pickles are most commonly used for network communication in a distributed environment. For example,if you have a cluster and you want to pass a specific python object from one machine to another , the most common way to do this is by pickling the object,pass it through the network and then unpickling it on the other machine. They can also be used as a compression technique for efficiently saving a huge python object on your local fs. For using pickles you have to import the library

    import pickle.py

and then allow the program to read/write pickled files

    allow_pickle = true
pandy giankoulidis
  • 174
  • 2
  • 6
  • Tried importing pickle. Got ModuleNotFoundError: No module named 'pickle.py'; 'pickle' is not a package – J.D. Aug 03 '19 at 07:44
  • Using ```import pickle``` instead of ```import pickle.py``` fixes that issue. Unfortunately, when I then try to write the data like this: ```eigs=np.linalg.eig(P@K@P) np.savez_compressed('eigs.npz',eigs,allow_pickle=True)``` I get: ValueError: could not broadcast input array from shape (10000,10000) into shape (10000) – J.D. Aug 03 '19 at 08:05
  • The `pickle` format for a `numpy` `ndarray` is the same one used by `np.save`. Conversely, when a `ndarray` contains non-numeric objects, it uses their `pickle` to serialize them. The two methods are inter related if not interchangeable. – hpaulj Aug 03 '19 at 08:27