1

I'm trying to load an io bytes stream into a numpy array. np.frombuffer() returns an array with too many elements and np.load() throws the error "UnicodeDecodeError: 'utf-8' codec can't decode byte 0x93 in position 0: invalid start byte"

Can anyone explain to me what is happening here? Thanks!

import io
import numpy as np

# Set up np array with shape (3, 2)
a = np.array([[1.5, 2], [3, 4], [5, 6]])

# Save to stream
stream = io.BytesIO()  
np.save(stream, a)  



# Read the same stream back with np.frombuffer()
b = np.frombuffer(stream.getvalue())
print('b.shape: ' + str(b.shape))

# np load throws error:
# 'utf-8' codec can't decode byte 0x93 in position 0: invalid start byte
c = np.load(stream.getvalue())

(I'm doing this to read npy-files stored in Azure blob storage. The Azure blob SDK provides a byte stream equivalent to the one in the example above.)

nallekavn
  • 11
  • 2
  • 1
    I'm not sure what's going on, but `pickle` might be easier to use. The underlying encoding is still `np.save/load`, but the result is a string. – hpaulj Jul 24 '20 at 16:18

0 Answers0