passing Array[Byte] from ironpython to python using pickle

Question

I recently started programming macros for a microscope that is controlled by ZEN Blue from Zeiss. The Zeiss macro environment uses IronPython 2.7.2.1. I need to write out a hdf5 file, but unfortunately hdf5 support for IronPython is quite bad.

The data I want to transfer is of type Array[Byte] which seems to be somehow connected to the CLR of .NET:

print "length: %i type: %s" %(len(data), type(data))

gives:

length: 4915200 type: <type 'Array[Byte]'>

I can sucessfully transfer a dict containing ints and Strings to a python server using pickle via a socket connection (suggested here). However, when I try to unpickle the Array[Byte] data, python wants to import the CLR, which of course fails:

ImportError: No module named clr

Question: What is the best (fastest) way to convert the .NET Array[Byte] to a basic python type/object which is not linked to the .NET clr?

many thanks, Dominic

Answer 1

After some testing, the most efficient solution that I found works like this:

First, convert Array[Byte] to a python string:

data_str = str(buffer(data))

I'm not entirely sure what buffer does, but it seems to be necessary for efficient computation, some explanation here.

Then send to cpython (in my case via a socket, cpython runs on a linux box) and convert to a tuple:

#data_str is an RGB image with dims sx and sy
format_str = str(sx*sy*3).strip() + 'B' 
im_data = struct.unpack(format_str, data_str) #conversion to tuple

Finally, use numpy and h5py to write the .h5 file:

#put into np.array
im = np.asarray(im_data, dtype=np.uint8) #this is the most Time consuming part
newshape = (3, sx, sy)
im = np.reshape(im, newshape, order='F')

#write out
f = h5py.File('zenimage_bgr24.h5', 'w')
f.create_dataset('/im', data = im, dtype='uint8')
f.close()