Why do values in my 3D numpy array change when I write it to file?

Question

strange problem in that I have a 3D array full of labels (lets say from 1-36) called labelled_stack. These are just areas where the value in the array is equal to the label given. A quick 2D example with 5 labels would be something like:

labelled_stack = (0 0 0 0 0 0 0 0 0 0)
                 (0 1 1 0 0 0 2 2 2 0)
                 (0 1 0 0 3 0 0 2 2 0)
                 (0 0 0 3 3 0 0 0 0 0)
                 (0 4 0 0 3 0 0 0 5 0)
                 (0 4 0 0 0 0 5 5 5 0)
                 (0 0 0 0 0 0 0 0 0 0)

But imagine its a numpy array...

I have tried using cv2.imwrite and scipy.misc.imsave to save the stack, but when i do this and then open them, their values have changed so that the values = 5, now equal 255, and the values equal to 1, equal 51 etc. This is not acceptable. I need the values to stay as they are.. increasing values of integers in steps of 1. i know cv2 and scipy.misc both write as 8bit images but that shouldnt mean I get this error.

I have even re-read the images back into python to check this is going on and it is.

My labelled_stack is of type np.uint32

Edit to include save commands:

for j in np.arange(0,l,1):
     misc.imsave(save_path+Folder+'/Labelled/slice_{:04d}of{:}.tif'.format(j+1,l),labelled_stack[:,:,j])

or...

cv2.imwrite(save_path+Folder+'/Labelled/slice_{:04d}of{:}.tif'.format(j+1,l),labelled_stack[:,:,j])

Why not use a numpy method to save the numpy array? Something like `savetxt` can handle just outputting a numpy array to a file. — zephyr, May 27 '16 at 13:06
Can you show how you call imwrite ? And posting the image you output could be useful too. — Sunreef, May 27 '16 at 13:09
Sorry I should have been clear, I need the image in a tif format for the next stage of processing. Sunreef, what way do you suggest posting a 3D array? — Scott Alistair, May 27 '16 at 13:19

score 3 · Answer 1 · answered May 27 '16 at 13:10

3

As you can read in the documentation:

The function imwrite saves the image to the specified file. The image format is chosen based on the filename extension (see imread() for the list of extensions). Only 8-bit (or 16-bit unsigned (CV_16U) in case of PNG, JPEG 2000, and TIFF) single-channel or 3-channel (with ‘BGR’ channel order) images can be saved using this function.

You said that "labelled_stack is of type np.uint32" (32-bit unsigned int). That is the problem. You need to handle the type convertion yourself.

answered May 27 '16 at 13:10

Berriel

12,659
4
43
67

change it to what, int? – Scott Alistair May 27 '16 at 13:18
@ScottAlistair uint8 or uint16 (read the limitations described in the documentation) – Berriel May 27 '16 at 13:25

score 3 · Accepted Answer · edited May 23 '17 at 11:44

The function imwrite saves the data in an 8 bit format. When the function converts your 32 bit data to 8 bit, it scales the data to use the full bit depth of 8 bits, so for example the maximum of your data is scaled up to 255. See scipy imsave saves wrong values for another example.

To avoid this, convert your data to numpy.uint8 before saving it. In the following example, a has data type numpy.uint32.

In [98]: from scipy.misc import imsave, imread

In [99]: a
Out[99]: 
array([[0, 3, 2, 2, 4, 0, 3, 0],
       [2, 0, 0, 3, 3, 1, 3, 0],
       [2, 4, 4, 0, 2, 3, 1, 3],
       [0, 1, 3, 1, 0, 0, 0, 4],
       [2, 1, 1, 2, 1, 1, 3, 1],
       [0, 4, 0, 1, 0, 0, 2, 3],
       [3, 1, 3, 3, 3, 2, 3, 4],
       [0, 4, 1, 4, 2, 2, 0, 2]], dtype=uint32)

The astype method is used to convert the array to numpy.uint8 before giving it to the imsave function.

In [100]: imsave("a.tif", a.astype(np.uint8))  # Convert to 8 bit before saving

Read it back with scipy.misc.imread:

In [101]: b = imread("a.tif")

In [102]: b
Out[102]: 
array([[0, 3, 2, 2, 4, 0, 3, 0],
       [2, 0, 0, 3, 3, 1, 3, 0],
       [2, 4, 4, 0, 2, 3, 1, 3],
       [0, 1, 3, 1, 0, 0, 0, 4],
       [2, 1, 1, 2, 1, 1, 3, 1],
       [0, 4, 0, 1, 0, 0, 2, 3],
       [3, 1, 3, 3, 3, 2, 3, 4],
       [0, 4, 1, 4, 2, 2, 0, 2]], dtype=uint8)

Note that when the data is read with scipy.misc.imread, the result has data type numpy.uint8.

Another recommended option, especially if you are trying to preserve data that requires 16 bits or 32 bits, is to use tifffile.

In this example, a has data type numpy.uint32, with some values that are larger than can fit in an unsigned 8 bit number.

In [152]: from tifffile import imsave, imread

In [153]: a
Out[153]: 
array([[  0,   5,  10,  15,  20,  25,  30,  35,  40,  45],
       [ 50,  55,  60,  65,  70,  75,  80,  85,  90,  95],
       [100, 105, 110, 115, 120, 125, 130, 135, 140, 145],
       [150, 155, 160, 165, 170, 175, 180, 185, 190, 195],
       [200, 205, 210, 215, 220, 225, 230, 235, 240, 245],
       [250, 255, 260, 265, 270, 275, 280, 285, 290, 295],
       [300, 305, 310, 315, 320, 325, 330, 335, 340, 345],
       [350, 355, 360, 365, 370, 375, 380, 385, 390, 395],
       [400, 405, 410, 415, 420, 425, 430, 435, 440, 445],
       [450, 455, 460, 465, 470, 475, 480, 485, 490, 495]], dtype=uint32)

In [154]: imsave("a.tif", a)

In [155]: b = imread("a.tif")

In [156]: b
Out[156]: 
array([[  0,   5,  10,  15,  20,  25,  30,  35,  40,  45],
       [ 50,  55,  60,  65,  70,  75,  80,  85,  90,  95],
       [100, 105, 110, 115, 120, 125, 130, 135, 140, 145],
       [150, 155, 160, 165, 170, 175, 180, 185, 190, 195],
       [200, 205, 210, 215, 220, 225, 230, 235, 240, 245],
       [250, 255, 260, 265, 270, 275, 280, 285, 290, 295],
       [300, 305, 310, 315, 320, 325, 330, 335, 340, 345],
       [350, 355, 360, 365, 370, 375, 380, 385, 390, 395],
       [400, 405, 410, 415, 420, 425, 430, 435, 440, 445],
       [450, 455, 460, 465, 470, 475, 480, 485, 490, 495]], dtype=uint32)

Do you have any suggestions for when the no. of labels is unknown on each occasion, and could easily be less than 255 or above 255?? — Scott Alistair, May 27 '16 at 14:33
@ScottAlistair For tiff files, a nice option is the tifffile package. It supports 16 and 32 bit files with no special coding required. I have updated my answer with an example. — Warren Weckesser, May 27 '16 at 15:20
This is something I also found today, but was worried this would act in the same way the my original code would and scale up the data, but i'm glad to see it doesnt! excellent, thank you — Scott Alistair, May 27 '16 at 15:31

Why do values in my 3D numpy array change when I write it to file?

2 Answers2