2

I'm using OpenEXR to read EXR files in Python. I have R, G and B channels with Half data (float16). Using Numpy, I tried, unsuccessfully to convert the data from float16 to uint8 (0-255 colors).

        rCh = getChanEXR(imageFile, 'R','HALF')
        rCh = np.array(rCh).astype('uint8')

So, I put the R channel pixel values to a variable rCh. Then I convert the array.array to an np.array so that I can use the astype method to convert it to uint8. I am new to this, so I clearly do not have it right as all values become 0. Originally, the values are like this: 0.0, 2.9567511226945634e-14, 1.2295237050707897e-10 etc.

In addition to the float16 values, I also have some regular float values that need to be normalized. I think I need to normalize the float16 values before they can be set in a range from 0-255.

Any ideas? Thank you.

Adding the code for the def mentioned in here getChanEXR (just a custom def based on code from the python OpenEXR docs for getting channel data.

def getChanEXR(curEXRStr, curChannel, dataType):
    #import OpenEXR, Imath, array
    pt = 'none'
    if dataType == 'HALF':
        pt = Imath.PixelType(Imath.PixelType.HALF)
    if dataType == 'FLOAT':
        pt = Imath.PixelType(Imath.PixelType.FLOAT)
    if dataType == 'UINT':
        pt = Imath.PixelType(Imath.PixelType.UINT)
    chanstr = OpenEXR.InputFile(curEXRStr).channel(curChannel, pt)
    chan = array.array('f', chanstr)
    return chan
Mad Guru
  • 41
  • 1
  • 7
  • 1
    Thanks for the response Filippo. I think your answer is correct, but there is one step keeping me from it. The data is in the form of an array.array, so when I use min or max, it tells me that it can't be used with an array.array. If I use np.asarray to convert it, all the values become 0. – Mad Guru May 15 '18 at 03:51
  • you should normalize the data *before* converting it to `np.uint8`, you can use standard python `min()` and `max()` with `array.array` or you can convert it to a numpy float array, normalize that and then convert to 8bits – filippo May 15 '18 at 18:40
  • What is `getChanEXR`? It doesn't show up in a [google search for `openexr getchanexr`](https://www.google.com/search?q=openexr+getchanexr) or an [OpenEXR documentation search for `getChanEXR`](http://excamera.com/articles/26/doc/search.html?q=getChanEXR). – user2357112 May 15 '18 at 19:02
  • getChanEXR is just a def I created based on openEXR python code from the documentation. This is how I get the channel data. `def getChanEXR(curEXRStr, curChannel, dataType): #import OpenEXR, Imath, array pt = 'none' if dataType == 'HALF': pt = Imath.PixelType(Imath.PixelType.HALF) if dataType == 'FLOAT': pt = Imath.PixelType(Imath.PixelType.FLOAT) if dataType == 'UINT': pt = Imath.PixelType(Imath.PixelType.UINT) chanstr = OpenEXR.InputFile(curEXRStr).channel(curChannel, pt) chan = array.array('f', chanstr) return chan` – Mad Guru May 16 '18 at 05:09

1 Answers1

1

I haven't got much experience with array.array but I believe you can convert it to a numpy float array so it's a bit easier to work with:

rCh = np.asarray(rCh, dtype=np.float)

If your data is normalized in [0,1] multiply it by 255 before the conversion:

rCh = np.asarray(rCh * 255, dtype=np.uint8)

I believe it's truncating away the fractional part though. Manually rounding it should be safer? (not so sure, see discussion in the comments, I believe the correct approach will be dithering here, but I guess the matter deserves better research regarding your specific use case)

rCh = np.asarray(np.around(rCh * 255), dtype=np.uint8)

If it's not normalized you could just do 

rCh -= rCh.min()
rCh /= rCh.max()

And then convert it to 8bits

rCh = np.asarray(rCh * 255, dtype=np.uint8)
filippo
  • 5,197
  • 2
  • 21
  • 44
  • Truncating is correct; rounding would introduce errors, such as having 255.6 round up to 256 and then become 0 due to 8-bit overflow. Also, `array.array` is a completely different array type. – user2357112 May 15 '18 at 18:55
  • Apparently the `array.array` confusion comes from Mad Guru, though. It's not clear what kind of arrays we're working with. – user2357112 May 15 '18 at 18:56
  • @user2357112 uh? you cannot have `255.6` if your data is normalized in `[0,1]` – filippo May 15 '18 at 18:57
  • bleh, whoops. I thought you were multiplying by 256, not 255. You should multiply by 256 and truncate, rather than multiplying by 255 and rounding. Multiplying by 255 and rounding doesn't evenly bin the results. – user2357112 May 15 '18 at 18:58
  • @user2357112 interesting, do you have any reference to prove truncating is better? is it something specific of image data, seems counter-intuitive – filippo May 15 '18 at 19:03
  • @user2357112 maybe dithering would be the best choice but I don't think truncating is inherently better than rounding – filippo May 15 '18 at 19:06
  • [Here's a demo](https://ideone.com/DkVlYb) where multiplying by 255 and rounding results in only about half as many 0s and 255s in the result as other values, while multiplying by 256 and truncating doesn't introduce this bias. – user2357112 May 15 '18 at 19:07
  • I'll look into it but I'm not that convinced... here `1.0` is a perfectly fine value and if you multiply it by `256` you will get `0` – filippo May 15 '18 at 19:17
  • 1.0 usually isn't a valid value; if it is, it should be special-cased. – user2357112 May 15 '18 at 19:19
  • @user2357112 how would you normalize data to avoid `1.0` then? – filippo May 15 '18 at 19:20
  • Start with a half-open interval and keep it that way, or reduce the value to below 1.0 if you have a closed interval or if rounding error introduces values at or above 1.0. – user2357112 May 15 '18 at 19:27
  • @user2357112 probably missing something, need to freshen up my memories about quantization, I'd say rounding unevenly binning of the edge cases is still preferable to the higher and non-zero mean error you get from truncation... – filippo May 15 '18 at 19:49
  • I tried np.asarray and round but those just make the float16 values equal 0. The array.array is the form that the pixels are saved in. I have tried some of the other methods on the below page, but the fromstring gives an error that it should be frombytes, which gives a size error: http://excamera.com/articles/26/doc/intro.html#interfacing-with-other-packages – Mad Guru May 16 '18 at 06:39
  • @MadGuru you should normalize before `np.asarray`, while you still have floating point data. If you convert data like `2.9567511226945634e-14` to `uint8` without normalization you will get `0` – filippo May 16 '18 at 06:44
  • @MadGuru you probably didn't set the correct `dtype` in your `np.fromstring` call, I'd try to avoid `array.array` and try to get data directly into a numpy array if that's your target – filippo May 16 '18 at 07:30
  • Thanks @filippo. I ended up using `chanStr = curEXR.channel(curChannel, pt); chan = np.fromstring(chanStr, dtype = curDtype); np.asarray(chan * 255, dtype=np.uint8)` – Mad Guru May 22 '18 at 14:12