0

I am attempting to use mxnet 1.10/mxnet-cu91 for image classification. I am currently attempting to use mxnet.image.ImageIter to iterate through and preprocess images. I have been able to successfully use the Augmenters to preprocess the images, but have received the following error when using Augmenters (with the only exception being ForceResizeAug):

Traceback (most recent call last):
  File "image.py", line 22, in <module>
    for batch in iterator:
  File "/usr/local/lib/python2.7/dist-packages/mxnet/image/image.py", line 1181, in next
    data = self.augmentation_transform(data)
  File "/usr/local/lib/python2.7/dist-packages/mxnet/image/image.py", line 1239, in augmentation_transform
    data = aug(data)
  File "/usr/local/lib/python2.7/dist-packages/mxnet/image/image.py", line 659, in __call__
    src = t(src)
  File "/usr/local/lib/python2.7/dist-packages/mxnet/image/image.py", line 721, in __call__
    gray = src * self.coef
  File "/usr/local/lib/python2.7/dist-packages/mxnet/ndarray/ndarray.py", line 235, in __mul__
    return multiply(self, other)
  File "/usr/local/lib/python2.7/dist-packages/mxnet/ndarray/ndarray.py", line 2566, in multiply
    None)
  File "/usr/local/lib/python2.7/dist-packages/mxnet/ndarray/ndarray.py", line 2379, in _ufunc_helper
    return fn_array(lhs, rhs)
  File "<string>", line 46, in broadcast_mul
  File "/usr/local/lib/python2.7/dist-packages/mxnet/_ctypes/ndarray.py", line 92, in _imperative_invoke
    ctypes.byref(out_stypes)))
  File "/usr/local/lib/python2.7/dist-packages/mxnet/base.py", line 146, in check_call
    raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [20:02:07] src/operator/contrib/../elemwise_op_common.h:123: Check failed: assign(&dattr, (*vec)[i]) Incompatible attr in node  at 1-th input: expected uint8, got float32

Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet/libmxnet.so(+0x2ab9a8) [0x7f5c873f09a8]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet/libmxnet.so(+0x2abdb8) [0x7f5c873f0db8]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet/libmxnet.so(+0x2d2078) [0x7f5c87417078]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet/libmxnet.so(+0x2d2b83) [0x7f5c87417b83]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet/libmxnet.so(+0x24c4c1e) [0x7f5c89609c1e]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet/libmxnet.so(+0x24c6e59) [0x7f5c8960be59]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet/libmxnet.so(+0x240539b) [0x7f5c8954a39b]
[bt] (7) /usr/local/lib/python2.7/dist-packages/mxnet/libmxnet.so(MXImperativeInvokeEx+0x63) [0x7f5c8954a903]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7f5cc334ae40]
[bt] (9) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7f5cc334a8ab]

The code needed to replicate the issue is below (shortened for brevity, closely resembles the code provided in the documentation):

import mxnet as mx
import glob

type1_paths = glob.glob('type1/*.jpg')
type1_list = [[1.0, path] for path in type1_paths]

type2_paths = glob.glob('type2/*.JPG')
type2_list = [[2.0, path] for path in type2_paths]
all_paths = type1_list + type2_list
iterator = mx.image.ImageIter(1, (3, 1000, 1000),
                              imglist=all_paths,
                              aug_list=[
                              mx.image.ColorJitterAug(0.1, 0.1, 0.1),
                              ])
for batch in iterator:
    print batch.data

I am not sure why the error is occurring, as I am not using any custom augmenters that could effect the discrepancy in dtype. I've also replicated this issue when using the following:

  • RandomGrayAug
  • HueJitterAug
  • ContrastJitterAug
  • SaturationJitterAug

NOTE: In case this matters, the only differences I know between the loaded jpg/JPG is that some photos were taken using a phone, and others using a DSLR camera.

Please let me know if I am missing any information that would be helpful in learning.

DFenstermacher
  • 564
  • 1
  • 9
  • 23
  • Do you have the possibility of posting a link to one the images causing the issue? Also have you tried using the Gluon API and the ImageFolderDataset with a DataLoader rather than the ImageIter? Have a look at this tutorials: http://mxnet.incubator.apache.org/tutorials/gluon/datasets.html and https://mxnet.incubator.apache.org/tutorials/gluon/data_augmentation.html – Thomas Apr 26 '18 at 18:09

1 Answers1

2

You're getting this issue because the images are loaded with a data type of int8 but the augmentations are expecting a data types of float32. Unfortunately the error message reads a little backwards to what you need to do in this case, because of a multiplication of the input image (int8) with a contrast jitter (float32). It's complaining about the data type of the contrast jitter instead of the input data. Same issue with hue and saturation augmenters.

So to fix this you need to convert your input image data type to float32. You can do this by adding mx.image.CastAug(typ='float32') at the start of your augmenter list.

iterator = mx.image.ImageIter(1, (3, 100, 100),
                              path_root=".",
                              imglist=all_paths,
                              aug_list=[
                                    mx.image.CastAug(typ='float32'),
                                    mx.image.ColorJitterAug(0.1, 0.1, 0.1),
                                    mx.image.CenterCropAug((100,100))
                              ])

And it's always a good idea to visualize your data after augmentation to confirm the steps are being applied as you expected.

Thom Lane
  • 993
  • 9
  • 9