2

I am doing augmentation to do segmentation task with caffe. The Python Layer that I have written is raising an error. The layer definition is like this:

layer {
    name: 'myaug'
    type: 'Python'
    bottom: 'data'
    bottom: 'label'
    top: 'data'
    top: 'label'
    python_param {
        module: 'augLayer'
        layer: 'CompactData'
    }
}

this is the net drawing: net drawing

The error seems to be related to numpy:

File "/home/usersc/caffe/python/caffe/pycaffe.py", line 11, in <module>
    import numpy as np
File "/home/usersc/anaconda2/envs/mycaffe/lib/python2.7/site-packages/numpy/__init__.py", line 142, in <module>
    from . import add_newdocs
  File "/home/usersc/anaconda2/envs/mycaffe/lib/python2.7/site-packages/numpy/add_newdocs.py", line 13, in <module>
    from numpy.lib import add_newdoc
  File "/home/usersc/anaconda2/envs/mycaffe/lib/python2.7/site-packages/numpy/lib/__init__.py", line 8, in <module>
    from .type_check import *
  File "/home/usersc/anaconda2/envs/mycaffe/lib/python2.7/site-packages/numpy/lib/type_check.py", line 11, in <module>
    import numpy.core.numeric as _nx
  File "/home/usersc/anaconda2/envs/mycaffe/lib/python2.7/site-packages/numpy/core/__init__.py", line 22, in <module>
    from . import _internal  # for freeze programs
  File "/home/usersc/anaconda2/envs/mycaffe/lib/python2.7/site-packages/numpy/core/_internal.py", line 14, in <module>
    import ctypes
  File "/home/usersc/anaconda2/envs/mycaffe/lib/python2.7/ctypes/__init__.py", line 7, in <module>
    from _ctypes import Union, Structure, Array
ImportError: /home/usersc/anaconda2/envs/mycaffe/lib/python2.7/lib-dynload/_ctypes.so: undefined symbol: _PySlice_Unpack

I am not sure, I am thinking should I add a MemoryData layer to keep the augmented data for me as in this link since both data and label images should be sent synchronously. Is it like that Data Layer memory should be cleared?

Shai
  • 111,146
  • 38
  • 238
  • 371
sc241
  • 189
  • 17

1 Answers1

1

You have a problem importing numpy: this has nothing to do with your code/layer, your code was not even run yet.
Make sure numpy is properly installed on your machine and that your $PYTHONPATH environment points to the right places.

Regarding memory: the way you defined your layer, it performs the augmentations "in-place", that is, it changes data and label blobs instead of making copies of the augmented inputs. Make sure you are okay with this kind of behavior. Furthermore, I don't think you need a "MemoryData" layer to carry out your augmentations, the "Python" layer should be enough.

Shai
  • 111,146
  • 38
  • 238
  • 371
  • Thanks, I have tried to import numpy separately, and it is working well in the environment. I do not want to be in-place, but the problem is that in normal flow (without augmentation) some other layers are using `data` and `label` blobs (and I do not know how to handle this) . What do you suggest? How should I send the augmented data into the Network? – sc241 May 08 '18 at 14:54
  • @sc241 some of your layers expects original "data" while others expects augmented "data"? does not sound like a robust plan... – Shai May 08 '18 at 14:56
  • No actually, both augmented and original data should go to the same path of layers in the network. That is why I was thinking whether to use a `MemoryData` to keep augmented one and then copy (send) it to the `data` and `label` layers (as tops in `MemoryData` layer)or my thinking is not correct? – sc241 May 08 '18 at 15:09