-1

I'm trying to find methods for dealing with occluded objects for a Tensorflow CNN image recognition project. The underlying objects are mostly visible, but similar objects often cross over. Most of the crossover points are literally a cross shape (but not always). Think of tossing a bunch of curvy-looking toothpicks or pieces of wire on a table. Possible advantage: Most of the objects are of similar structure, but of various sizes, and various bends.

Does this require some kind of preprocessing step, or is it possible to handle it strictly by some process within the CNN? I can get outlines of the objects via Sobel/Canny/whatever. The question is whether I can avoid having to write tedious hand-coded functions to separate the objects.

Any ideas or references to known work or papers appreciated!

StringTheory
  • 117
  • 3
  • 8

1 Answers1

0

Look at the basic structure of a CNN: in the first layers, you have many maps describing possible local features. Since the object overlap is limited, you'll still get a strong signal from the non-overlapping parts. The subsequent layers should be able to work on that, especially if they've been trained with such overlap.

Obviously, you need training for this. A CNN doesn't know a priori what an object is, so differentiating between VV and W depends on whether the network is trained with W. If a network has never seen W but only V, then a W will be recognized as VV.

MSalters
  • 173,980
  • 10
  • 155
  • 350
  • Thanks, MS. Interesting, and good analogy. I've been considering multiple passes, trying to remove 'top' objects manually and filling in the resulting gaps. That seems tedious, and possibly error-prone. But it also seems difficult to train with lots of combinations of overlapping objects. Even though there are only three basic types, like pieces of bent wire, they could assume various curves. And variable numbers of them could overlap at random angles. Is this possibly in the realm of training? (I have found no papers or research so far) – StringTheory May 30 '17 at 21:01