3

I am doing a image segmentation task based on deep convolutional neural network. The network structure is from this paper, and the structure can be seen in the picture:FCN used in image segmentation. The network is designed based on Fully Convolutional Network and DCAN.

The data set is the public benchmark dataset of Gland Segmentation Chalenge Contest in MICCAI 2015(also named as Warwick-QU dataset.)

I use this network to do the segmentation task. However in my result, I always noticed a fixed pattern of noise: the little white crossing in the predicted image:Prediction results with crossing shape noise

Can someone please explain what does these noise pixels mean? Are they the common noises in image processing? It is worth noting that this phenomena does not appear only in a specific image, but for the whole prediction.

Wentai Chen
  • 177
  • 1
  • 8
  • That has also happened to me before, but I've never found a satisfactory explanation. Can I ask what cost function you are using? – Ash Mar 29 '18 at 14:56
  • Thanks for the reply. I am using `categorical_crossentropy`. Are you also doing image segmentation on MICCAI 2015 dataset? – Wentai Chen Mar 29 '18 at 16:07
  • 1
    Hello Ash, I found the trick. These crossing-shape noises are caused by the deconvolutional layer in my network. Let's say we have a feature map of **32x32** and we want to get the feature map with the size of **256x256**, so we have to use the deconvolutional layer with a stride of **8x8** to do the deconvolution, thus a lot of zero pixels are added to the feature map, that is why these crossing shape occurs. To deal with this, we have to enlarge our kernel size twice as the stride. In the above example we need choose a *16x16* kernel so that our filter will not fall into those zeros. – Wentai Chen Apr 05 '18 at 13:59
  • 1
    FYI, the operation of deconvolution is illustrated here: [deconvolution](https://github.com/vdumoulin/conv_arithmetic) – Wentai Chen Apr 05 '18 at 14:10
  • Great explanation, but I suggest that you add it to your answer since comments seem to often be deleted by the system after a while. – Ash Apr 07 '18 at 19:30
  • 1
    Thanks for this notification Ash, I already updated my answer. – Wentai Chen Apr 08 '18 at 13:05

2 Answers2

6

These crossing-shape noises are caused by the deconvolutional layer in my network, and it is solved by changing the kernel size in deconvolutional layer.

First we should understand how does the deconvolutonal layer work. A clear illustration can be found here. What we need to bear in mind is that the stride in deconvolution helps to enlarge the size of the feature map.

Back to this example. Let's say we have a feature map of 32x32 and we want to get the feature map with the size of 256x256, so we have to use the deconvolutional layer with a stride of 8x8 to do the deconvolution, thus a lot of zero pixels are added to the feature map, that is why these crossing shape occurs. To deal with this, we have to enlarge our kernel size twice as the stride. In the above example we need choose a 16x16 kernel so that our filter will not fall into those zeros.

The result image can be shown here: result
We can see clearly that the crossing-shape noises are removed compared with the picture attached in the question.

Wentai Chen
  • 177
  • 1
  • 8
0

I also meet this problem, but i find the reason is that the output channel set falsely which it should be 256 instead of 1

JJ Zeng
  • 1
  • 1
  • As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Jun 01 '22 at 06:08