5

enter image description hereI'm doing an image segmentation with UNet-like CNN architecture by Pytorch 0.4.0.It mark foreground as 1 and background as 0 in the final segmentation result.I use a pre-trained VGG's feature extractor as my encoder, so I need to upsampling the encoder output many times.But the result shows a weird lattice parttern in the result like this:

I suspect these different shape of black parttern were caused by the deconvolutional layers.It's said that deconv layer add (s-1) zeros between the input pixel in the official documentation.the deconvolutional layer's hyperparameter are listed below:

kernel_size=3,stride=2,padding=1,output_padding=1

(the output_padding is used to fit the skip connection feature size, for instance, with a input size 40*30,I want it size enlarge twice in one deconvolutional layer,under my calculation with the formula:out=s(in-1)+k-2p,I must use padding 1 and output-padding 1 to keep size,or there might be other better choice I don’t know.)

I'm wondering the reason that generate the black lattice.They looks like a grid or square parttern.And how to solve it.Besides,I need to adjust all deconv layer hyperparamater to one uniform or modify them separately?

environment:Ubuntu 16.04,Pytorch 0.4.0,GTX 1080Ti, the architecture of upsampling is three deconv layer,they are one after another.

Update

after I modified the deconv hyperparameter,(inspired by noise in image segmentation result use

kernel_size=4,stride=2,padding=1

thus to avoid output-padding and satisfy the size.But after 100 epoch I met similiar problem.

after another 30 epoch training, it seems like this,enter image description here

the black points just scattered,It seems these black points just change their parttern and jump into another parttern,I don't know why it occurs.I'm don't know how to modify my network hyperparameter. 
self.conv1=Basic(1024,512,kernel_size=3,stride=1,padding=1)
        self.conv2=Basic(512,512,kernel_size=3,stride=1,padding=1)
        self.deconv1=Basic(512,256,kernel_size=4,stride=2,conv=False,padding=1)
        self.deconv2=Basic(256,128,kernel_size=4,stride=2,conv=False,padding=1)
        self.deconv3=Basic(128,64,kernel_size=4,stride=2,conv=False,padding=1)
        #output  480*640 segmap
        self.conv4=Basic(64,2,kernel_size=1,stride=1)
        # self.avgpool=nn.AvgPool2d(kernel_size=2,stride=2)

    def forward(self, input):
        input=self.conv1(input)
        input=self.conv2(input)
        input=self.deconv1(input)
        input=self.deconv2(input)
        input=self.deconv3(input)
        input=self.conv4(input)
        # print(input.shape)
        #a channel-wise probability map
        raw=input
        return raw`
Kevin
  • 71
  • 5
  • Do you apply any operation after the last layer of the network? Can we see some code? – Manuel Lagunas Jul 03 '18 at 12:37
  • yeah,thx for ur help! I don't know how to add pic in the comment ,so I attach the decoder code at the bottom of question description. Basic block is a conv+batchNorm+Relu operator.I upsampling from the VGG layer 3 output,so I need three deconv operation to recover input size. – Kevin Jul 03 '18 at 12:39
  • There is an option in the StackOverflow editor to add code as text, it will make it more readable. What if you add at the end `input = F.sigmoid(input)`. Like this, we restrict the pixels to be within the 0-1 range. Then, since you want to do a binary segmentation add `input = input>0.5`. Like this, we set the pixels that have a value higher than 0.5 to be 1 and the ones that are under 0.5 to be 0. If `F` fails do `import torch.nn.functional as F`. Post the output afterwards :) – Manuel Lagunas Jul 03 '18 at 12:48
  • yeah, I'do the softmax and argmax to convert the channel-wise digit into probability.and do a argmax operation to get the output.I think these operation is not wrong.But I think the deconv or net architecture which causes these parttern. – Kevin Jul 03 '18 at 12:52
  • Better update the post :) – Manuel Lagunas Jul 03 '18 at 12:54
  • Yeah,already update the question,just neglect the comments in code block cause the segmentation result is just a part of my network. – Kevin Jul 03 '18 at 12:57
  • Have you tried to add what I mentioned at the end of the forward method? – Manuel Lagunas Jul 03 '18 at 13:07
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/174261/discussion-between-kevin-and-manuel-lagunas). – Kevin Jul 03 '18 at 13:08

0 Answers0