0

Im studying style-transfer networks and right now working with this work and here is network description. The problem that even with adding TV loss there is still visible noise which is breaking quality of result. Can someone recommend some articles of ways of removing such noise during network training?

Thanks

Example of noise

Andrey Nikishaev
  • 3,759
  • 5
  • 40
  • 55

1 Answers1

1

The deconvolution noise is because of the uneven overlaps between the input and the kernel which creates a checkerboard-like pattern of varying magnitudes. One fix is to use resize-conv method as mentioned in this article.

Resize-conv replaces transpose convolution with image scaling followed by a 2D convolution. In tensor flow, the 2 steps are: tf.image.resize_images(...) and tf.nn.conv2d(...). Another tip from the authors is to call tf.pad(...) prior to the convolution method and only use Nearest Neighbour resize method.

Vijay Mariappan
  • 16,921
  • 3
  • 40
  • 59
  • yeah, i tried to do upsampling using tf.image.resize_nearest_neighbor but that gave not such good result in style transfer overall and increase training time. Is there any other methods of doing this? what if i use kernel size 4x4 with stride 2 instead of 3x3, or 3x3 with stride 3? It will give even number of overlaps. – Andrey Nikishaev Aug 08 '17 at 07:32
  • 1
    yes having kernel size multiple of stride might also help. The above method worked for me in GANs to produce smoother image. – Vijay Mariappan Aug 08 '17 at 08:04