4

When I build the FCN for segmentation, I want the images to keep the original size of input data, so I use the fully convolution layers. When I choose the fixed input size, such as (224, 224), the transpose conv works fine. However, when I changed the code of using (224, 224) to (h, w), I meet the following error. I googled before, but I didn't figure it out. Can anyone help me? Thanks.

Error information:

InvalidArgumentError (see above for traceback): Conv2DSlowBackpropInput: Size 
of out_backprop doesn't match computed: actual = 62, computed = 
63spatial_dim: 2 input: 500 filter: 16 output: 62 stride: 8 dilation: 1
     [[Node: deconv_layer/conv2d_transpose_2 = 
Conv2DBackpropInput[T=DT_FLOAT, data_format="NCHW", dilations=[1, 1, 1, 1], 
padding="SAME", strides=[1, 1, 8, 8], use_cudnn_on_gpu=true, 
_device="/job:localhost/replica:0/task:0/device:GPU:0"] 
(deconv_layer/conv2d_transpose_2-0-VecPermuteNHWCToNCHW- 
LayoutOptimizer/_1961, deconv_layer/deconv3/kernel/read, 
deconv_layer/Add_1)]]
     [[Node: losses/_2091 = _Recv[client_terminated=false, 
recv_device="/job:localhost/replica:0/task:0/device:CPU:0", 
send_device="/job:localhost/replica:0/task:0/device:GPU:0", 
send_device_incarnation=1, tensor_name="edge_4480_losses", 
tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"] 
()]]

Code:

with tf.variable_scope("deconv_layer"):
    deconv_shape1 = block2.get_shape()
    W_t1 = deconv_utils.weight_variable([4, 4, deconv_shape1[3].value, 2048], 
                                        name="deconv1/kernel")
    b_t1 = deconv_utils.bias_variable([deconv_shape1[3].value], 
                                      name="deconv1/biases")
    deconv_t1 = deconv_utils.conv2d_transpose_strided(block4, W_t1, b_t1, 
                                       output_shape=tf.shape(block2))
    fuse1 = tf.add(deconv_t1, block2)
    print("deconv_t1: ", deconv_t1.shape)
    print("fuse_1: ", fuse1.shape)
    tf.identity(fuse1, name="fuse1")

    deconv_shape2 = block1.get_shape()
    W_t2 = deconv_utils.weight_variable([4, 4, deconv_shape2[3].value, 
                        deconv_shape1[3].value], name="deconv2/kernel")
    b_t2 = deconv_utils.bias_variable([deconv_shape2[3].value], 
                                      name="deconv2/biases")
    deconv_t2 = deconv_utils.conv2d_transpose_strided(fuse1, W_t2, b_t2, 
                        output_shape=tf.shape(block1))
    fuse2 = tf.add(deconv_t2, block1)
    print("deconv_t2: ", deconv_t2.shape)
    print("fuse2: ", fuse2.shape)
    tf.identity(fuse2, name="fuse2")

    shape = tf.shape(features)
    deconv_shape3 = tf.stack([shape[0], shape[1], shape[2], num_classes])
    W_t3 = deconv_utils.weight_variable([16, 16, num_classes, 
                       deconv_shape2[3].value], name="deconv3/kernel")
    b_t3 = deconv_utils.bias_variable([num_classes], name="deconv3/biases")
    deconv_t3 = deconv_utils.conv2d_transpose_strided(fuse2, W_t3, b_t3, 
                       output_shape=deconv_shape3, stride=8)
    print("deconv_t3: ", deconv_t3.shape)

The version with out custom functions is here:

    with tf.variable_scope("deconv_layer"):
    deconv1_shape = block2.get_shape()
    shape1 = [4, 4, deconv1_shape[3].value, 2048]
    deconv1_kernel = tf.Variable(initial_value=tf.truncated_normal(shape1, 
                                 stddev=0.02),
                                 trainable=True,
                                 name="deconv1/kernel")
    deconv1 = tf.nn.conv2d_transpose(value=block4,
                                     filter=deconv1_kernel,
                                     # output_shape=[BATCH_SIZE, 
                             tf.shape(block2)[1], tf.shape(block2)[2], 512],
                                     output_shape=tf.shape(block2),
                                     strides=[1, 2, 2, 1],
                                     padding='SAME',
                                     data_format='NHWC'
                                     )
    print('deconv1', deconv1.shape)
    fuse1 = tf.add(deconv1, block2)  # fuse1 = pool4 + deconv2(pool5)
    tf.identity(fuse1, name="fuse1")

    deconv2_shape = block1.get_shape()
    shape2 = [4, 4, deconv2_shape[3].value, deconv1_shape[3].value]
    deconv2_kernel = tf.Variable(initial_value=tf.truncated_normal(shape2, 
                                 stddev=0.02),
                                 trainable=True,
                                 name="deconv2/kernel")
    deconv2 = tf.nn.conv2d_transpose(value=fuse1,
                                     filter=deconv2_kernel,
                                     output_shape=tf.shape(block1),
                                     strides=[1, 2, 2, 1],
                                     padding='SAME',
                                     data_format='NHWC'
                                     )
    print('deconv2', deconv2.shape)
    fuse2 = tf.add(deconv2, block1)
    tf.identity(fuse2, name="fuse2")

    deconv3_shape = tf.stack([tf.shape(features)[0], tf.shape(features)[1], 
                              tf.shape(features)[2], num_classes])
    shape3 = [16, 16, num_classes, deconv2_shape[3].value]
    deconv_final_kernel = tf.Variable(initial_value=tf.truncated_normal(shape3, stddev=0.02),
                                      trainable=True,
                                      name="deconv3/kernel")

    seg_logits = tf.nn.conv2d_transpose(value=fuse2,
                                        filter=deconv_final_kernel,
                                        output_shape=deconv3_shape,
                                        strides=[1, 8, 8, 1],
                                        padding='SAME',
                                        data_format='NHWC')
Cwang
  • 371
  • 3
  • 9
  • I'll add my comment here instead of an answer. Make sure you're using the same arguments as training data generator for the test data generation. In my case, I had to make sure both input sizes and seeds matched. – NelsonGon Aug 13 '20 at 23:46

3 Answers3

2

The conv Net and Deconv Net in FCN, which are built by different structures, are maybe not consistent with each other. In this case, the conv net use conv with padding='VALID', while the deconv net uses all conv_transpose with padding='SAME. Thus the shapes are not the same, which causes the problem above.

Cwang
  • 371
  • 3
  • 9
0

This is because of your stride > 1. The calculation can not be correct at all time. This GitHub post explains it.

Lau
  • 1,353
  • 7
  • 26
  • 1
    Under my current knowledge, in FCN for segmentation, the up sampling is done by conv2d_transpose with "stride > 1". So I think what you mentioned may not be the key of the problem I met. – Cwang Jul 03 '18 at 11:41
  • The error message you posted gives a Stride of `stride: 8`. So you use a stride > 1. – Lau Jul 03 '18 at 12:04
  • I did use `stride > 1`. My opinion is that when you build the FCN for segmentation, it is necessary to use `conv2d_transpose` with `stride > 1` for up sampling. [This code](https://github.com/shekkizh/FCN.tensorflow) do use it that way. So I believe the error I met is not caused by `stride > 1`. – Cwang Jul 03 '18 at 12:32
  • You don't have to use stride > 1 for upsampling. But you sample up more quickly, when you have a larger stride. Maybe they use sepcial cases, where you compute the output shape, but it is common knowledge that you can not – Lau Jul 03 '18 at 13:29
0

I had similar issue while trying to replicate pytorch's transposeconv2d function in tensorflow. I was trying to do padding on input before passing to the conv2d_transpose() function and doing padding again on the deconvolved output. This was the reason why graph was initialized properly but there was error in calculating the gradients. I solved the error by removing all manual paddings and changing padding="SAME" inside the function. I guess this is handeled internally in the function. Correct me if I am wrong. I don't know how much this affects the actual output.

Akhil_Singh_Rana
  • 355
  • 1
  • 2
  • 10