Using im2col layer, reshape layer and inner product layer to implement locally connected layer?

Question

I am using caffe and it doesn't have a locally connected layer. So any example on how to use im2col layer, reshape layer and inner product layer to implement locally connected layer? Thanks

Answer 1

Personal View of Point:

I have also tried to use Crop, Im2col, Reshape and InnerProduct layer to implement locally connected layer but failed.

Because when I want to implement a convolution operation using InnerProduct layer, I find that in InnerProductLayer<Dtype>::Forward_cpu() function:

caffe_cpu_gemm<Dtype>(CblasNoTrans, transpose_ ? CblasNoTrans : CblasTrans,
                      M_, N_, K_, (Dtype)1.,
                      bottom_data, weight, (Dtype)0., top_data);

and in BaseConvolutionLayer<Dtype>::forward_cpu_gemm() function:

caffe_cpu_gemm<Dtype>(CblasNoTrans, CblasNoTrans, conv_out_channels_ /
    group_, conv_out_spatial_dim_, kernel_dim_,
    (Dtype)1., weights + weight_offset_ * g, col_buff + col_offset_ * g,
    (Dtype)0., output + output_offset_ * g);

the weight(s), which should be used as convolution kernels, are passed to different arguments of caffe_cpu_gemm().

So I can't implement a convolution operation using InnerProductLayer<Dtype>::Forward_cpu() function and thus can't implement a local connected layer(I mean local convolution here) using Crop, Im2col, Reshape and InnerProduct layers.

My solution:

However, I implemented a local convolution layer here and its idea is to divide input feature maps into N*N grid(even with overlap) and performs convolution on each of the grid using different kernels. For example, the input feature maps have a shape (2, 3, 8, 8) and you want to divide the spatial feature map 8*8 into 16 2*2 local regions and then perform convolution on each local region with different bank of kernels, you can write a prototxt like this:

layer { 
  name: "local_conv" 
  type: "LocalConvolution"
  bottom: "bottom"  # shape (2,3,8,8)
  top: "top"
  param {
    lr_mult: 1
    decay_mult: 1
    }
  param {
    lr_mult: 2
    decay_mult: 0
    }
  local_conv_param {
    local_region_number_h: 4
    local_region_number_w: 4
    local_region_ratio_h: 0.3 # determin the height/width of local regions
    local_region_ratio_w: 0.3 # local_region_size = floor(local_region_ratio * input_size)
    local_region_step_h: 2    # step between local regions on the top left part 
                              # and other regions will lie in the axial symmetry positions
                              # automatically
    local_region_step_w: 2
    num_output: 5 
    kernel_h: 3
    kernel_w: 1
    stride: 1
    pad: 0
    weight_filler {
        type: "xavier"
        }    
    bias_filler {      
        type: "constant"
        }  
  }
}

You can easily add this layer to your caffe and the related files are:

include/caffe/layers/local_conv_layer.hpp
src/caffe/layers/local_conv_layer.cpp(cu)

and you should also add message LocalConvolutionParameter, optional LocalConvolutionParameter local_conv_param from src/caffe/proto/caffe.proto to your caffe.proto. .