Caffe Element-Wise multiplication with fixed blobs

Question

I think I will be asking multiple quesitons here, I'd love any comment because I'm new to Caffe.

In my network input images have size 1x41x41 Since I am using 64 batch size I think the data size will be 64x1x41x41 (Please correct me if this is wrong)

After some convolutional layers (that don't change the data size), I would like to multiply the resulting data with predefined blobs of size 1x41x41. It seems convenient to use EltwiseLayer to do the multiplication. So in order to define second bottom layer of the Eltwise I need to have another input data for the blobs. (Please advise if this can be done in other way)

The first question: Batch training confuses me. If I want to multiply a batch of images with a single blob in an EltwiseLayer should the bottom sizes be the same? In other words should I use repmat (matlab) to clone 64 blobs to have a size of 64x1x41x41 or can I just plug single blob of size 1x1x41x41?

Second question: I want to multiply the data with 100 different blobs and then take the mean of 100 results. Do I need to define 100 EltwiseLayers to do the job? Or can I collect blobs in a single data of size 1x100x41x41 (or 64x100x41x41) and clone the data to be multipled 100 times? And if so how can I do it? An example would be very useful. (I've seen a TileLayer somewhere but the info is spread across the galaxy.)

Thanks in advance.

what's the point of multiplying 100 times with **fixed** matrices and then taking the mean? Take the mean of the 100 matrices **once** and multiply by the blob — Shai, Sep 30 '17 at 22:05
@Shai You're actually right about taking the mean of blobs. I am not sure how I will merge 100 results, maybe I can use maxpooling etc. For the tile layer [link](https://github.com/BVLC/caffe/issues/4422) was unable to answer how was I supposed to use it; and I also cannot follow doxygen documentation to understand which parameters to set. — Tarhan, Oct 01 '17 at 07:35

score 2 · Accepted Answer · answered Oct 01 '17 at 07:45

In order to do element-wise multiplication in caffe both blobs must have exactly the same shape. Caffe does not "broadcast" along singleton dimensions.
So, if you want to multiply a batch of 64 blobs of shape 1x41x41 each, you'll have to provide two 64x1x41x41 bottom blobs.
As you already noted, you can use "Tile" layer to do the repmating:

layer {
  name: "repmat"
  type: "Tile"
  bottom: "const_1x1x41x41_blob"
  top: "const_64x1x41x41_blob"
  tile_param {
    axis = 0   # you want to "repmat" along the first axis
    tiles = 64 # you want 64 repetitions
  }
}

Now you can do "Eltwise" multiplication

layer {
  name: "mul"
  type: "Eltwise"
  bottom: "const_64x1x41x41_blob"
  bottom: "other_blob"
  top: "mul"
  eltwise_param {
    operation: MUL
  }
}

Please use operation: PROD, not operation: MUL https://caffe.berkeleyvision.org/tutorial/layers/eltwise.html — myth510, Aug 05 '20 at 00:19

Caffe Element-Wise multiplication with fixed blobs

1 Answers1