Residual Neural Network: Concatenation or Element Addition?

Question

With the residual block in residual neural networks, is the addition at the end of the block true element addition or is it concatenation?

For example, would addition([1, 2], [3, 4]) produce [1, 2, 3, 4] or [4, 6] ?

It is element-wise addition. – Dr. Snoopy Oct 24 '17 at 08:10 — Dr. Snoopy, Oct 24 '17 at 08:10

score 5 · Answer 1 · edited Jun 20 '18 at 17:27

5

It would result in [4, 6], and you can find out more in this paper

edited Jun 20 '18 at 17:27

Julien

answered Oct 24 '17 at 07:19

TajyMany

score 2 · Answer 2 · answered Oct 08 '20 at 11:30

2

The operation F + x is performed by a shortcut connection and element-wise addition

It is from the popular ResNet paper by Microsoft Research. Therefore it is element-wise addition, hence [4, 6]

answered Oct 08 '20 at 11:30

Aditya Saurabh

2 Answers2