0

I would like to create my own operations to merge networks. So I've taken a look to the code, and I modified engine/topology.py to create my new operation.

I didn't modified layers/wrappers.py because it's only for RNN and when I modify it I get an error.

Are there other files/classes to modify? Don't I have to do something else somewhere else to specify what to do during the backward pass?

FiReTiTi
  • 5,597
  • 12
  • 30
  • 58

1 Answers1

1

You don't have to change any other files if you have implemented your operation properly with backend operations only. The backend is clever and takes care of the computation of the gradients for the backpropagation by itself.

This means that all parameters that will change over time have to be defined with K.variable, and you only use mathematical operations defined in keras.backend. Otherwise the backend will not be able to perform the backpropagation properly.

Side-note: Instead of modifying the source code of keras, you could implement your own class that extends the Merge class and override the call function for your custom operation.

oscfri
  • 2,558
  • 2
  • 27
  • 24
  • Thanks a lot for your answer. I am gonna test it right away! But I have to admit that I am a little bit surprised because in the file topology.py, the basic operations like 'sum', 'ave', and 'max' do not use the backend. How does it work? – FiReTiTi Mar 24 '17 at 06:14
  • The 'sum' and 'ave' actually do use the backend. It's just hidden behind the standard arithmetic operations (+, -, * and /) which are overriden by the Keras variables. For 'max' I'm not sure. Where is it defined in the code? – oscfri Mar 24 '17 at 07:03
  • Are you sure that the +, -, * and / are overridden somewhere? Because I only need these operations to perform my new merging operation. – FiReTiTi Mar 24 '17 at 16:51
  • I took a look to the code, and I didn't see where the arithmetic operations were overridden :-(. – FiReTiTi Mar 24 '17 at 17:22