6

I'm trying to use Keras's Siamese layer in conjunction with a shared Convolution2D layer. I don't need the input to pass through any other layers before the Siamese layer but the Siamese layer requires that input layers be specified. I can't figure out how to create the input layers to match the input of the conv layer. The only concrete example of the Siamese layer being used I could find is in the tests where Dense layers (with vector inputs) are used as input. Basically, I want an input layer that allows me to specify the image dimensions as input so they can be passed on to the shared conv layer.

In code I have something like the following:

img_rows = 28
img_cols = 28
img_input_shape = (1, img_rows, img_cols)

shared = Sequential()

shared.add(Convolution2D(nb_filters, nb_conv, nb_conv,
                        border_mode='valid',
                        input_shape=img_input_shape))
shared.add(Activation('relu'))
# .... more layers, etc.

right_input_layer = SomeInputLayer(input_shape=img_input_shape) # what should SomeInputLayer be?
left_input_layer = SomeInputLayer(input_shape=img_input_shape)
siamese = Siamese(shared, [left_input_layer, right_input_layer], merge_mode='concat')

model = Sequential()
model.add(siamese)
# ...
model.compile(loss=contrastive_loss, optimizer='rmsprop')

What should SomeInputLayer be? Or is my appraoch in general incorrect?

Ben Mabey
  • 1,269
  • 10
  • 18

1 Answers1

3

Okay, I figured it out. The "abstract" Layer class is basically a pass through layer which is just what I need. I was also making a mistake where I thought Siamese could take an entire model (i.e. multiple layers) but it in fact only takes a single layer. To make the creation of these Siamese layers less painful there is a add_shared_layer helper function.

I should also point out this pull request that would allow a shared layer to the first layer in a model, exactly what I am trying to do.

Ben Mabey
  • 1,269
  • 10
  • 18
  • 2
    I think the `Graph` API is more straightforward to use when dealing with siamese networks. I have added such a network to the Keras examples [here](https://github.com/fchollet/keras/blob/52f48e1f462090db19b03ae11dce2991487f70c9/examples/mnist_siamese_graph.py). – Mikael Rousson Feb 14 '16 at 15:43
  • Thanks @MikaelRousson! The Graph API does seem like a more natural fit for this. I had just implemented the same MNIST example you added to Keras (but with a conv net) and I agree that the graph version reads better. – Ben Mabey Feb 14 '16 at 23:24
  • @MikaelRousson is there an easy way to extend your code with triplets? In that case it will need 2 outputs in the `add_node` part (2 pairwise distances) – entropiece Mar 15 '16 at 16:38
  • @entropiece yes, that should be relatively straightforward: one more input to the net and a new distance function that takes 3 inputs. Let me know if you get it to work. – Mikael Rousson Mar 17 '16 at 04:40
  • 1
    Am I correct in assuming that by now the **functional API** should be used instead of Graph? Graph does not seem to be mentioned in the Keras documentation at all anymore. – Plankalkül Nov 30 '16 at 14:58
  • 1
    That's right, the new functional API replaced the graph API and is much easier to use IMO. You can find an example of how to create a Siamese net using the functional API in the Keras repo. – Ben Mabey Dec 04 '16 at 15:18