How to use lambda layer in keras?

Question

I want to define lambda layer to combine features with cross product, then merge those models,just like the fig. ,What should I do?

enter image description here

Test model_1, get 128 dimensions form dense, use pywt get two 64 dimensions feature(cA,cD), then return cA*cD //of course I want to combine two models ,but try model_1 first.

from keras.models import Sequential,Model
from keras.layers import Input,Convolution2D,MaxPooling2D
from keras.layers.core import Dense,Dropout,Activation,Flatten,Lambda
import pywt

def myFunc(x):
    (cA, cD) = pywt.dwt(x, 'db1')
#    x=x*x
    return cA*cD

batch_size=32
nb_classes=3
nb_epoch=20
img_rows,img_cols=200,200
img_channels=1
nb_filters=32
nb_pool=2
nb_conv=3

inputs=Input(shape=(1,img_rows,img_cols))
x=Convolution2D(nb_filters,nb_conv,nb_conv,border_mode='valid',
                  input_shape=(1,img_rows,img_cols),activation='relu')(inputs)
x=Convolution2D(nb_filters,nb_conv,nb_conv,activation='relu')(x)
x=MaxPooling2D(pool_size=(nb_pool,nb_pool))(x)
x=Dropout(0.25)(x)
x=Flatten()(x)
y=Dense(128,activation='relu')(x)
cross=Lambda(myFunc,output_shape=(64,))(y)   
predictions=Dense(nb_classes,activation='softmax')(cross)
model = Model(input=inputs, output=predictions)
model.compile(loss='categorical_crossentropy',optimizer='adadelta',metrics=['accuracy'])

model.fit(X_train,Y_train,batch_size=batch_size,nb_epoch=nb_epoch,
          verbose=1,validation_data=(X_test,Y_test))

Sorry, can I ask a question about tensor?

import tensorflow as tf
W1 = tf.Variable(np.array([[1,2],[3,4]]))
init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init)
array = W1.eval(sess)
print (array)

That's right! However,

from keras import backend as K
import numpy as np
kvar=K.variable(np.array([[1,2],[3,4]]))
K.eval(kvar)
print(kvar)

I got <CudaNdarrayType(float32, matrix)> and kvar.eval() I got b'CudaNdarray([[ 1. 2.]\n [ 3. 4.]])'. I use keras, so how can I get array like tensorflow using keras?

So, keras lambda functions need all operations to use "tensors". The common operations are all listed in https://keras.io/backend/. You must find a way to rewrite pywt.dwt in a tensor way. Unfortunately, that is not an easy task. I believe this question is very important and I can't answer it properly. — Daniel Möller, Jul 06 '17 at 14:40
Is lambda layer must use keras backends function? Can I transform tensor into array then use pywt.dwt, then transform array into tensor? — Ting Li, Jul 06 '17 at 15:06
It's possible, but that will interrupt the "graph" and will bring an error. To convert a tensor in an array, use "tensorVar.eval()". To create a tensor from an array use "K.variable(arrayVar)", where K is the keras.backend. — Daniel Möller, Jul 06 '17 at 18:00
I `print(karr)`, got `b'CudaNdarray([[ 1. 2.]\n [ 3. 4.]])'` — Ting Li, Jul 07 '17 at 05:46

Daniel Möller · Accepted Answer · 2017-07-05T16:25:31.423

7

I would probaly duplicate the dense layers. Instead of having 2 layers with 128 units, have 4 layers with 64 units. The result is the same, but you will be able to perform the cross products better.

from keras.models import Model

#create dense layers and store their output tensors, they use the output of models 1 and to as input    
d1 = Dense(64, ....)(Model_1.output)   
d2 = Dense(64, ....)(Model_1.output)   
d3 = Dense(64, ....)(Model_2.output)   
d4 = Dense(64, ....)(Model_2.output)   

cross1 = Lambda(myFunc, output_shape=....)([d1,d4])
cross2 = Lambda(myFunc, output_shape=....)([d2,d3])

#I don't really know what kind of "merge" you want, so I used concatenate, there are Add, Multiply and others....
output = Concatenate()([cross1,cross2])
    #use the "axis" attribute of the concatenate layer to define better which axis will be doubled due to the concatenation    

model = Model([Model_1.input,Model_2.input], output)

Now, for the lambda function:

import keras.backend as K

def myFunc(x):
    return x[0] * x[1]

edited Jul 05 '17 at 16:25

answered Jul 04 '17 at 12:55

Daniel Möller

84,878
18
192
214

Thanks a lot! when I use model.add(Dense(128)) get 128 dimensions, I add model.add(Lambda(wavelets,output_shape=input_shape[0])), but I don't know how to use lambda layer. I want to use wavelets(function:(cA, cD) = pywt.dwt(x, 'db1')) to get Approximation and detail coefficients(they both 64 dimension,like that fig.). Then cross product Approximation coefficients with model 1 and 2. Finally merge the Approximation and detail coefficients with mode concat. Can you help to write lambda layers? – Ting Li Jul 05 '17 at 14:56
sorry not cross product, it's * – Ting Li Jul 05 '17 at 15:15
Updated my answer. – Daniel Möller Jul 05 '17 at 16:25
sorry, output_shape=..? I use theano as backend, it can't auto-inferred. – Ting Li Jul 06 '17 at 02:06
Yes, you must put the output_shape in lambda layers. If you're using one dimension vectors, the dense layers will output `(64,)`, so the output shape in the lambda will also be `(64,)` – Daniel Möller Jul 06 '17 at 02:22
`import pywt y=Dense(128,activation='relu')(x) cross=Lambda(myFunc,output_shape=(64,))(y) def myFunc(x): (cA, cD) = pywt.dwt(x, 'db1') return cA*cD` – Ting Li Jul 06 '17 at 07:00
When I run this code, I got a error:`TypeError: Cannot convert str to numpy.dtype` If `myFunc(x)` is `x=x*x` and `return x`, that's correct! why I can't deal x with `pywt.dwt`?Please help – Ting Li Jul 06 '17 at 08:41
Could you post your code? We can't hepl if we don't know exactly what you're doing. But in short: you must use "keras tensors" inside a lambda function (see: https://keras.io/backend/). You cannot use normal lists, arrays, etc. You can transform a list or array in a tensor with `tensor = K.variable(array)` – Daniel Möller Jul 06 '17 at 12:30

How to use lambda layer in keras?

1 Answers1

Linked