Seq2Seq Bidirectional Encoder Decoder in Keras

Question

I am trying to implement a seq2seq encoder-decoder using Keras, with bidirectional lstm on the encoder as follows:

from keras.layers import LSTM,Bidirectional,Input,Concatenate
from keras.models import Model

n_units = 8
n_input = 1
n_output = 1

# encoder
encoder_inputs = Input(shape=(None, n_input))
encoder = Bidirectional(LSTM(n_units, return_state=True))
encoder_outputs, forward_h, forward_c, backward_h, backward_c = encoder(encoder_inputs)
state_h = Concatenate()([forward_h, backward_h])
state_c = Concatenate()([forward_c, backward_c])
encoder_states = [state_h, state_c]

# decoder
decoder_inputs = Input(shape=(None, n_output))    
decoder_lstm = LSTM(n_units*2, return_sequences=True, return_state=True)
decoder_outputs, _, _ = decoder_lstm(decoder_inputs, initial_state=encoder_states)

Here is the following error I got on the last line:

ValueError: Dimensions must be equal, but are 8 and 16 for 
'lstm_2_1/MatMul_4' (op: 'MatMul') with input shapes: [?,8], [16,16].

Any ideas?

Your code runs on my machine without any errors (though, after creating a `Model` object and calling `compile` and `fit`). Could you post the complete code you are using? Maybe there is a problem in the parts you have not posted. — today, Jun 12 '18 at 13:37
You were right, the error pointed to the last line of this block, however the real error was propagated from another line relating to the inference decoder! Thanks, it is solved now! — terenceflow, Jun 12 '18 at 14:53

score 7 · Answer 1 · answered Jun 12 '18 at 14:57

Although the error pointed to the last line of the block in the question, however it was due to the wrong number of hidden units in the inference decoder. Solved!

Full working code:

from keras.layers import LSTM,Bidirectional,Input,Concatenate
from keras.models import Model

n_units = 8
n_input = 1
n_output = 1

# encoder
encoder_inputs = Input(shape=(None, n_input))
encoder = Bidirectional(LSTM(n_units, return_state=True))
encoder_outputs, forward_h, forward_c, backward_h, backward_c = encoder(encoder_inputs)
state_h = Concatenate()([forward_h, backward_h])
state_c = Concatenate()([forward_c, backward_c])
encoder_states = [state_h, state_c]

# decoder
decoder_inputs = Input(shape=(None, n_output))    
decoder_lstm = LSTM(n_units*2, return_sequences=True, return_state=True)
decoder_outputs, _, _ = decoder_lstm(decoder_inputs, initial_state=encoder_states)
decoder_dense = Dense(n_output, activation='softmax')
decoder_outputs = decoder_dense(decoder_outputs)
model = Model([encoder_inputs, decoder_inputs], decoder_outputs)


# define inference encoder
encoder_model = Model(encoder_inputs, encoder_states)
# define inference decoder
decoder_state_input_h = Input(shape=(n_units*2,))
decoder_state_input_c = Input(shape=(n_units*2,))
decoder_states_inputs = [decoder_state_input_h, decoder_state_input_c]
decoder_outputs, state_h, state_c = decoder_lstm(decoder_inputs, initial_state=decoder_states_inputs)
decoder_states = [state_h, state_c]
decoder_outputs = decoder_dense(decoder_outputs)
decoder_model = Model([decoder_inputs] + decoder_states_inputs, [decoder_outputs] + decoder_states)

Hi, now that you have defined your encoder and decoder models, how would you go about training them? How would you go about combining these two Keras models into a single auto-encoder model which could be trained using mode.fit()? Thanks! — StopReadingThisUsername, Dec 29 '18 at 05:05
Use this as reference https://blog.keras.io/a-ten-minute-introduction-to-sequence-to-sequence-learning-in-keras.html — terenceflow, Dec 30 '18 at 06:26

Hemanth Yernagula · Answer 2 · 2019-12-15T03:17:29.180

https://stackoverflow.com/a/50820218/10706937

from keras.layers import LSTM,Bidirectional,Input,Concatenate
from keras.models import Model

n_units = 8
n_input = 1
n_output = 1

# encoder
encoder_inputs = Input(shape=(None, n_input))
encoder = Bidirectional(LSTM(n_units, return_state=True))
encoder_outputs, forward_h, forward_c, backward_h, backward_c = encoder(encoder_inputs)
state_h = Concatenate()([forward_h, backward_h])
state_c = Concatenate()([forward_c, backward_c])
encoder_states = [state_h, state_c]

# decoder
decoder_inputs = Input(shape=(None, n_output))    
decoder_lstm = LSTM(n_units*2, return_sequences=True, return_state=True)
decoder_outputs, _, _ = decoder_lstm(decoder_inputs, initial_state=encoder_states)
decoder_dense = Dense(n_output, activation='softmax')
decoder_outputs = decoder_dense(decoder_outputs)
model = Model([encoder_inputs, decoder_inputs], decoder_outputs)


# define inference encoder
encoder_model = Model(encoder_inputs, encoder_states)
# define inference decoder
decoder_state_input_h = Input(shape=(n_units*2,))
decoder_state_input_c = Input(shape=(n_units*2,))
decoder_states_inputs = [decoder_state_input_h, decoder_state_input_c]
decoder_outputs, state_h, state_c = decoder_lstm(decoder_inputs, initial_state=decoder_states_inputs)
decoder_states = [state_h, state_c]
decoder_outputs = decoder_dense(decoder_outputs)
decoder_model = Model([decoder_inputs] + decoder_states_inputs, [decoder_outputs] + decoder_states)

How to remove inference encoder in the above comment. I mean how to solve the error without inference encoder

Seq2Seq Bidirectional Encoder Decoder in Keras

2 Answers2

Linked