2

A Keras introduction to Seq2Seq model have been published by Chollet on the Keras blog: here. One thing I do not understand is what will be the inference model for the GRU seq2seq model. The code he gave for creating the encoder-decoder is given below, however, he doesn't show how to change the inference code accordingly.

# encoder decoder model given by fchollet
encoder_inputs = Input(shape=(None, num_encoder_tokens))
encoder = GRU(latent_dim, return_state=True)
encoder_outputs, state_h = encoder(encoder_inputs)

decoder_inputs = Input(shape=(None, num_decoder_tokens))
decoder_gru = GRU(latent_dim, return_sequences=True)
decoder_outputs = decoder_gru(decoder_inputs, initial_state=state_h)
decoder_dense = Dense(num_decoder_tokens, activation='softmax')
decoder_outputs = decoder_dense(decoder_outputs)
model = Model([encoder_inputs, decoder_inputs], decoder_outputs)
model.compile(optimizer=OPTIMIZER, loss=LOSS, metrics=["accuracy"])
model.fit([encoder_input_data, decoder_input_data], decoder_target_data,
                        batch_size=BATCH_SIZE,
                        epochs=EPOCHS)

# inference model I tried making

encoder_model = Model(encoder_inputs, state_h)
decoder_state_input_h = Input(shape=(LATENT_DIMENSIONS,))
decoder_states_inputs = [decoder_state_input_h]
decoder_outputs, state_h, state_c = decoder_gru(
    decoder_inputs, initial_state=decoder_states_inputs)
decoder_states = [state_h]
decoder_outputs = decoder_dense(decoder_outputs)
decoder_model = Model(
    [decoder_inputs] + decoder_states_inputs,
    [decoder_outputs] + decoder_states)
reverse_input_char_index = dict(
    (i, char) for char, i in source_token_index.items())
reverse_target_char_index = dict(
    (i, char) for char, i in target_token_index.items())

The training portion is working fine, but the inference model is giving the error

TypeError: 'Tensor' object is not iterable.

Thanks, Soumil.

Ioannis Nasios
  • 8,292
  • 4
  • 33
  • 55
S.Mandal
  • 171
  • 2
  • 8
  • It's not entirely clear what you're asking. The inference consists of: input sequence to the encoder and any sequence to the decoder (will be ignored); the output is the decoder's output – Maxim Feb 06 '18 at 13:20
  • If you are talking about the feedback loop in the decoder it may be handled using the keras variable training_phase – WellDone2094 Feb 06 '18 at 13:22
  • 1
    In the main article, fchollet gives the code for both the LSTM encoder decoder model, along with the code for the inference model i.e for testing, but in the FAQ section, he only gives the encoder decoder code using GRU but not the code for inference model. – S.Mandal Feb 06 '18 at 18:10

0 Answers0