Graph disconnect in inference in Keras RNN + Encoder/Decoder + Attention

Question

I've successfully trained a model in Keras using an encoder/decoder structure + attention + glove following several examples, most notably this one and this one. It's based on a modification of machine translation. This is a chatbot, so the input is words and so is the output. However, I've struggled to setup inference (prediction) properly and can't figure it out how to get past a graph disconnect. My bidirectional RNN encoder/decoder with embedding and attention is training fine. I've tried modifying the decoder, but feel there is something obvious that I'm not seeing.

Here is the basic model:

from keras.models import Model
from keras.layers.recurrent import LSTM
from keras.layers import Dense, Input, Embedding, Bidirectional, RepeatVector, concatenate, Concatenate

## PARAMETERS
HIDDEN_UNITS = 100
encoder_max_seq_length = 1037 # maximum size of input sequence
decoder_max_seq_length = 187 # maximum size of output sequence
num_encoder_tokens = 6502 # a.k.a the size of the input vocabulary
num_decoder_tokens = 4802 # a.k.a the size of the output vocabulary

## ENCODER
encoder_inputs = Input(shape=(encoder_max_seq_length, ), name='encoder_inputs')
encoder_embedding = Embedding(input_dim = num_encoder_tokens, 
                              output_dim = HIDDEN_UNITS,
                              input_length = encoder_max_seq_length,
                              weights = [embedding_matrix],
                              name='encoder_embedding')(encoder_inputs)

encoder_lstm = Bidirectional(LSTM(units = HIDDEN_UNITS,
                                  return_sequences=True,
                                  name='encoder_lstm'))(encoder_embedding)
## ATTENTION
attention = AttentionL(encoder_max_seq_length)(encoder_lstm)
attention = RepeatVector(decoder_max_seq_length)(attention)

## DECODER
decoder_inputs = Input(shape = (decoder_max_seq_length, num_decoder_tokens),
                       name='decoder_inputs')

merge = concatenate([attention, decoder_inputs])

decoder_lstm = Bidirectional(LSTM(units = HIDDEN_UNITS*2,
                    return_sequences = True,
                    name='decoder_lstm'))(merge)

decoder_dense = Dense(units=num_decoder_tokens, 
                      activation='softmax', 
                      name='decoder_dense')(decoder_lstm)

decoder_outputs = decoder_dense

## Configure the model
model = Model([encoder_inputs, decoder_inputs], decoder_outputs)
model.load_weights('trained_models/viable-2/word-weights.h5')
encoder_model = Model(encoder_inputs, attention)
model.compile(loss='categorical_crossentropy', optimizer='adam')

It looks like this: enter image description here

Here is where I run into trouble:

## INFERENCE decoder setup
decoder_inputs_2 = Concatenate()([decoder_inputs, attention])
decoder_lstm = Bidirectional(LSTM(units=HIDDEN_UNITS*2, return_state = True, name='decoder_lstm'))
decoder_outputs, forward_h, forward_c, backward_h, backward_c= decoder_lstm(decoder_inputs_2)
state_h = Concatenate()([forward_h, backward_h])
state_c = Concatenate()([forward_c, backward_c])
decoder_states = [state_h, state_c]
decoder_dense = Dense(units=num_decoder_tokens, activation='softmax', name='decoder_dense')
decoder_outputs = decoder_dense(decoder_outputs)

decoder_model = Model([decoder_inputs, attention], [decoder_outputs] + decoder_states)

This generates a graph disconnection error: Graph disconnected: cannot obtain value for tensor Tensor("encoder_inputs_61:0", shape=(?, 1037), dtype=float32) at layer "encoder_inputs". The following previous layers were accessed without issue: []

It should be possible to do inference like this, but I can't get past this error. It isn't possible for me to simply add decoder_output and attention together because they're of different shapes.

Graph disconnect in inference in Keras RNN + Encoder/Decoder + Attention

0 Answers0