I am trying to code seq2seq example with Tensorflow with Beam Search and Attention. Till now I have found no concrete example which explains the both the decoding and/or attention together using current tf version.
Either the previous versions have an issue with the beam search decoder which will tell you to do a batch_tile.
I have a working example of simple decoder example, which for time being is Ok but I want to use Beam Search
inference_decoder = tf.contrib.seq2seq.BeamSearchDecoder(cell = dec_cell,
embedding = embedings,
start_tokens = starting_id_vec,
end_token = word_to_id['<EOS>'],
initial_state = enc_states,#dec_cell.zero_state(config.BATCH_SIZE * config.BEAM_SIZE,tf.float32).clone(cell_state=enc_states),
beam_width = config.BEAM_SIZE,
output_layer = output_layer
)
I am getting this error
ValueError: Cannot reshape a tensor with 32768 elements to shape [64,10,512] (327680 elements) for 'decoder_1/Reshape' (op: 'Reshape') with input shapes: [64,512], [3] and with input tensors computed as partial shapes: input[1] = [64,10,512].
I believe the beam_width is adding extra dimension which changes the rank of the tensor as the code works without BeamSearch with BasicDecoder. If I set the beam_size =1 (fun sake) it throws this error
ValueError: Shape must be rank 3 but is rank 2 for 'decoder/decoder/while/BasicDecoderStep/decoder/Attention_Wrapper/concat_3' (op: 'ConcatV2') with input shapes: [64,?,512], [64,512], [].
I would be glad if anyone can help me this error. If I set the
