1

It's my first time I work with Bidirectional layers, but I don't find any examples with the use of the functional API from Keras. mask_mastrix is a tensor with shape (samples, timesteps) with binary values True / False.

with strategy.scope():
    input_tensor = Input(shape=(timesteps, features))

    enc = Bidirectional(LSTM(timesteps * 2, activation = 'tanh', return_sequences = True))(input_tensor, mask=mask_matrix)
    enc = Bidirectional(LSTM(timesteps * 1.5, activation = 'tanh', return_sequences = False))(enc)
    
    decode1 = RepeatVector(timesteps)(enc)
    decode1 = Bidirectional(LSTM(200, activation = 'tanh', return_sequences = True))(decode1)
    decode1 = Bidirectional(LSTM(timesteps, activation = 'tanh', return_sequences = True))(decode1)
    decode1 = TimeDistributed(Dense(8, activation="softmax"), name="dec1")(decode1)

    decode2 = RepeatVector(timesteps)(enc)
    decode2 = Bidirectional(LSTM(timesteps, activation = 'tanh', return_sequences = True))(decode2)
    decode2 = TimeDistributed(Dense(2, activation = "tanh"), name="dec2")(decode2)
    
    new_model = Model(inputs=input_tensor, outputs = [decode1, decode2])
    new_model.compile(loss={"dec1":"categorical_crossentropy", "dec2":'mse'}, optimizer='adam')
    plot_model(new_model, to_file='model.png')

I get the following error:

Dimension value must be integer or None or have an __index__ method, got value '1932.0' with type '<class 'float'>'

I have two questions about this:

  1. What caused the problem and how to solve it?
  2. How does the mask value works when you use Bidirectional layers? Because from my understanding, this layer creates two LSTMs with: first with the forward sequence and the second with the backward sequence (to get the whole context). Does the mask argument also gets reversed?

Thanks in advance!

DataPlanningEngineer
  • 56
  • 6

1 Answers1

0

Change this line:

enc = Bidirectional(LSTM(timesteps * 2 // 3, activation = 'tanh', return_sequences = False))(enc)
Andrey
  • 5,932
  • 3
  • 17
  • 35
  • Thanks! That worked! Any idea for my second question to? – DataPlanningEngineer Nov 16 '20 at 11:57
  • I didn't use mask with LSTM. As far as I understand - reversing mask is the expected behaviour (for me it looks stupid to mask the first word on the forward step and the last word on backward). So it should work. – Andrey Nov 16 '20 at 12:02