2

I have built a Keras ConvLSTM neural network, and I want to predict one frame ahead based on a sequence of 10 time steps:

Model:

from keras.models import Sequential
from keras.layers.convolutional import Conv3D
from keras.layers.convolutional_recurrent import ConvLSTM2D
from keras.layers.normalization import BatchNormalization
import numpy as np
import pylab as plt
from keras import layers

# We create a layer which take as input movies of shape
# (n_frames, width, height, channels) and returns a movie
# of identical shape.

model = Sequential()
model.add(ConvLSTM2D(filters=40, kernel_size=(3, 3),
                   input_shape=(None, 64, 64, 1),
                   padding='same', return_sequences=True))
model.add(BatchNormalization())

model.add(ConvLSTM2D(filters=40, kernel_size=(3, 3),
                   padding='same', return_sequences=True))
model.add(BatchNormalization())

model.add(ConvLSTM2D(filters=40, kernel_size=(3, 3),
                   padding='same', return_sequences=True))
model.add(BatchNormalization())

model.add(ConvLSTM2D(filters=40, kernel_size=(3, 3),
                   padding='same', return_sequences=True))
model.add(BatchNormalization())

model.add(Conv3D(filters=1, kernel_size=(3, 3, 3),
               activation='sigmoid',
               padding='same', data_format='channels_last'))
model.compile(loss='binary_crossentropy', optimizer='adadelta')

training:

data_train_x = data_4[0:20, 0:10, :, :, :]
data_train_y = data_4[0:20, 10:11, :, :, :]

model.fit(data_train_x, data_train_y, batch_size=10, epochs=1, 
validation_split=0.05)

and I test the model:

test_x = np.reshape(data_test_x[2,:,:,:,:], [1,10,64,64,1])
next_frame = model.predict(test_x,batch_size=1, verbose=1, steps=None)

but the problem is that 'next_frame' shape is: (1, 10, 64, 64, 1) which based on the train data it should be (1, 1, 64, 64, 1)

And this is the results of 'model.summary()':

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv_lst_m2d_1 (ConvLSTM2D)  (None, None, 64, 64, 40)  59200     
_________________________________________________________________
batch_normalization_1 (Batch (None, None, 64, 64, 40)  160       
_________________________________________________________________
conv_lst_m2d_2 (ConvLSTM2D)  (None, None, 64, 64, 40)  115360    
_________________________________________________________________
batch_normalization_2 (Batch (None, None, 64, 64, 40)  160       
_________________________________________________________________
conv_lst_m2d_3 (ConvLSTM2D)  (None, None, 64, 64, 40)  115360    
_________________________________________________________________
batch_normalization_3 (Batch (None, None, 64, 64, 40)  160       
_________________________________________________________________
conv_lst_m2d_4 (ConvLSTM2D)  (None, None, 64, 64, 40)  115360    
_________________________________________________________________
batch_normalization_4 (Batch (None, None, 64, 64, 40)  160       
_________________________________________________________________
conv3d_1 (Conv3D)            (None, None, 64, 64, 1)   1081      
=================================================================
Total params: 407,001
Trainable params: 406,681
Non-trainable params: 320
MRM
  • 1,099
  • 2
  • 12
  • 29
  • Not sure we can help much if we can see the model or a miniature version it. What does your `model.summary()` look like. – parsethis Mar 28 '18 at 21:07
  • @putonspectacles , I updated the question and added the code. – MRM Mar 28 '18 at 21:12
  • okay, also add `model.summary()` – parsethis Mar 28 '18 at 21:17
  • @putonspectacles , I added the 'model.summary()' – MRM Mar 28 '18 at 21:20
  • 1
    Why do you think the output shape would be `(1, 1, 64, 64, 1)`? There's no any pooling over axis 1 and you use same padding in `Conv3D`. The strides in `Conv3D` are also 1, by default. So the output will also have 10 time steps. – Yu-Yang Mar 29 '18 at 02:31

0 Answers0