TF 2.0 sequential CNN into LSTM for regression "Negative dimension size" error

Question

I'm trying to build a model that predict the price of a certain commodity based on current market conditions, my data are shaped similar to

num_samples = 100
sample_dimension = 10
XXX = np.random.random((num_samples,sample_dimension)).reshape(-1,1,sample_dimension)
YYY = np.random.random(num_samples).reshape(-1,1)

so I've got 100 ordered samples of X data, each consisting of 10 variables. My model looks like the following

model = keras.Sequential()
model.add(tf.keras.layers.Conv1D(4,
                                 kernel_size = (2),
                                 activation='sigmoid',
                                 input_shape=(None, sample_dimension),
                                 batch_input_shape = [1,1,sample_dimension]))

model.add(tf.keras.layers.AveragePooling1D(pool_size=2))
model.add(tf.keras.layers.Reshape((1, sample_dimension)))
model.add(tf.keras.layers.LSTM(100,
                                    stateful = True,
                                    return_sequences=False,
                                    activation='sigmoid'))
model.add(keras.layers.Dense(1))

model.compile(optimizer='adam',
              loss='mean_squared_error',
              metrics=['accuracy'])

so it's a 1D convolution, a pooling, a reshape (so it plays nice with the lstm) and then casting down to a prediction

but when I try to run it, I get the following error

Negative dimension size caused by subtracting 2 from 1 for 'conv1d/conv1d' (op: 'Conv2D') with input shapes: [1,1,1,10], [1,2,10,4].

I've tried a few different values for the kernel size, pool size, and batch_input_shape (have to batch my inputs because my actual data are spread across several large files, so I want to read one at a time and kick it into training the model), but nothing seems to work.

What am I doing wrong? How can I track/predict the shape of my data as it goes through this model? What are the data/variables supposed to look like?

Then the only other 2 is your kernel size -- are you positive none of your inputs are shorter than 2? — Andy, Oct 25 '19 at 18:46
My input data is 100 ordered samples, each having 10 variables `XXX = np.random.random((num_samples,sample_dimension)).reshape(-1,1,sample_dimension)` — Mohammad Athar, Oct 25 '19 at 18:49

score 0 · Answer 1 · answered Oct 26 '19 at 21:01

I ended up looking through tutorials for conv2D, and then converting stuff to conv1D (please edit as you feel appropriate)

conv2D solution

model = keras.Sequential()
model.add(tf.keras.layers.Conv2D(4,
                                 kernel_size = (**1**,2),
                                 activation = 'sigmoid',
                                 input_shape = (**1**,sample_dimension,1),
                                 batch_input_shape = [None,**1**,sample_dimension,1]))

model.add(tf.keras.layers.AveragePooling2D(pool_size=(1,2)))
#model.add(tf.keras.layers.Reshape((1,sample_dimension)))
model.add(tf.keras.layers.Flatten())
model.add(keras.layers.Dense(1))

Then I converted it to conv1D by taking out a dimension from each of the necessary arguments (the bold 1s)

model = keras.Sequential()
model.add(tf.keras.layers.Conv1D(4,
                                 kernel_size = 2,
                                 activation = 'sigmoid',
                                 input_shape = (sample_dimension,1),
                                 batch_input_shape = [None,sample_dimension,1]))

model.add(tf.keras.layers.AveragePooling1D(pool_size=2))
#model.add(tf.keras.layers.Reshape((1,sample_dimension)))
model.add(tf.keras.layers.Flatten())
model.add(keras.layers.Dense(1))

i guess the key takeaway is that tensorflow isn't designed to deal with vectors or even matrices, so the last dimension has to be the dimension of the tensor- in this case, it's a 1D tensor (just a number) being held in a sample_dimension

TF 2.0 sequential CNN into LSTM for regression "Negative dimension size" error

1 Answers1