1

I'm setting up a model which has an image input (130,130,1) and 3 outputs each containing a (10,1) vector where softmax is applied individually.

(Inspired by J. Goodfellow, Yaroslav Bulatov, Julian Ibarz, Sacha Arnoud, and Vinay D. Shet. Multi-digit number recognition from street view imagery using deep convolutional neural networks. CoRR, abs/1312.6082, 2013. URL http://arxiv.org/abs/1312.6082 , sadly they didn't publish their network).

input = keras.layers.Input(shape=(130,130, 1)
l0 = keras.layers.Conv2D(32, (5, 5), padding="same")(input)
[conv-blocks etc]
l12 = keras.layers.Flatten()(l11)
l13 = keras.layers.Dense(4096, activation="relu")(l12)
l14 = keras.layers.Dense(4096, activation="relu")(l13)
output1 = keras.layers.Dense(10, activation="softmax")(l14)
output2 = keras.layers.Dense(10, activation="softmax")(l14)
output3 = keras.layers.Dense(10, activation="softmax")(l14)

model = keras.models.Model(inputs=input, outputs=[output1, output2, output3])
model.compile(loss=['categorical_crossentropy', 'categorical_crossentropy', 
              'categorical_crossentropy'],
              loss_weights=[1., 1., 1.],
              optimizer=optimizer,
              metrics=['accuracy'])

train_generator = train_datagen.flow(x_train,
              [[y_train[:, 0, :], y_train[:, 1, :], y_train[:, 2, :]], 
              batch_size=batch_size)

But then i'm getting: ValueError: x (images tensor) and y (labels) should have the same length. Found: x.shape = (1000, 130, 130, 1), y.shape = (3, 1000, 10)

But if I change it to:

 [same as before]
 train_generator = train_datagen.flow(x_train,
              y_train, 
              batch_size=batch_size)

Then i'm getting: ValueError: Error when checking model target: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 3 array(s)

  • dimension(x_train) = (1000, 130, 130, 1)
    • where each single image is (130, 130, 1) and there are 1000 images
  • dimension(y_train) = (1000, 3, 10)

In the documentation it is stated that it should be like that;

model = Model(inputs=[main_input, auxiliary_input], outputs= 
[main_output, auxiliary_output])

However, I don't know how you should be able to have the same length for outputs and inputs?

OnlyRightNow
  • 11
  • 2
  • I've used multi-input and multi-output models before and the first way is definitely the way to go. Make sure that the 3 `y` arrays (`y_train[:, 0, :]`, ...) all have a shape of `(1000, 10)`. Also, the way you wrote it the `train_generator` has one more `[` bracket than necessary (this would normally raise a syntax error so I guess there isn't one in your actual code. – Djib2011 Jun 23 '19 at 16:51
  • thanks @Djib2011 ,sharp eye for the bracket, you're right ;) However, as i said, the first way gives me an error because the length of the combined output `[y1, y2, y3]` has not the same length as the input, since `[y1, y2, y3]` has shape `(3,1000,10)` even though i took care that each of the y has shape `(1000, 10)` – OnlyRightNow Jun 23 '19 at 20:03
  • `[y1, y2, y3]` should be a list of 3 arrays, I'm not sure why it treats it as a *combined input*. I'd try naming the outputs if I were you (e.g. `output1 = keras.layers.Dense(10, activation="softmax", name='out1')`) and then try passing the label arrays in a dictionary: `{'out1': y1, 'out2': y2, 'out3': y3}`. Maybe this will help... There is a chance you'll have to also change `model.compile()` so that it takes dictionaries instead of lists, but I'm not sure. – Djib2011 Jun 23 '19 at 21:05

1 Answers1

0

Thanks to @Djib2011. When I looked up examples in the documentation for passing it in a dictionary, I remarked that all the examples make use of model.fit() and not model.fit_generator().

So I did research and found that there is still a bug (open since 2016!) for ImageDataGenerator with single input and multiple outputs. Sad story.

So the solution is to use model.fit() instead of model.fit_generator().

OnlyRightNow
  • 11
  • 2