0

this is actually my first post on Stackoverflow and i am pretty new to Machine Learning. I am currently trying to train a convolutional neural network for simple image classification tasks, using VGG16. It is based on this tutorial: http://www.codesofinterest.com/2017/08/bottleneck-features-multi-class-classification-keras.html. My objective is to convert this model to CoreML and use it for image classification on iOS.

After some struggle i was able to successfully convert the Keras model to a CoreML model by using following conversion lines:

coreml_model = coremltools.converters.keras.convert('Model Path',image_input_names = 'data',class_labels = 'class path')

However after copying the CoreML model into Xcode, it requests a multiarray with shape 512,1,1 ( i guess this occurs due to the image-array formatting of the neural network) There were several other posts mentioning that it is important to use specific versions of keras,tensorflow and python(2.7) mentioned here This did not change the issue.

In the Apple Developer forum they suggested to change the conversion code to following lines:

coreml_model = coremltools.converters.keras.convert('Model path',input_names='data',image_input_names = 'data',is_bgr=True,class_labels = 'Class path')

This in fact seemed to change something, however now i am not able to convert my keras model to a CoreML Model, getting this specific error:

ValueError: Channel Value 512 not supported for image inputs

Now it seems like the issue lies in the code of the convolutional neural network. Some suggested that this might occur due to dim_ordering of keras (which is last channel in my case) and/or the input shape. However, at least the input_shape looks like an image.

Anyone has an idea how to change the input from multi array to an image successfully?

Thanks in advance for any suggestion.

BraveAnt
  • 11
  • 6

2 Answers2

1

It looks like the model you have created does not accept images but a vector of 512 elements, since the first "real" layer is a Dense layer and dense (or fully-connected layers) take 1-dimensional vectors as input, not 2-dimensional images.

But there is a bigger problem here: you fine-tuned VGG16 using bottleneck features, which is OK. But to make predictions on new images you need the entire model, i.e. not just the small classifier model that you trained but also all the VGG16 layers. But you're not converting the entire model, only the classifier part.

This means your app now expects bottleneck features as input, not images. And those bottleneck features are 512-element vectors.

Matthijs Hollemans
  • 7,706
  • 2
  • 16
  • 23
  • I was looking how the input actually is shaped and you are right. However it looks like that the images are loaded as an array with shape (1,1,512) by this code: bottleneck_features_validation = model.predict_generator( generator, predict_size_validation) np.save('bottleneck_features_validation.npy', bottleneck_features_validation)) Would i have to solve this issue and/or actually change the first layer model.add(Dense(10, activation='relu')) In the case of new predictions, would i have to add the VGG16 layers and save it like before? – BraveAnt Dec 08 '17 at 17:38
  • If you want to use this model to make predictions on images on iOS, then you have to add the VGG16 layers (excluding the top layer) followed by your own classifier layers, and convert the entire thing to Core ML. – Matthijs Hollemans Dec 08 '17 at 19:24
0

I add the same problem (usin VGG16) and figured it out thanks to this post and Matthijs Hollemans answer : you have to save the whole model, including VGG16 layers.

In my code, I was only saving the top model.

Here is what I did to solve it :

# add the model on top of the convolutional base  
fullModel = Model(inputs=base_model.input, outputs=top_model(base_model.output))

with :

  • base_model being your base model (VGG16) without its top layer
  • top_model being your top model

in my case something like this :

base_model = applications.VGG16(include_top=False, weights='imagenet', input_shape=(img_width, img_height, 3))  
....  
top_model = Sequential()  top_model.add(Flatten(input_shape=train_data.shape[1:]))  
top_model.add(Dense(256, activation='relu'))  
top_model.add(Dropout(0.5))  
top_model.add(Dense(num_classes, activation='softmax'))
Nicoduj
  • 1