How to use 1-channel images as inputs to a VGG model

Question

I first used 3-channel images as input to a VGG16 model with NO problem:

input_images = Input(shape=(img_width, img_height, 3), name='image_input')
vgg_out = base_model(input_images)  # Here base_model is a VGG16

Now I would like to use 1-channel images instead. So I did it like this:

input_images = Input(shape=(img_width, img_height, 1), name='image_input')
repeat_2 = concatenate([input_images, input_images])
repeat_3 = concatenate([repeat_2, input_images])
vgg_out = base_model(repeat_3)

But I got an error message:

File "test.py", line 423, in <module>
model = Model(inputs=[input_images], outputs=[vgg_out])
File "C:\Users\wzhou\AppData\Local\Continuum\Anaconda2\envs\tensorflow\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "C:\Users\wzhou\AppData\Local\Continuum\Anaconda2\envs\tensorflow\lib\site-packages\keras\engine\network.py", line 93, in __init__
self._init_graph_network(*args, **kwargs)
File "C:\Users\wzhou\AppData\Local\Continuum\Anaconda2\envs\tensorflow\lib\site-packages\keras\engine\network.py", line 237, in _init_graph_network
self.inputs, self.outputs)
File "C:\Users\wzhou\AppData\Local\Continuum\Anaconda2\envs\tensorflow\lib\site-packages\keras\engine\network.py", line 1430, in _map_graph_network
str(layers_with_complete_input))
ValueError: Graph disconnected: cannot obtain value for tensor Tensor("input_1:0", shape=(?, 64, 64, 3), dtype=float32) at layer "input_1". The following previous layers were accessed without issue: []

What's the correct way to turn a 1-channel image into a 3-channel one within Keras?

score 5 · Answer 1 · answered Oct 16 '20 at 13:15

I ran into a similar solution on Kaggle, but one that takes advantage of existing Keras layer classes:

from keras.applications.vgg16 import VGG16
from keras.layers import *

img_size_target = 224
img_input = Input(shape=(img_size_target, img_size_target, 1))
img_conc = Concatenate()([img_input, img_input, img_input])  
model = VGG16(input_tensor=img_conc)

The first few layers will look like this:

Model: "vgg16"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_20 (InputLayer)           [(None, 224, 224, 1) 0                                            
__________________________________________________________________________________________________
concatenate_1 (Concatenate)     (None, 224, 224, 3)  0           input_20[0][0]                   
                                                                 input_20[0][0]                   
                                                                 input_20[0][0]                   
__________________________________________________________________________________________________
block1_conv1 (Conv2D)           (None, 224, 224, 64) 1792        concatenate_1[0][0]

score 4 · Answer 2 · answered Aug 29 '18 at 04:58

Not sure why you can't define the model in your way, but the following one works. It also fixes the error that you made in your original definition, i.e. you have to normalize your input gray scale image in the right way to match the original image preprocessing used in the pretrained VGG network. Otherwise, it is meaningless to load the pretrained weights.

from keras.applications.vgg16 import VGG16
from keras.layers import *
from keras import backend as K
from keras.models import Model
import numpy as np 

class Gray2VGGInput( Layer ) :
    """Custom conversion layer
    """
    def build( self, x ) :
        self.image_mean = K.variable(value=np.array([103.939, 116.779, 123.68]).reshape([1,1,1,3]).astype('float32'), 
                                     dtype='float32', 
                                     name='imageNet_mean' )
        self.built = True
        return
    def call( self, x ) :
        rgb_x = K.concatenate( [x,x,x], axis=-1 )
        norm_x = rgb_x - self.image_mean
        return norm_x
    def compute_output_shape( self, input_shape ) :
        return input_shape[:3] + (3,)

# 1. load pretrain
backbone = VGG16(input_shape=(224,224,3) )
# 2. define gray input
gray_image = Input( shape=(224,224,1), name='gray_input' )
# 3. convert to VGG input
vgg_input_image = Gray2VGGInput( name='gray_to_rgb_norm')( gray_image )
# 4. process by pretrained VGG
pred = backbone( vgg_input_image )
# 5. define the model end-to-end
model = Model( input=gray_image, output=pred, name='my_gray_vgg' )
print model.summary()

# 6. test model
a = np.random.randint(0,255,size=(2,224,224,1))
p = model.predict(a)
print p.shape

Depending on the pretrained model you use, preprocessing steps can be different (see this for more details).

Thanks for the reply! It's very strange. I still get the same error message on my side. My TensorFlow is 1.10.0 and Keras 2.2.2. — willz, Aug 29 '18 at 21:29
My understanding of the error message is that it thinks the first layer of the vgg model is not getting a tensor of shape=(?, 64, 64, 3). Is my understanding right? — willz, Aug 29 '18 at 21:32
Partially yes. For a convolution layer, the input dimension must be a fixed number, in the case of VGG, it is 3 instead of 1. — pitfall, May 03 '19 at 19:01

How to use 1-channel images as inputs to a VGG model

2 Answers2