Keras model.predict is nearly always incorrect on training dataset; even when training it to near 100% accuracy

Question

I am trying to do multiclass classification with keras for videogame characters. The problem is that even when the training accuracy/validation accuracy hits near 100% and 70% respectively; when I actually run model.predict() on an image that I literally just trained on; it classifies it completely incorrectly. I have around 300 images per class.

I have tried everything I can think of; tried many ways to load the data initially, changed the model architecture a million times and the preprocessing functions to prepare the image that model.predict will be ran on.

Here is my code:

`class_names = ['ana', 'ashe','baby', 'ball','bap', 'bastion','brig', 'cass','doom', 'dva'
,'echo', 'genji','hanzo', 'hog','jq', 'junkrat','kiriko', 'lucio','mei', 'mercy','moira', 'orisa'
,'pharah', 'ram','reaper', 'rein','sigma', 'sojourn','soldier', 'sombra','sym', 'torb','tracer', 'widow'
,'winston', 'zarya','zen']

class_names_label = {class_name:i for i, class_name in enumerate(class_names)}
nb_classes = len(class_names)
print(class_names_label)
IMAGE_SIZE = (128,128)

def load_data():
    DIRECTORY = r'D:\crop-id\enemy-crop'
    CATEGORY = ['train', 'test']

    output = []

    for category in CATEGORY:
        path = os.path.join(DIRECTORY, category)
        print(path)
        images = []
        labels = []

        print('Loading {}'.format(category))

        for folder in os.listdir(path):
            label = class_names_label[folder]

            #iterate through each img in folder
            for file in os.listdir(os.path.join(path,folder)):

                #get path name of img
                img_path = os.path.join(os.path.join(path, folder), file)

                #open and resize img
                image = cv.imread(img_path)
                image = cv.cvtColor(image, cv.COLOR_BGR2RGB)
                image = cv.resize(image, IMAGE_SIZE)

                #append the image and its corresponding label to output
                images.append(image)
                labels.append(label)

        images = np.array(images, dtype = 'float32')
        labels = np.array(labels, dtype='int32')

        output.append((images, labels))

    return output

(train_images, train_labels), (test_images, test_labels) = load_data()

train_images, train_labels = shuffle(train_images, train_labels, random_state=25)

model = Sequential()
model.add(Conv2D(32, (3,3), activation='relu', input_shape=(128, 128, 3)))
model.add(MaxPooling2D())
model.add(BatchNormalization())

model.add(Conv2D(64, (3,3), activation='relu'))
model.add(MaxPooling2D())
model.add(BatchNormalization())

model.add(Conv2D(64, (3,3), activation='relu'))
model.add(MaxPooling2D()) 
model.add(BatchNormalization())

model.add(Flatten())
model.add(Dense(256, activation='relu'))
model.add(Dense(37, activation='softmax'))
model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.0001), loss='sparse_categorical_crossentropy', metrics=['sparse_categorical_accuracy'])
model.summary()
model.fit(train_images, train_labels, batch_size=32, epochs=10, validation_split=.2)

model.save(os.path.join('models','plzwork14.h5'))

predictions = model.predict(test_images)
pred_labels = np.argmax(predictions, axis = 1)
print(classification_report(test_labels, pred_labels))`

That is for the model and this is the preprocessing I run on the image for the model.predict():

`def prepare(img):
    img = cv.resize(img, (128,128))
    img = np.reshape(img, (128,128,3))
    img = np.expand_dims(img/255, 0)
    prediction = model.predict(img)
    prediction = prediction[0]
    print(prediction)
    print(class_names[np.argmax(prediction)])

img1 = cv.imread(r'C:\Users\andrew\Desktop\sombra.jpg')

prepare(img1)`

Thanks for your help!

Your preprocessing for the training set and prediction is completely different, not only the use of cvtColor but also your training set does not seem to be normalized by dividing with 255. — Dr. Snoopy, Feb 27 '23 at 09:25

score 1 · Answer 1 · answered Feb 27 '23 at 05:38

in the code for generation of your training and test images you have the code

image = cv.cvtColor(image, cv.COLOR_BGR2RGB)

Thus your train and test data are RGB images. However when you do your predictions you do not have this line of code so the image your trying to predict on are BGR images. So just add the code to convert from BGR to RGB for the images you want to predict

Keras model.predict is nearly always incorrect on training dataset; even when training it to near 100% accuracy

1 Answers1