Troubles Training Tensor flow Neural Network, how can I fix this problem?

Question

I am currently training an image classification model with three categories of vehicles (Vans/SUVs, Cars and Trucks). I have 1800 training images and 210 validation images. When I try to plug in the data. I pre-process data with keras.preprocessing.image.ImageDataGenerator() and Val_Data.flow(. It seems like absolutely is happening because of my accuracy staying constant. Below are my code and my results. I have tried to fix this for so long and cannot seem to fix this problem.

The Code:

    # Creating Training Data Shuffled and Organized
Train_Data = keras.preprocessing.image.ImageDataGenerator()

Train_Gen = Train_Data.flow(
        Train_Img, 
        Train_Labels,
        batch_size=BATCH_SIZE,
        shuffle=True)

# Creating Validation Data Shuffled and Organized
Val_Data = keras.preprocessing.image.ImageDataGenerator()

Val_Gen = Val_Data.flow(
        Train_Img, 
        Train_Labels,
        batch_size=BATCH_SIZE,
        shuffle=True)


print(Train_Gen)


###################################################################################
###################################################################################


#Outline the Model
hidden_layer_size = 300
output_size = 3

#Model Core
model = tf.keras.Sequential([

                             tf.keras.layers.Flatten(input_shape=(IMG_HEIGHT,IMG_WIDTH,CHANNELS)),
                             tf.keras.layers.Dense(hidden_layer_size, activation = 'relu'),
                             tf.keras.layers.Dense(hidden_layer_size, activation = 'relu'),
                             tf.keras.layers.Dense(hidden_layer_size, activation = 'relu'),
                             tf.keras.layers.Dense(hidden_layer_size, activation = 'relu'),
                             tf.keras.layers.Dense(hidden_layer_size, activation = 'relu'),
                             tf.keras.layers.Dense(output_size, activation = 'softmax')

                            ])

custom_optimizer = tf.keras.optimizers.SGD(learning_rate=0.001)

#Compile Model
model.compile(optimizer='adam', loss ='sparse_categorical_crossentropy', metrics = ['accuracy'])

#Train Model
NUM_EPOCHS = 15;
model.fit(Train_Gen, validation_steps = 10, epochs = NUM_EPOCHS, validation_data = Val_Gen, verbose = 2)

The Results:

    180/180 - 27s - loss: 10.7153 - accuracy: 0.3333 - val_loss: 10.7991 - val_accuracy: 0.3300
Epoch 2/15
180/180 - 23s - loss: 10.7454 - accuracy: 0.3333 - val_loss: 10.7991 - val_accuracy: 0.3300
Epoch 3/15
180/180 - 23s - loss: 10.7454 - accuracy: 0.3333 - val_loss: 10.7991 - val_accuracy: 0.3300
Epoch 4/15
180/180 - 22s - loss: 10.7454 - accuracy: 0.3333 - val_loss: 10.7991 - val_accuracy: 0.3300
Epoch 5/15
180/180 - 22s - loss: 10.7454 - accuracy: 0.3333 - val_loss: 10.7991 - val_accuracy: 0.3300
Epoch 6/15
180/180 - 21s - loss: 10.7454 - accuracy: 0.3333 - val_loss: 10.7991 - val_accuracy: 0.3300
Epoch 7/15
180/180 - 22s - loss: 10.7454 - accuracy: 0.3333 - val_loss: 10.7991 - val_accuracy: 0.3300
Epoch 8/15
180/180 - 22s - loss: 10.7454 - accuracy: 0.3333 - val_loss: 10.7991 - val_accuracy: 0.3300
Epoch 9/15
180/180 - 23s - loss: 10.7454 - accuracy: 0.3333 - val_loss: 10.7991 - val_accuracy: 0.3300
Epoch 10/15
180/180 - 22s - loss: 10.7454 - accuracy: 0.3333 - val_loss: 10.7991 - val_accuracy: 0.3300
Epoch 11/15
180/180 - 22s - loss: 10.7454 - accuracy: 0.3333 - val_loss: 10.7991 - val_accuracy: 0.3300
Epoch 12/15
180/180 - 22s - loss: 10.7454 - accuracy: 0.3333 - val_loss: 10.7991 - val_accuracy: 0.3300
Epoch 13/15
180/180 - 22s - loss: 10.7454 - accuracy: 0.3333 - val_loss: 10.7991 - val_accuracy: 0.3300
Epoch 14/15
180/180 - 22s - loss: 10.7454 - accuracy: 0.3333 - val_loss: 10.7991 - val_accuracy: 0.3300
Epoch 15/15
180/180 - 22s - loss: 10.7454 - accuracy: 0.3333 - val_loss: 10.7991 - val_accuracy: 0.3300

score 0 · Answer 1 · answered Apr 07 '20 at 20:19

I guess the first thing you'll have to look into are convolutional neural networks since i can see that you are trying to use a Dense network to solve an image based problem. It will work but not as good as the CNN.

There are many reasons that the models stuck in tensorflow, the most common that i have fallen into are:

there was a time that i had a model stuck because my input were not numpy arrays
the optimizer learning rate is too high..

Start by making a custom optimizer learning rate that is lower than the default, that means:

custom_optimizer = tf.keras.optimizers.Adam(lr=0.0001)
model.compile(optimizer=custom_optimizer, loss="sparse_categorical_crossentropy", metrics = ['acc'])

check this link which is a CNN implementation that is close to yours https://gist.github.com/RyanAkilos/3808c17f79e77c4117de35aa68447045

Troubles Training Tensor flow Neural Network, how can I fix this problem?

1 Answers1