How can I add noise (jittering) to my python keras ANN to avoid overfitting?

Question

I am implementing an Artificial Neural Network model in Python Keras, and I get high accuracy in training but low accuracy for testing. This means that some overfitting is present in the data.

I would like to avoid overfitting and one of the techniques is jittering or noise addition. But, my question is: How can I do it in Python?

Here is my code for the ANN:

def designANN(input_nodes, dropout, layer_nodes, output_nodes):

    classifier = Sequential()

    classifier.add(Dense(units = layer_nodes, kernel_initializer = "uniform",
                 activation = "relu", input_dim = input_nodes)) 

    classifier.add(Dropout(dropout))

    classifier.add(Dense(units = layer_nodes, kernel_initializer = "uniform",
                 activation = "relu"))
    classifier.add(Dropout(dropout))


    classifier.add(Dense(units = output_nodes, kernel_initializer = "uniform",
                 activation = "sigmoid"))


    classifier.compile(optimizer = "adam", loss = "binary_crossentropy", metrics = [npv])

    return classifier

score 1 · Accepted Answer · answered Jun 04 '20 at 09:36

all you need is simply GaussianNoise layer. you can put it inside your network. I suggest using it before the activation function. this is the case of relu where output values may be out-of-range (<0) if we add random noise

def designANN(input_nodes, dropout, layer_nodes, output_nodes):

    classifier = Sequential()

    classifier.add(Dense(units = layer_nodes, kernel_initializer = "uniform",
                         input_dim = input_nodes))
    classifier.add(GaussianNoise(0.1))
    classifier.add(Activation('relu'))
    classifier.add(Dropout(dropout))

    classifier.add(Dense(units = layer_nodes, kernel_initializer = "uniform"))
    classifier.add(GaussianNoise(0.1))
    classifier.add(Activation('relu'))
    classifier.add(Dropout(dropout))

    classifier.add(Dense(units = output_nodes, kernel_initializer = "uniform",
                 activation = "sigmoid"))

    classifier.compile(optimizer = "adam", loss = "binary_crossentropy", metrics = [npv])

    return classifier

How can I add noise (jittering) to my python keras ANN to avoid overfitting?

1 Answers1